[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:58:45.127089   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:38159
I20260709 12:58:45.129192   431 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:58:45.130600   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:45.153703   440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:45.154580   437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:45.156641   438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:46.434500   439 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260709 12:58:46.434613   431 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 12:58:46.439231   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:46.439504   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:46.439786   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601926439752 us; error 0 us; skew 500 ppm
I20260709 12:58:46.440747   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:46.448863   431 webserver.cc:533] Webserver started at http://127.0.107.254:36151/ using document root <none> and password file <none>
I20260709 12:58:46.450044   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:46.450291   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:46.450942   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:46.459028   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/master-0-root/instance:
uuid: "faeef2d50afb4f5a8745a7c78b38b9f1"
format_stamp: "Formatted at 2026-07-09 12:58:46 on dist-test-slave-l8g5"
I20260709 12:58:46.469906   431 fs_manager.cc:696] Time spent creating directory manager: real 0.010s	user 0.011s	sys 0.000s
I20260709 12:58:46.476645   446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:46.478161   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:58:46.478592   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/master-0-root
uuid: "faeef2d50afb4f5a8745a7c78b38b9f1"
format_stamp: "Formatted at 2026-07-09 12:58:46 on dist-test-slave-l8g5"
I20260709 12:58:46.478941   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:46.565828   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:46.568427   431 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:58:46.569018   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:46.676981   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:38159
I20260709 12:58:46.677052   497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:38159 every 8 connection(s)
I20260709 12:58:46.687845   498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:46.717870   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1: Bootstrap starting.
I20260709 12:58:46.726186   498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:46.728598   498 log.cc:826] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1: Log is configured to *not* fsync() on all Append() calls
I20260709 12:58:46.735383   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1: No bootstrap required, opened a new log
I20260709 12:58:46.763199   498 raft_consensus.cc:359] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeef2d50afb4f5a8745a7c78b38b9f1" member_type: VOTER }
I20260709 12:58:46.763952   498 raft_consensus.cc:385] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:46.764197   498 raft_consensus.cc:740] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faeef2d50afb4f5a8745a7c78b38b9f1, State: Initialized, Role: FOLLOWER
I20260709 12:58:46.766198   498 consensus_queue.cc:260] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeef2d50afb4f5a8745a7c78b38b9f1" member_type: VOTER }
I20260709 12:58:46.766899   498 raft_consensus.cc:399] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:58:46.767202   498 raft_consensus.cc:493] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:58:46.767598   498 raft_consensus.cc:3060] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:46.772902   498 raft_consensus.cc:515] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeef2d50afb4f5a8745a7c78b38b9f1" member_type: VOTER }
I20260709 12:58:46.774731   498 leader_election.cc:304] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faeef2d50afb4f5a8745a7c78b38b9f1; no voters: 
I20260709 12:58:46.777925   498 leader_election.cc:290] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:58:46.778234   501 raft_consensus.cc:2804] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:46.782421   501 raft_consensus.cc:697] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [term 1 LEADER]: Becoming Leader. State: Replica: faeef2d50afb4f5a8745a7c78b38b9f1, State: Running, Role: LEADER
I20260709 12:58:46.783555   501 consensus_queue.cc:237] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeef2d50afb4f5a8745a7c78b38b9f1" member_type: VOTER }
I20260709 12:58:46.785567   498 sys_catalog.cc:565] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:58:46.801503   502 sys_catalog.cc:455] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faeef2d50afb4f5a8745a7c78b38b9f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeef2d50afb4f5a8745a7c78b38b9f1" member_type: VOTER } }
I20260709 12:58:46.802264   502 sys_catalog.cc:458] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:58:46.805162   503 sys_catalog.cc:455] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faeef2d50afb4f5a8745a7c78b38b9f1. Latest consensus state: current_term: 1 leader_uuid: "faeef2d50afb4f5a8745a7c78b38b9f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faeef2d50afb4f5a8745a7c78b38b9f1" member_type: VOTER } }
I20260709 12:58:46.806068   503 sys_catalog.cc:458] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:58:46.807245   510 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:58:46.825246   510 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:58:46.828943   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:58:46.848225   510 catalog_manager.cc:1383] Generated new cluster ID: 23a0d707296042edbd4bf905f46f051c
I20260709 12:58:46.848611   510 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:58:46.875808   510 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:58:46.878985   510 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:58:46.898356   510 catalog_manager.cc:6070] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1: Generated new TSK 0
I20260709 12:58:46.899611   510 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:58:46.960880   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:46.973793   520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:46.976526   521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:46.980829   523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:46.980922   431 server_base.cc:1061] running on GCE node
I20260709 12:58:46.982299   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:46.982625   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:46.982860   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601926982837 us; error 0 us; skew 500 ppm
I20260709 12:58:46.983780   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:46.990005   431 webserver.cc:533] Webserver started at http://127.0.107.193:40571/ using document root <none> and password file <none>
I20260709 12:58:46.991000   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:46.991330   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:46.991714   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:46.993561   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-0-root/instance:
uuid: "d7f1c894cd2c4c11b36b837667423755"
format_stamp: "Formatted at 2026-07-09 12:58:46 on dist-test-slave-l8g5"
I20260709 12:58:47.002416   431 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:58:47.008526   528 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:47.010238   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 12:58:47.010643   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-0-root
uuid: "d7f1c894cd2c4c11b36b837667423755"
format_stamp: "Formatted at 2026-07-09 12:58:46 on dist-test-slave-l8g5"
I20260709 12:58:47.011073   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:47.046804   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:47.049212   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:47.053558   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:47.061259   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:47.061671   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:58:47.062001   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:47.062240   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:47.191596   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:45333
I20260709 12:58:47.191655   590 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:45333 every 8 connection(s)
I20260709 12:58:47.201233   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:47.221572   596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:47.221539   595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:47.229081   598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:47.230043   431 server_base.cc:1061] running on GCE node
I20260709 12:58:47.231211   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:47.231506   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:47.231735   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601927231717 us; error 0 us; skew 500 ppm
I20260709 12:58:47.233388   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:47.237725   431 webserver.cc:533] Webserver started at http://127.0.107.194:46201/ using document root <none> and password file <none>
I20260709 12:58:47.238530   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:47.238814   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:47.239197   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:47.240556   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-1-root/instance:
uuid: "8300451f93f44610a0778716b5890b71"
format_stamp: "Formatted at 2026-07-09 12:58:47 on dist-test-slave-l8g5"
I20260709 12:58:47.247149   591 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38159
I20260709 12:58:47.247761   591 heartbeater.cc:461] Registering TS with master...
I20260709 12:58:47.248903   591 heartbeater.cc:507] Master 127.0.107.254:38159 requested a full tablet report, sending...
I20260709 12:58:47.249853   431 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.007s	sys 0.002s
I20260709 12:58:47.252970   463 ts_manager.cc:194] Registered new tserver with Master: d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333)
I20260709 12:58:47.256369   603 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:47.256883   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52108
I20260709 12:58:47.257421   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 12:58:47.257834   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-1-root
uuid: "8300451f93f44610a0778716b5890b71"
format_stamp: "Formatted at 2026-07-09 12:58:47 on dist-test-slave-l8g5"
I20260709 12:58:47.258230   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:47.278610   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:47.281415   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:47.284369   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:47.289572   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:47.289877   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:47.290172   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:47.290390   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:47.413111   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:41795
I20260709 12:58:47.414780   665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:41795 every 8 connection(s)
I20260709 12:58:47.422118   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:47.432876   669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:47.435487   670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:47.442060   666 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38159
I20260709 12:58:47.442800   666 heartbeater.cc:461] Registering TS with master...
I20260709 12:58:47.444523   666 heartbeater.cc:507] Master 127.0.107.254:38159 requested a full tablet report, sending...
I20260709 12:58:47.448428   463 ts_manager.cc:194] Registered new tserver with Master: 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:47.448959   431 server_base.cc:1061] running on GCE node
W20260709 12:58:47.449533   672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:47.450924   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:58:47.450922   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52118
W20260709 12:58:47.451398   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:47.451648   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601927451627 us; error 0 us; skew 500 ppm
I20260709 12:58:47.452595   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:47.460350   431 webserver.cc:533] Webserver started at http://127.0.107.195:43211/ using document root <none> and password file <none>
I20260709 12:58:47.461251   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:47.461730   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:47.462131   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:47.463505   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-2-root/instance:
uuid: "8f14acb5606549a4ac96fa14aacc9824"
format_stamp: "Formatted at 2026-07-09 12:58:47 on dist-test-slave-l8g5"
I20260709 12:58:47.469306   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 12:58:47.473835   677 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:47.474826   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:58:47.475240   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-2-root
uuid: "8f14acb5606549a4ac96fa14aacc9824"
format_stamp: "Formatted at 2026-07-09 12:58:47 on dist-test-slave-l8g5"
I20260709 12:58:47.475636   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601924925497-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:47.504463   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:47.506398   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:47.508389   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:47.514333   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:47.514693   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:47.515030   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:47.515447   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:47.643971   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:35727
I20260709 12:58:47.644019   739 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:35727 every 8 connection(s)
I20260709 12:58:47.666692   740 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38159
I20260709 12:58:47.667119   740 heartbeater.cc:461] Registering TS with master...
I20260709 12:58:47.668136   740 heartbeater.cc:507] Master 127.0.107.254:38159 requested a full tablet report, sending...
I20260709 12:58:47.671470   463 ts_manager.cc:194] Registered new tserver with Master: 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:47.672253   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.021023584s
I20260709 12:58:47.673391   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52122
I20260709 12:58:47.747671   463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52132:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:58:47.752408   463 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 12:58:47.811219   511 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:58:47.892866   704 tablet_service.cc:1511] Processing CreateTablet for tablet caa377949276484ea703e81b235ee2cf (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:58:47.893493   705 tablet_service.cc:1511] Processing CreateTablet for tablet 228000099d3449dcb36878d71bcdbe86 (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:58:47.900082   705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 228000099d3449dcb36878d71bcdbe86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:47.901507   555 tablet_service.cc:1511] Processing CreateTablet for tablet caa377949276484ea703e81b235ee2cf (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:58:47.902951   554 tablet_service.cc:1511] Processing CreateTablet for tablet 5e9f83b7307c45fda8df8c9fbc844c4a (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:58:47.907821   554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e9f83b7307c45fda8df8c9fbc844c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:47.900791   556 tablet_service.cc:1511] Processing CreateTablet for tablet 228000099d3449dcb36878d71bcdbe86 (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:58:47.913962   630 tablet_service.cc:1511] Processing CreateTablet for tablet caa377949276484ea703e81b235ee2cf (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:58:47.914539   704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caa377949276484ea703e81b235ee2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:47.913182   703 tablet_service.cc:1511] Processing CreateTablet for tablet 5e9f83b7307c45fda8df8c9fbc844c4a (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:58:47.911399   631 tablet_service.cc:1511] Processing CreateTablet for tablet 228000099d3449dcb36878d71bcdbe86 (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:58:47.917131   630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caa377949276484ea703e81b235ee2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:47.914669   555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caa377949276484ea703e81b235ee2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:47.918179   703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e9f83b7307c45fda8df8c9fbc844c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:47.917868   629 tablet_service.cc:1511] Processing CreateTablet for tablet 5e9f83b7307c45fda8df8c9fbc844c4a (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:58:47.920734   631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 228000099d3449dcb36878d71bcdbe86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:47.928463   629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e9f83b7307c45fda8df8c9fbc844c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:47.929804   556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 228000099d3449dcb36878d71bcdbe86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:47.934593   553 tablet_service.cc:1511] Processing CreateTablet for tablet b10c7df448e04f738e1571969c80925d (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:58:47.937877   553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10c7df448e04f738e1571969c80925d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:47.953246   628 tablet_service.cc:1511] Processing CreateTablet for tablet b10c7df448e04f738e1571969c80925d (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:58:47.956264   628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10c7df448e04f738e1571969c80925d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:47.954536   702 tablet_service.cc:1511] Processing CreateTablet for tablet b10c7df448e04f738e1571969c80925d (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:58:47.960297   702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10c7df448e04f738e1571969c80925d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.002679   552 tablet_service.cc:1511] Processing CreateTablet for tablet b495693a89fb4dddbd68886b3f62f833 (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:58:48.010473   627 tablet_service.cc:1511] Processing CreateTablet for tablet b495693a89fb4dddbd68886b3f62f833 (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:58:48.008285   552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b495693a89fb4dddbd68886b3f62f833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.015597   627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b495693a89fb4dddbd68886b3f62f833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.055962   758 tablet_bootstrap.cc:492] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824: Bootstrap starting.
I20260709 12:58:48.063992   759 tablet_bootstrap.cc:492] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755: Bootstrap starting.
I20260709 12:58:48.065927   702 tablet_service.cc:1511] Processing CreateTablet for tablet b495693a89fb4dddbd68886b3f62f833 (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:58:48.067813   702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b495693a89fb4dddbd68886b3f62f833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.079840   551 tablet_service.cc:1511] Processing CreateTablet for tablet 3b383c50beef4a54b0efa339dd0ff48e (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:58:48.088459   551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b383c50beef4a54b0efa339dd0ff48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.162824   626 tablet_service.cc:1511] Processing CreateTablet for tablet 3b383c50beef4a54b0efa339dd0ff48e (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:58:48.166043   626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b383c50beef4a54b0efa339dd0ff48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.177865   704 tablet_service.cc:1511] Processing CreateTablet for tablet 3b383c50beef4a54b0efa339dd0ff48e (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:58:48.187556   704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b383c50beef4a54b0efa339dd0ff48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.185804   759 tablet_bootstrap.cc:654] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.180661   760 tablet_bootstrap.cc:492] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71: Bootstrap starting.
I20260709 12:58:48.196619   552 tablet_service.cc:1511] Processing CreateTablet for tablet f13212b54b914d63b0f1852edf096eba (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:58:48.197656   703 tablet_service.cc:1511] Processing CreateTablet for tablet f13212b54b914d63b0f1852edf096eba (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:58:48.198303   552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f13212b54b914d63b0f1852edf096eba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.199033   703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f13212b54b914d63b0f1852edf096eba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.200078   553 tablet_service.cc:1511] Processing CreateTablet for tablet 106212907bb64e75a6a5204fae36c61f (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:58:48.201463   553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 106212907bb64e75a6a5204fae36c61f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.204526   630 tablet_service.cc:1511] Processing CreateTablet for tablet f13212b54b914d63b0f1852edf096eba (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:58:48.205668   758 tablet_bootstrap.cc:654] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.205930   630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f13212b54b914d63b0f1852edf096eba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.225178   760 tablet_bootstrap.cc:654] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.235134   759 tablet_bootstrap.cc:492] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755: No bootstrap required, opened a new log
I20260709 12:58:48.235772   759 ts_tablet_manager.cc:1403] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755: Time spent bootstrapping tablet: real 0.173s	user 0.009s	sys 0.011s
I20260709 12:58:48.240198   627 tablet_service.cc:1511] Processing CreateTablet for tablet 106212907bb64e75a6a5204fae36c61f (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:58:48.243642   627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 106212907bb64e75a6a5204fae36c61f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.248523   702 tablet_service.cc:1511] Processing CreateTablet for tablet 106212907bb64e75a6a5204fae36c61f (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:58:48.252262   702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 106212907bb64e75a6a5204fae36c61f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.261688   591 heartbeater.cc:499] Master 127.0.107.254:38159 was elected leader, sending a full tablet report...
I20260709 12:58:48.264459   759 raft_consensus.cc:359] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.266557   758 tablet_bootstrap.cc:492] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824: No bootstrap required, opened a new log
I20260709 12:58:48.268536   758 ts_tablet_manager.cc:1403] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824: Time spent bootstrapping tablet: real 0.214s	user 0.030s	sys 0.046s
I20260709 12:58:48.260229   626 tablet_service.cc:1511] Processing CreateTablet for tablet 9043d0fea676404f9ce2edb2b42132bc (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:58:48.269013   552 tablet_service.cc:1511] Processing CreateTablet for tablet 9043d0fea676404f9ce2edb2b42132bc (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:58:48.276371   758 raft_consensus.cc:359] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.277249   758 raft_consensus.cc:385] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.278451   758 raft_consensus.cc:740] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.279372   758 consensus_queue.cc:260] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.269516   759 raft_consensus.cc:385] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.280697   759 raft_consensus.cc:740] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.281042   552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9043d0fea676404f9ce2edb2b42132bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.283010   759 consensus_queue.cc:260] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.288367   626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9043d0fea676404f9ce2edb2b42132bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.288834   760 tablet_bootstrap.cc:492] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71: No bootstrap required, opened a new log
I20260709 12:58:48.289379   760 ts_tablet_manager.cc:1403] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71: Time spent bootstrapping tablet: real 0.111s	user 0.019s	sys 0.009s
I20260709 12:58:48.291208   703 tablet_service.cc:1511] Processing CreateTablet for tablet 9043d0fea676404f9ce2edb2b42132bc (DEFAULT_TABLE table=test-workload [id=271f770ba5ed43219bf488b6e816c6d6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:58:48.293073   703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9043d0fea676404f9ce2edb2b42132bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:48.298913   760 raft_consensus.cc:359] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:48.300195   760 raft_consensus.cc:385] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.300627   760 raft_consensus.cc:740] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.302385   760 consensus_queue.cc:260] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:48.321210   758 ts_tablet_manager.cc:1434] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.010s
I20260709 12:58:48.332449   740 heartbeater.cc:499] Master 127.0.107.254:38159 was elected leader, sending a full tablet report...
I20260709 12:58:48.333037   759 ts_tablet_manager.cc:1434] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755: Time spent starting tablet: real 0.097s	user 0.017s	sys 0.005s
I20260709 12:58:48.334293   759 tablet_bootstrap.cc:492] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755: Bootstrap starting.
I20260709 12:58:48.339085   758 tablet_bootstrap.cc:492] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824: Bootstrap starting.
I20260709 12:58:48.339417   760 ts_tablet_manager.cc:1434] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71: Time spent starting tablet: real 0.050s	user 0.015s	sys 0.007s
I20260709 12:58:48.345808   666 heartbeater.cc:499] Master 127.0.107.254:38159 was elected leader, sending a full tablet report...
I20260709 12:58:48.346138   760 tablet_bootstrap.cc:492] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71: Bootstrap starting.
I20260709 12:58:48.358654   759 tablet_bootstrap.cc:654] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.361140   758 tablet_bootstrap.cc:654] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.363152   760 tablet_bootstrap.cc:654] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.379025   760 tablet_bootstrap.cc:492] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71: No bootstrap required, opened a new log
I20260709 12:58:48.382650   758 tablet_bootstrap.cc:492] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824: No bootstrap required, opened a new log
I20260709 12:58:48.383831   758 ts_tablet_manager.cc:1403] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824: Time spent bootstrapping tablet: real 0.045s	user 0.007s	sys 0.010s
I20260709 12:58:48.383258   760 ts_tablet_manager.cc:1403] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.012s
I20260709 12:58:48.389933   758 raft_consensus.cc:359] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:48.390769   758 raft_consensus.cc:385] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.391398   758 raft_consensus.cc:740] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.390635   760 raft_consensus.cc:359] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.392020   760 raft_consensus.cc:385] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.392370   758 consensus_queue.cc:260] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:48.392904   760 raft_consensus.cc:740] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.394052   760 consensus_queue.cc:260] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.396591   760 ts_tablet_manager.cc:1434] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:58:48.397763   760 tablet_bootstrap.cc:492] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71: Bootstrap starting.
I20260709 12:58:48.402683   758 ts_tablet_manager.cc:1434] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 12:58:48.403731   758 tablet_bootstrap.cc:492] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824: Bootstrap starting.
W20260709 12:58:48.406989   741 tablet.cc:2406] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:58:48.413210   766 raft_consensus.cc:493] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:48.414019   766 raft_consensus.cc:515] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:48.415856   760 tablet_bootstrap.cc:654] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.421962   766 leader_election.cc:290] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727), d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333)
I20260709 12:58:48.422701   759 tablet_bootstrap.cc:492] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755: No bootstrap required, opened a new log
I20260709 12:58:48.423216   759 ts_tablet_manager.cc:1403] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755: Time spent bootstrapping tablet: real 0.089s	user 0.039s	sys 0.018s
I20260709 12:58:48.427590   758 tablet_bootstrap.cc:654] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.436126   766 raft_consensus.cc:493] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:48.436774   766 raft_consensus.cc:515] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.440685   766 leader_election.cc:290] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:48.441581   759 raft_consensus.cc:359] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.442443   759 raft_consensus.cc:385] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.442792   759 raft_consensus.cc:740] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.448108   760 tablet_bootstrap.cc:492] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71: No bootstrap required, opened a new log
I20260709 12:58:48.446532   759 consensus_queue.cc:260] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.449561   760 ts_tablet_manager.cc:1403] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.003s
I20260709 12:58:48.452605   759 ts_tablet_manager.cc:1434] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755: Time spent starting tablet: real 0.029s	user 0.010s	sys 0.000s
I20260709 12:58:48.453734   759 tablet_bootstrap.cc:492] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755: Bootstrap starting.
I20260709 12:58:48.466223   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755" is_pre_election: true
I20260709 12:58:48.466943   758 tablet_bootstrap.cc:492] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824: No bootstrap required, opened a new log
I20260709 12:58:48.466794   760 raft_consensus.cc:359] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.467626   758 ts_tablet_manager.cc:1403] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824: Time spent bootstrapping tablet: real 0.064s	user 0.010s	sys 0.016s
I20260709 12:58:48.468494   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa377949276484ea703e81b235ee2cf" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755" is_pre_election: true
I20260709 12:58:48.468116   760 raft_consensus.cc:385] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.470047   760 raft_consensus.cc:740] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Initialized, Role: FOLLOWER
W20260709 12:58:48.470643   607 leader_election.cc:343] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:48.471194   759 tablet_bootstrap.cc:654] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.472052   760 consensus_queue.cc:260] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
W20260709 12:58:48.473562   607 leader_election.cc:343] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:48.475009   758 raft_consensus.cc:359] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.477172   758 raft_consensus.cc:385] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.477682   758 raft_consensus.cc:740] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.478538   758 consensus_queue.cc:260] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.480554   760 ts_tablet_manager.cc:1434] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.002s
I20260709 12:58:48.481765   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa377949276484ea703e81b235ee2cf" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:48.482656   760 tablet_bootstrap.cc:492] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71: Bootstrap starting.
I20260709 12:58:48.482733   714 raft_consensus.cc:2468] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 0.
I20260709 12:58:48.485108   605 leader_election.cc:304] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:48.485713   758 ts_tablet_manager.cc:1434] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.006s
I20260709 12:58:48.487654   766 raft_consensus.cc:2804] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:48.487811   764 raft_consensus.cc:493] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:48.488106   766 raft_consensus.cc:493] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:48.488833   766 raft_consensus.cc:3060] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.490164   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:48.490737   758 tablet_bootstrap.cc:492] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824: Bootstrap starting.
I20260709 12:58:48.488703   764 raft_consensus.cc:515] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:48.498152   715 raft_consensus.cc:2468] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 0.
I20260709 12:58:48.500779   765 raft_consensus.cc:493] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:48.501408   765 raft_consensus.cc:515] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.509840   764 leader_election.cc:290] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:48.511047   765 leader_election.cc:290] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:48.544891   766 raft_consensus.cc:515] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.548657   766 leader_election.cc:290] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:48.551049   758 tablet_bootstrap.cc:654] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.557447   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa377949276484ea703e81b235ee2cf" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
W20260709 12:58:48.559073   607 leader_election.cc:343] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:48.559477   760 tablet_bootstrap.cc:654] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.566671   759 tablet_bootstrap.cc:492] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755: No bootstrap required, opened a new log
I20260709 12:58:48.585846   764 raft_consensus.cc:493] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:48.586874   764 raft_consensus.cc:515] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.591400   764 leader_election.cc:290] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:48.593055   759 ts_tablet_manager.cc:1403] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755: Time spent bootstrapping tablet: real 0.140s	user 0.019s	sys 0.020s
I20260709 12:58:48.615646   605 leader_election.cc:304] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:48.628278   766 raft_consensus.cc:2804] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:48.628737   766 raft_consensus.cc:493] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:48.630414   765 raft_consensus.cc:493] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:48.631590   766 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.631780   765 raft_consensus.cc:515] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.633093   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10c7df448e04f738e1571969c80925d" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755" is_pre_election: true
I20260709 12:58:48.630410   759 raft_consensus.cc:359] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.634683   759 raft_consensus.cc:385] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.635023   759 raft_consensus.cc:740] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.636129   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755" is_pre_election: true
I20260709 12:58:48.627367   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa377949276484ea703e81b235ee2cf" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:48.628158   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71" is_pre_election: true
I20260709 12:58:48.639555   765 leader_election.cc:290] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:48.640635   715 raft_consensus.cc:3060] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.639308   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b495693a89fb4dddbd68886b3f62f833" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71" is_pre_election: true
I20260709 12:58:48.640937   641 raft_consensus.cc:2468] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 0.
W20260709 12:58:48.641686   681 leader_election.cc:343] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:48.636878   759 consensus_queue.cc:260] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
W20260709 12:58:48.649428   532 leader_election.cc:343] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:58:48.651062   681 leader_election.cc:343] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:48.654194   532 leader_election.cc:304] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:48.655961   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:48.659411   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b495693a89fb4dddbd68886b3f62f833" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:48.660372   765 raft_consensus.cc:2804] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:48.662319   765 raft_consensus.cc:493] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:48.663110   758 tablet_bootstrap.cc:492] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824: No bootstrap required, opened a new log
I20260709 12:58:48.663614   758 ts_tablet_manager.cc:1403] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824: Time spent bootstrapping tablet: real 0.173s	user 0.019s	sys 0.010s
I20260709 12:58:48.667264   758 raft_consensus.cc:359] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.667811   758 raft_consensus.cc:385] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.667997   758 raft_consensus.cc:740] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.668721   758 consensus_queue.cc:260] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
W20260709 12:58:48.661419   530 leader_election.cc:343] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:48.665030   766 raft_consensus.cc:515] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
W20260709 12:58:48.670423   530 leader_election.cc:343] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:48.663105   765 raft_consensus.cc:3060] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.659068   759 ts_tablet_manager.cc:1434] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755: Time spent starting tablet: real 0.058s	user 0.009s	sys 0.003s
I20260709 12:58:48.673131   530 leader_election.cc:304] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7f1c894cd2c4c11b36b837667423755; no voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824
I20260709 12:58:48.675766   715 raft_consensus.cc:2468] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 1.
I20260709 12:58:48.682583   605 leader_election.cc:304] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:48.686486   758 ts_tablet_manager.cc:1434] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:58:48.686985   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:48.687745   758 tablet_bootstrap.cc:492] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824: Bootstrap starting.
W20260709 12:58:48.688668   607 leader_election.cc:343] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:48.688635   760 tablet_bootstrap.cc:492] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71: No bootstrap required, opened a new log
I20260709 12:58:48.691079   760 ts_tablet_manager.cc:1403] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71: Time spent bootstrapping tablet: real 0.209s	user 0.027s	sys 0.024s
I20260709 12:58:48.684077   766 leader_election.cc:290] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 election: Requested vote from peers 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727), d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333)
I20260709 12:58:48.698297   759 tablet_bootstrap.cc:492] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755: Bootstrap starting.
I20260709 12:58:48.708341   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:48.700580   766 raft_consensus.cc:2804] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:48.709108   715 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.709514   785 raft_consensus.cc:2749] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:48.712450   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71" is_pre_election: true
I20260709 12:58:48.709867   766 raft_consensus.cc:697] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Becoming Leader. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:48.716295   641 raft_consensus.cc:2393] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f14acb5606549a4ac96fa14aacc9824 in current term 1: Already voted for candidate 8300451f93f44610a0778716b5890b71 in this term.
I20260709 12:58:48.715217   760 raft_consensus.cc:359] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.716820   760 raft_consensus.cc:385] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.721582   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10c7df448e04f738e1571969c80925d" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71" is_pre_election: true
I20260709 12:58:48.721863   681 leader_election.cc:304] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824; no voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:48.721575   785 raft_consensus.cc:493] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:48.717279   760 raft_consensus.cc:740] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.727597   758 tablet_bootstrap.cc:654] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.722997   785 raft_consensus.cc:515] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.726109   766 consensus_queue.cc:237] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.727404   760 consensus_queue.cc:260] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.733018   641 raft_consensus.cc:2468] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 0.
I20260709 12:58:48.733685   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa377949276484ea703e81b235ee2cf" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71" is_pre_election: true
I20260709 12:58:48.737054   765 raft_consensus.cc:515] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.737963   760 ts_tablet_manager.cc:1434] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.002s
I20260709 12:58:48.738175   681 leader_election.cc:304] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:48.739161   760 tablet_bootstrap.cc:492] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71: Bootstrap starting.
I20260709 12:58:48.740222   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa377949276484ea703e81b235ee2cf" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:48.740921   715 raft_consensus.cc:2468] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 1.
I20260709 12:58:48.741959   713 raft_consensus.cc:2393] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7f1c894cd2c4c11b36b837667423755 in current term 1: Already voted for candidate 8300451f93f44610a0778716b5890b71 in this term.
I20260709 12:58:48.742215   764 raft_consensus.cc:2749] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:48.743666   530 leader_election.cc:304] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7f1c894cd2c4c11b36b837667423755; no voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824
I20260709 12:58:48.746286   785 leader_election.cc:290] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:48.746472   787 raft_consensus.cc:3060] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.745523   764 raft_consensus.cc:2804] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:48.759402   764 raft_consensus.cc:493] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:48.765112   759 tablet_bootstrap.cc:654] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.767375   765 leader_election.cc:290] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 election: Requested vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:48.763685   764 raft_consensus.cc:3060] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.769513   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:48.770220   640 raft_consensus.cc:3060] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.772667   605 leader_election.cc:304] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:48.779417   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:48.780181   715 raft_consensus.cc:3060] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.785022   640 raft_consensus.cc:2468] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 1.
I20260709 12:58:48.789095   532 leader_election.cc:304] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:48.791105   765 raft_consensus.cc:2804] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:48.791662   765 raft_consensus.cc:697] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Becoming Leader. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:48.792619   765 consensus_queue.cc:237] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.780860   766 raft_consensus.cc:2804] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:48.794157   760 tablet_bootstrap.cc:654] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.796624   766 raft_consensus.cc:697] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Becoming Leader. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:48.786329   462 catalog_manager.cc:5697] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8300451f93f44610a0778716b5890b71 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8300451f93f44610a0778716b5890b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:48.798375   787 raft_consensus.cc:2749] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:48.799347   759 tablet_bootstrap.cc:492] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755: No bootstrap required, opened a new log
I20260709 12:58:48.801858   766 consensus_queue.cc:237] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:48.803445   715 raft_consensus.cc:2468] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 1.
I20260709 12:58:48.786602   764 raft_consensus.cc:515] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.807951   759 ts_tablet_manager.cc:1403] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755: Time spent bootstrapping tablet: real 0.117s	user 0.009s	sys 0.013s
W20260709 12:58:48.820495   511 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:58:48.823108   758 tablet_bootstrap.cc:492] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824: No bootstrap required, opened a new log
I20260709 12:58:48.824190   758 ts_tablet_manager.cc:1403] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824: Time spent bootstrapping tablet: real 0.137s	user 0.013s	sys 0.015s
I20260709 12:58:48.828552   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10c7df448e04f738e1571969c80925d" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:48.830806   764 leader_election.cc:290] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:48.832448   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10c7df448e04f738e1571969c80925d" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:48.833248   640 raft_consensus.cc:3060] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:58:48.837846   681 leader_election.cc:343] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:48.838409   760 tablet_bootstrap.cc:492] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71: No bootstrap required, opened a new log
I20260709 12:58:48.838960   760 ts_tablet_manager.cc:1403] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71: Time spent bootstrapping tablet: real 0.100s	user 0.021s	sys 0.005s
I20260709 12:58:48.843140   640 raft_consensus.cc:2468] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 1.
I20260709 12:58:48.843605   760 raft_consensus.cc:359] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.844436   760 raft_consensus.cc:385] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.844563   681 leader_election.cc:304] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:48.844786   759 raft_consensus.cc:359] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.845543   759 raft_consensus.cc:385] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.845854   759 raft_consensus.cc:740] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.846820   759 consensus_queue.cc:260] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.847930   758 raft_consensus.cc:359] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.848733   758 raft_consensus.cc:385] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.849102   758 raft_consensus.cc:740] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.846058   764 raft_consensus.cc:2804] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:48.849918   759 ts_tablet_manager.cc:1434] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.000s
I20260709 12:58:48.850862   764 raft_consensus.cc:697] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 1 LEADER]: Becoming Leader. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Running, Role: LEADER
I20260709 12:58:48.851135   759 tablet_bootstrap.cc:492] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755: Bootstrap starting.
I20260709 12:58:48.851919   764 consensus_queue.cc:237] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.854293   758 consensus_queue.cc:260] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.844800   760 raft_consensus.cc:740] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.861887   760 consensus_queue.cc:260] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.866909   758 ts_tablet_manager.cc:1434] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824: Time spent starting tablet: real 0.042s	user 0.010s	sys 0.001s
I20260709 12:58:48.867925   758 tablet_bootstrap.cc:492] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824: Bootstrap starting.
I20260709 12:58:48.886023   758 tablet_bootstrap.cc:654] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.886703   760 ts_tablet_manager.cc:1434] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71: Time spent starting tablet: real 0.047s	user 0.013s	sys 0.000s
I20260709 12:58:48.884200   461 catalog_manager.cc:5697] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7f1c894cd2c4c11b36b837667423755 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d7f1c894cd2c4c11b36b837667423755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:48.891682   460 catalog_manager.cc:5697] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8f14acb5606549a4ac96fa14aacc9824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:48.894613   760 tablet_bootstrap.cc:492] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71: Bootstrap starting.
I20260709 12:58:48.896916   759 tablet_bootstrap.cc:654] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.917068   766 raft_consensus.cc:493] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:48.917969   766 raft_consensus.cc:515] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.925765   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b495693a89fb4dddbd68886b3f62f833" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755" is_pre_election: true
I20260709 12:58:48.926573   565 raft_consensus.cc:2468] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 0.
I20260709 12:58:48.927284   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b495693a89fb4dddbd68886b3f62f833" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:48.929525   766 leader_election.cc:290] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:48.935501   760 tablet_bootstrap.cc:654] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:48.937145   607 leader_election.cc:304] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:48.929299   715 raft_consensus.cc:2468] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 0.
I20260709 12:58:48.938985   758 tablet_bootstrap.cc:492] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824: No bootstrap required, opened a new log
I20260709 12:58:48.942126   758 ts_tablet_manager.cc:1403] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824: Time spent bootstrapping tablet: real 0.074s	user 0.010s	sys 0.011s
I20260709 12:58:48.943645   766 raft_consensus.cc:2804] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:48.944281   766 raft_consensus.cc:493] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:48.944660   766 raft_consensus.cc:3060] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.946045   758 raft_consensus.cc:359] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.954254   758 raft_consensus.cc:385] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.954844   758 raft_consensus.cc:740] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.955850   758 consensus_queue.cc:260] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.959345   758 ts_tablet_manager.cc:1434] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:58:48.955484   766 raft_consensus.cc:515] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.962399   759 tablet_bootstrap.cc:492] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755: No bootstrap required, opened a new log
I20260709 12:58:48.965529   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b495693a89fb4dddbd68886b3f62f833" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:48.963002   759 ts_tablet_manager.cc:1403] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755: Time spent bootstrapping tablet: real 0.112s	user 0.020s	sys 0.018s
I20260709 12:58:48.966245   565 raft_consensus.cc:3060] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.966800   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b495693a89fb4dddbd68886b3f62f833" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:48.967481   715 raft_consensus.cc:3060] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:48.970364   759 raft_consensus.cc:359] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:48.971204   759 raft_consensus.cc:385] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:48.971515   759 raft_consensus.cc:740] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Initialized, Role: FOLLOWER
I20260709 12:58:48.973084   758 tablet_bootstrap.cc:492] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824: Bootstrap starting.
I20260709 12:58:48.964651   462 catalog_manager.cc:5697] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8300451f93f44610a0778716b5890b71 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8300451f93f44610a0778716b5890b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:48.977001   715 raft_consensus.cc:2468] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 1.
I20260709 12:58:48.978111   759 consensus_queue.cc:260] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:48.980831   759 ts_tablet_manager.cc:1434] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:58:48.980002   605 leader_election.cc:304] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: 
I20260709 12:58:48.982228   759 tablet_bootstrap.cc:492] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755: Bootstrap starting.
I20260709 12:58:48.982355   786 raft_consensus.cc:2804] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:48.981304   565 raft_consensus.cc:2468] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 1.
I20260709 12:58:48.977629   766 leader_election.cc:290] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:48.988643   786 raft_consensus.cc:697] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Becoming Leader. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:48.991366   786 consensus_queue.cc:237] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:48.997432   759 tablet_bootstrap.cc:654] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:49.005791   758 tablet_bootstrap.cc:654] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:49.006021   760 tablet_bootstrap.cc:492] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71: No bootstrap required, opened a new log
I20260709 12:58:49.006858   760 ts_tablet_manager.cc:1403] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71: Time spent bootstrapping tablet: real 0.113s	user 0.015s	sys 0.006s
I20260709 12:58:49.011323   760 raft_consensus.cc:359] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.012331   760 raft_consensus.cc:385] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:49.012733   760 raft_consensus.cc:740] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Initialized, Role: FOLLOWER
I20260709 12:58:49.013769   760 consensus_queue.cc:260] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.017663   760 ts_tablet_manager.cc:1434] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.002s
I20260709 12:58:49.018924   760 tablet_bootstrap.cc:492] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71: Bootstrap starting.
I20260709 12:58:49.019315   463 catalog_manager.cc:5697] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8300451f93f44610a0778716b5890b71 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8300451f93f44610a0778716b5890b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:49.028597   760 tablet_bootstrap.cc:654] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:49.038820   765 raft_consensus.cc:493] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:49.039454   765 raft_consensus.cc:515] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:49.041409   760 tablet_bootstrap.cc:492] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71: No bootstrap required, opened a new log
I20260709 12:58:49.042178   765 leader_election.cc:290] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:49.044445   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71" is_pre_election: true
I20260709 12:58:49.042037   760 ts_tablet_manager.cc:1403] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.012s
I20260709 12:58:49.046697   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:49.048019   715 raft_consensus.cc:2393] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7f1c894cd2c4c11b36b837667423755 in current term 1: Already voted for candidate 8300451f93f44610a0778716b5890b71 in this term.
I20260709 12:58:49.049463   760 raft_consensus.cc:359] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.050266   760 raft_consensus.cc:385] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:49.050112   530 leader_election.cc:304] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7f1c894cd2c4c11b36b837667423755; no voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824
I20260709 12:58:49.050657   760 raft_consensus.cc:740] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Initialized, Role: FOLLOWER
I20260709 12:58:49.051874   765 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.051699   760 consensus_queue.cc:260] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.055173   760 ts_tablet_manager.cc:1434] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:58:49.056201   760 tablet_bootstrap.cc:492] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71: Bootstrap starting.
I20260709 12:58:49.077169   765 raft_consensus.cc:2749] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:49.079203   760 tablet_bootstrap.cc:654] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:49.081110   758 tablet_bootstrap.cc:492] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824: No bootstrap required, opened a new log
I20260709 12:58:49.081651   758 ts_tablet_manager.cc:1403] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824: Time spent bootstrapping tablet: real 0.109s	user 0.013s	sys 0.012s
I20260709 12:58:49.084345   759 tablet_bootstrap.cc:492] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755: No bootstrap required, opened a new log
I20260709 12:58:49.085543   759 ts_tablet_manager.cc:1403] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755: Time spent bootstrapping tablet: real 0.104s	user 0.036s	sys 0.039s
I20260709 12:58:49.085917   758 raft_consensus.cc:359] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.086737   758 raft_consensus.cc:385] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:49.087038   758 raft_consensus.cc:740] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Initialized, Role: FOLLOWER
I20260709 12:58:49.087996   758 consensus_queue.cc:260] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.092514   760 tablet_bootstrap.cc:492] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71: No bootstrap required, opened a new log
I20260709 12:58:49.092994   760 ts_tablet_manager.cc:1403] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71: Time spent bootstrapping tablet: real 0.037s	user 0.003s	sys 0.014s
I20260709 12:58:49.093236   759 raft_consensus.cc:359] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.094094   759 raft_consensus.cc:385] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:49.094444   759 raft_consensus.cc:740] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Initialized, Role: FOLLOWER
I20260709 12:58:49.095306   759 consensus_queue.cc:260] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.095844   760 raft_consensus.cc:359] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.096539   760 raft_consensus.cc:385] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:49.096800   760 raft_consensus.cc:740] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Initialized, Role: FOLLOWER
I20260709 12:58:49.097685   760 consensus_queue.cc:260] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.098389   759 ts_tablet_manager.cc:1434] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:58:49.099493   759 tablet_bootstrap.cc:492] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755: Bootstrap starting.
I20260709 12:58:49.102665   760 ts_tablet_manager.cc:1434] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:58:49.103633   760 tablet_bootstrap.cc:492] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71: Bootstrap starting.
I20260709 12:58:49.109452   759 tablet_bootstrap.cc:654] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:49.111207   760 tablet_bootstrap.cc:654] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:49.123535   759 tablet_bootstrap.cc:492] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755: No bootstrap required, opened a new log
I20260709 12:58:49.124251   760 tablet_bootstrap.cc:492] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71: No bootstrap required, opened a new log
I20260709 12:58:49.124392   759 ts_tablet_manager.cc:1403] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.006s
I20260709 12:58:49.124754   760 ts_tablet_manager.cc:1403] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 12:58:49.128674   760 raft_consensus.cc:359] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.129405   760 raft_consensus.cc:385] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:49.129712   760 raft_consensus.cc:740] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Initialized, Role: FOLLOWER
I20260709 12:58:49.128532   759 raft_consensus.cc:359] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.130483   759 raft_consensus.cc:385] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:49.131778   759 raft_consensus.cc:740] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Initialized, Role: FOLLOWER
I20260709 12:58:49.133037   759 consensus_queue.cc:260] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.133635   758 ts_tablet_manager.cc:1434] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.000s
I20260709 12:58:49.135743   758 tablet_bootstrap.cc:492] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824: Bootstrap starting.
I20260709 12:58:49.136793   759 ts_tablet_manager.cc:1434] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 12:58:49.137976   759 tablet_bootstrap.cc:492] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755: Bootstrap starting.
I20260709 12:58:49.143368   758 tablet_bootstrap.cc:654] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:49.130497   760 consensus_queue.cc:260] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.147337   760 ts_tablet_manager.cc:1434] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:58:49.147382   759 tablet_bootstrap.cc:654] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:49.154491   764 raft_consensus.cc:493] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:49.155110   764 raft_consensus.cc:515] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.157759   788 raft_consensus.cc:493] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:49.158300   788 raft_consensus.cc:515] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.162773   786 raft_consensus.cc:493] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:49.165695   765 raft_consensus.cc:493] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:49.166353   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755" is_pre_election: true
I20260709 12:58:49.166270   765 raft_consensus.cc:515] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.167038   565 raft_consensus.cc:2468] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 0.
I20260709 12:58:49.169286   681 leader_election.cc:304] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:49.163524   786 raft_consensus.cc:515] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.170691   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b383c50beef4a54b0efa339dd0ff48e" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:49.171346   715 raft_consensus.cc:2468] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 0.
I20260709 12:58:49.172825   530 leader_election.cc:304] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:49.173580   786 leader_election.cc:290] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:49.173759   787 raft_consensus.cc:2804] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:49.174119   787 raft_consensus.cc:493] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:49.174491   787 raft_consensus.cc:3060] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.175155   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b383c50beef4a54b0efa339dd0ff48e" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71" is_pre_election: true
I20260709 12:58:49.176517   640 raft_consensus.cc:2468] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 0.
I20260709 12:58:49.177508   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9043d0fea676404f9ce2edb2b42132bc" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755" is_pre_election: true
W20260709 12:58:49.179153   607 leader_election.cc:343] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:49.175961   765 leader_election.cc:290] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:49.180678   765 raft_consensus.cc:493] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:49.183389   765 raft_consensus.cc:515] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.184840   788 leader_election.cc:290] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:49.185410   788 raft_consensus.cc:2804] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:49.185801   788 raft_consensus.cc:493] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:49.186110   788 raft_consensus.cc:3060] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.182798   764 leader_election.cc:290] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:49.195186   785 consensus_queue.cc:1048] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:49.199975   787 raft_consensus.cc:515] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.205740   787 leader_election.cc:290] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 election: Requested vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:49.205600   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10c7df448e04f738e1571969c80925d" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:49.208120   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b383c50beef4a54b0efa339dd0ff48e" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:49.208191   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b383c50beef4a54b0efa339dd0ff48e" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:49.208819   641 raft_consensus.cc:3060] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.208873   713 raft_consensus.cc:3060] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.201086   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10c7df448e04f738e1571969c80925d" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71" is_pre_election: true
I20260709 12:58:49.210439   640 raft_consensus.cc:2393] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7f1c894cd2c4c11b36b837667423755 in current term 1: Already voted for candidate 8f14acb5606549a4ac96fa14aacc9824 in this term.
I20260709 12:58:49.212478   532 leader_election.cc:304] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7f1c894cd2c4c11b36b837667423755; no voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824
I20260709 12:58:49.200512   765 leader_election.cc:290] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:49.208926   788 raft_consensus.cc:515] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.214226   759 tablet_bootstrap.cc:492] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755: No bootstrap required, opened a new log
I20260709 12:58:49.214700   759 ts_tablet_manager.cc:1403] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755: Time spent bootstrapping tablet: real 0.077s	user 0.031s	sys 0.024s
I20260709 12:58:49.213374   787 raft_consensus.cc:3060] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:58:49.217535   681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45333, user_credentials={real_user=slave}} blocked reactor thread for 47866.1us
I20260709 12:58:49.227844   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71" is_pre_election: true
I20260709 12:58:49.228533   640 raft_consensus.cc:2468] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 0.
I20260709 12:58:49.231456   641 raft_consensus.cc:2468] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 1.
I20260709 12:58:49.232934   532 leader_election.cc:304] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:49.233383   787 raft_consensus.cc:2749] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:49.233798   765 raft_consensus.cc:2804] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:49.234342   765 raft_consensus.cc:697] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Becoming Leader. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:49.234912   788 leader_election.cc:290] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:49.235252   765 consensus_queue.cc:237] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.236517   713 raft_consensus.cc:2468] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 1.
I20260709 12:58:49.238039   758 tablet_bootstrap.cc:492] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824: No bootstrap required, opened a new log
I20260709 12:58:49.238529   758 ts_tablet_manager.cc:1403] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824: Time spent bootstrapping tablet: real 0.103s	user 0.013s	sys 0.008s
I20260709 12:58:49.238567   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b383c50beef4a54b0efa339dd0ff48e" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755" is_pre_election: true
I20260709 12:58:49.241369   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b383c50beef4a54b0efa339dd0ff48e" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71" is_pre_election: true
I20260709 12:58:49.242146   641 raft_consensus.cc:2393] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f14acb5606549a4ac96fa14aacc9824 in current term 1: Already voted for candidate d7f1c894cd2c4c11b36b837667423755 in this term.
I20260709 12:58:49.242429   786 consensus_queue.cc:1048] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:49.242789   758 raft_consensus.cc:359] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.243465   758 raft_consensus.cc:385] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:49.243742   758 raft_consensus.cc:740] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Initialized, Role: FOLLOWER
I20260709 12:58:49.244488   758 consensus_queue.cc:260] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.245281   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:49.245448   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9043d0fea676404f9ce2edb2b42132bc" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:49.245954   566 raft_consensus.cc:3060] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.246822   758 ts_tablet_manager.cc:1434] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:58:49.247705   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:49.248239   758 tablet_bootstrap.cc:492] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824: Bootstrap starting.
I20260709 12:58:49.248335   641 raft_consensus.cc:3060] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.234922   759 raft_consensus.cc:359] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.249249   759 raft_consensus.cc:385] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:49.250200   759 raft_consensus.cc:740] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Initialized, Role: FOLLOWER
I20260709 12:58:49.250861   759 consensus_queue.cc:260] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.255705   759 ts_tablet_manager.cc:1434] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 12:58:49.260170   759 tablet_bootstrap.cc:492] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755: Bootstrap starting.
I20260709 12:58:49.261675   681 leader_election.cc:304] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824; no voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755
W20260709 12:58:49.266839   605 leader_election.cc:343] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:49.267417   605 leader_election.cc:304] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8300451f93f44610a0778716b5890b71; no voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:49.268708   786 raft_consensus.cc:2749] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:49.268976   788 raft_consensus.cc:2749] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:49.275893   461 catalog_manager.cc:5697] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7f1c894cd2c4c11b36b837667423755 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d7f1c894cd2c4c11b36b837667423755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:49.287289   566 raft_consensus.cc:2468] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 1.
I20260709 12:58:49.288616   641 raft_consensus.cc:2468] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 1.
I20260709 12:58:49.300387   681 leader_election.cc:304] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:49.302363   758 tablet_bootstrap.cc:654] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:49.303903   788 raft_consensus.cc:2804] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:49.304895   788 raft_consensus.cc:697] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 1 LEADER]: Becoming Leader. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Running, Role: LEADER
I20260709 12:58:49.306679   788 consensus_queue.cc:237] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.323930   804 consensus_queue.cc:1048] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:58:49.328310   814 consensus_queue.cc:1048] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:58:49.329349   759 tablet_bootstrap.cc:654] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:49.330729   786 raft_consensus.cc:493] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:49.331252   786 raft_consensus.cc:515] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.335429   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755" is_pre_election: true
I20260709 12:58:49.336318   566 raft_consensus.cc:2468] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 0.
I20260709 12:58:49.338305   607 leader_election.cc:304] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:49.340299   786 leader_election.cc:290] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:49.341123   786 raft_consensus.cc:2804] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:49.341523   786 raft_consensus.cc:493] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:49.343664   787 consensus_queue.cc:1048] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:49.341881   786 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.349931   804 consensus_queue.cc:1048] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:58:49.354902   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:49.358716   715 raft_consensus.cc:2468] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 0.
I20260709 12:58:49.396661   758 tablet_bootstrap.cc:492] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824: No bootstrap required, opened a new log
I20260709 12:58:49.397457   758 ts_tablet_manager.cc:1403] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824: Time spent bootstrapping tablet: real 0.149s	user 0.019s	sys 0.014s
W20260709 12:58:49.403151   607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45333, user_credentials={real_user=slave}} blocked reactor thread for 64842.2us
I20260709 12:58:49.420650   758 raft_consensus.cc:359] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.421420   758 raft_consensus.cc:385] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:49.421766   758 raft_consensus.cc:740] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Initialized, Role: FOLLOWER
I20260709 12:58:49.423207   758 consensus_queue.cc:260] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.419339   824 consensus_queue.cc:1048] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:58:49.429059   786 raft_consensus.cc:515] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.429255   788 consensus_queue.cc:1048] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:49.429987   759 tablet_bootstrap.cc:492] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755: No bootstrap required, opened a new log
I20260709 12:58:49.430521   759 ts_tablet_manager.cc:1403] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755: Time spent bootstrapping tablet: real 0.174s	user 0.020s	sys 0.020s
I20260709 12:58:49.431718   758 ts_tablet_manager.cc:1434] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824: Time spent starting tablet: real 0.034s	user 0.010s	sys 0.011s
I20260709 12:58:49.435644   786 leader_election.cc:290] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:49.440738   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:49.442078   563 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.441423   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:49.444252   713 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.449159   786 consensus_queue.cc:1048] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:49.452176   563 raft_consensus.cc:2468] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 1.
I20260709 12:58:49.458961   713 raft_consensus.cc:2468] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 1.
I20260709 12:58:49.461277   605 leader_election.cc:304] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: 
I20260709 12:58:49.462538   824 raft_consensus.cc:2804] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:49.463063   824 raft_consensus.cc:697] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Becoming Leader. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:49.465931   824 consensus_queue.cc:237] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
W20260709 12:58:49.475654   530 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:35727, user_credentials={real_user=slave}} blocked reactor thread for 63859.4us
I20260709 12:58:49.477900   759 raft_consensus.cc:359] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.478688   759 raft_consensus.cc:385] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:49.478996   759 raft_consensus.cc:740] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Initialized, Role: FOLLOWER
I20260709 12:58:49.480475   759 consensus_queue.cc:260] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.482712   759 ts_tablet_manager.cc:1434] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
W20260709 12:58:49.484629   828 log.cc:927] Time spent T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:58:49.487851   766 consensus_queue.cc:1048] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:49.543116   461 catalog_manager.cc:5697] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8f14acb5606549a4ac96fa14aacc9824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:49.568578   463 catalog_manager.cc:5697] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8300451f93f44610a0778716b5890b71 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8300451f93f44610a0778716b5890b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:49.619208   765 consensus_queue.cc:1048] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:49.633359   765 consensus_queue.cc:1048] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:49.642527   765 raft_consensus.cc:493] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:49.643076   765 raft_consensus.cc:515] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.646572   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9043d0fea676404f9ce2edb2b42132bc" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824" is_pre_election: true
I20260709 12:58:49.647059   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9043d0fea676404f9ce2edb2b42132bc" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71" is_pre_election: true
I20260709 12:58:49.647264   714 raft_consensus.cc:2468] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 0.
I20260709 12:58:49.648028   641 raft_consensus.cc:2468] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 0.
I20260709 12:58:49.648413   765 leader_election.cc:290] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:49.648914   530 leader_election.cc:304] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:49.651582   765 raft_consensus.cc:2804] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:49.652050   765 raft_consensus.cc:493] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:49.652441   765 raft_consensus.cc:3060] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.660502   765 raft_consensus.cc:515] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.664002   765 leader_election.cc:290] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 election: Requested vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:49.664701   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9043d0fea676404f9ce2edb2b42132bc" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:49.664755   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9043d0fea676404f9ce2edb2b42132bc" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:49.665452   714 raft_consensus.cc:3060] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.665459   640 raft_consensus.cc:3060] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:49.671837   640 raft_consensus.cc:2468] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 1.
I20260709 12:58:49.673228   532 leader_election.cc:304] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:49.673990   714 raft_consensus.cc:2468] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 1.
I20260709 12:58:49.674304   765 raft_consensus.cc:2804] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:49.674949   765 raft_consensus.cc:697] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Becoming Leader. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:49.675818   765 consensus_queue.cc:237] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.686159   463 catalog_manager.cc:5697] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7f1c894cd2c4c11b36b837667423755 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d7f1c894cd2c4c11b36b837667423755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:49.711443   814 consensus_queue.cc:1048] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:49.730029   764 consensus_queue.cc:1048] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:49.834069   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86"
dest_uuid: "8300451f93f44610a0778716b5890b71"
mode: GRACEFUL
new_leader_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:46236
I20260709 12:58:49.834908   640 raft_consensus.cc:606] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Received request to transfer leadership to TS d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:49.843677   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "caa377949276484ea703e81b235ee2cf"
dest_uuid: "8300451f93f44610a0778716b5890b71"
mode: GRACEFUL
new_leader_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:46236
I20260709 12:58:49.844452   640 raft_consensus.cc:606] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Received request to transfer leadership to TS d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:49.870144   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b10c7df448e04f738e1571969c80925d"
dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
mode: GRACEFUL
new_leader_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:33158
I20260709 12:58:49.870823   714 raft_consensus.cc:606] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 1 LEADER]: Received request to transfer leadership to TS d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:49.891403   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b383c50beef4a54b0efa339dd0ff48e"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
mode: GRACEFUL
new_leader_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:48584
I20260709 12:58:49.892344   566 raft_consensus.cc:606] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Received request to transfer leadership to TS 8300451f93f44610a0778716b5890b71
I20260709 12:58:49.899992   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba"
dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
mode: GRACEFUL
new_leader_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:33158
I20260709 12:58:49.900789   714 raft_consensus.cc:606] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 1 LEADER]: Received request to transfer leadership to TS 8300451f93f44610a0778716b5890b71
I20260709 12:58:49.909584   824 consensus_queue.cc:1048] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:49.910562   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9043d0fea676404f9ce2edb2b42132bc"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
mode: GRACEFUL
new_leader_uuid: "8f14acb5606549a4ac96fa14aacc9824"
 from {username='slave'} at 127.0.0.1:48584
I20260709 12:58:49.911250   566 raft_consensus.cc:606] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Received request to transfer leadership to TS 8f14acb5606549a4ac96fa14aacc9824
I20260709 12:58:49.941401   827 consensus_queue.cc:1048] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:49.942232   786 raft_consensus.cc:993] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71: : Instructing follower d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:49.942795   824 raft_consensus.cc:1081] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Signalling peer d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:49.944639   566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:48576
I20260709 12:58:49.945273   566 raft_consensus.cc:493] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:49.945700   566 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:49.963443   566 raft_consensus.cc:515] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:49.967523   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:49.970341   764 raft_consensus.cc:993] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824: : Instructing follower d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:49.970391   715 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:49.970484   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:49.970922   764 raft_consensus.cc:1081] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 1 LEADER]: Signalling peer d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:49.971202   640 raft_consensus.cc:3055] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:49.971566   640 raft_consensus.cc:740] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:49.972467   640 consensus_queue.cc:260] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:49.973135   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b10c7df448e04f738e1571969c80925d"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:48578
I20260709 12:58:49.973906   565 raft_consensus.cc:493] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:49.973927   640 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:49.969133   766 raft_consensus.cc:993] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71: : Instructing follower d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:49.978538   715 raft_consensus.cc:2468] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 2.
I20260709 12:58:49.978798   766 raft_consensus.cc:1081] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Signalling peer d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:49.974309   565 raft_consensus.cc:3060] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:49.980962   564 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "caa377949276484ea703e81b235ee2cf"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:48576
I20260709 12:58:49.981593   564 raft_consensus.cc:493] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:49.981998   564 raft_consensus.cc:3060] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:49.982336   530 leader_election.cc:304] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:49.983234   765 raft_consensus.cc:2804] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:49.968590   566 leader_election.cc:290] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 2 election: Requested vote from peers 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:49.983871   765 raft_consensus.cc:697] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Becoming Leader. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:49.987402   765 consensus_queue.cc:237] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:49.993503   640 raft_consensus.cc:2468] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 2.
I20260709 12:58:49.993578   564 raft_consensus.cc:515] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:49.998319   463 catalog_manager.cc:5697] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 reported cstate change: term changed from 1 to 2, leader changed from 8300451f93f44610a0778716b5890b71 (127.0.107.194) to d7f1c894cd2c4c11b36b837667423755 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "d7f1c894cd2c4c11b36b837667423755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:49.999984   565 raft_consensus.cc:515] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.003453   565 leader_election.cc:290] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 2 election: Requested vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:50.003566   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa377949276484ea703e81b235ee2cf" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:50.004359   640 raft_consensus.cc:3055] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:50.004765   640 raft_consensus.cc:740] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:50.005272   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10c7df448e04f738e1571969c80925d" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:50.005883   641 raft_consensus.cc:3060] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.005904   640 consensus_queue.cc:260] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.008601   640 raft_consensus.cc:3060] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.009042   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa377949276484ea703e81b235ee2cf" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:50.009824   714 raft_consensus.cc:3060] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.013005   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10c7df448e04f738e1571969c80925d" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:50.013850   715 raft_consensus.cc:3055] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:50.014196   715 raft_consensus.cc:740] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Running, Role: LEADER
I20260709 12:58:50.016676   715 consensus_queue.cc:260] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.019435   715 raft_consensus.cc:3060] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.019331   714 raft_consensus.cc:2468] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 2.
I20260709 12:58:50.020375   641 raft_consensus.cc:2468] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 2.
I20260709 12:58:50.009033   564 leader_election.cc:290] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 2 election: Requested vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:50.021384   530 leader_election.cc:304] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:50.021435   532 leader_election.cc:304] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:50.022362   765 raft_consensus.cc:2804] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:50.022744   787 raft_consensus.cc:2804] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:50.022821   765 raft_consensus.cc:697] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Becoming Leader. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:50.023298   787 raft_consensus.cc:697] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Becoming Leader. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:50.023790   765 consensus_queue.cc:237] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.024096   787 consensus_queue.cc:237] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.019325   640 raft_consensus.cc:2468] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 2.
I20260709 12:58:50.031176   715 raft_consensus.cc:2468] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 2.
I20260709 12:58:50.037906   463 catalog_manager.cc:5697] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 reported cstate change: term changed from 1 to 2, leader changed from 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195) to d7f1c894cd2c4c11b36b837667423755 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "d7f1c894cd2c4c11b36b837667423755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:50.039393   463 catalog_manager.cc:5697] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 reported cstate change: term changed from 1 to 2, leader changed from 8300451f93f44610a0778716b5890b71 (127.0.107.194) to d7f1c894cd2c4c11b36b837667423755 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "d7f1c894cd2c4c11b36b837667423755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:50.047811   765 raft_consensus.cc:993] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755: : Instructing follower 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:50.048493   787 raft_consensus.cc:1081] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Signalling peer 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:50.050680   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b383c50beef4a54b0efa339dd0ff48e"
dest_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:46220
I20260709 12:58:50.051396   640 raft_consensus.cc:493] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:50.051836   640 raft_consensus.cc:3060] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.060581   640 raft_consensus.cc:515] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.062866   640 leader_election.cc:290] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 2 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:50.063833   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b383c50beef4a54b0efa339dd0ff48e" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:50.064687   564 raft_consensus.cc:3055] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:50.064873   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b383c50beef4a54b0efa339dd0ff48e" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:50.065101   564 raft_consensus.cc:740] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:50.065672   715 raft_consensus.cc:3060] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.066188   564 consensus_queue.cc:260] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.067481   564 raft_consensus.cc:3060] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.074267   715 raft_consensus.cc:2468] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 2.
I20260709 12:58:50.075943   564 raft_consensus.cc:2468] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 2.
I20260709 12:58:50.075951   605 leader_election.cc:304] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: 
I20260709 12:58:50.077137   766 raft_consensus.cc:2804] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:50.077791   766 raft_consensus.cc:697] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Becoming Leader. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:50.078804   766 consensus_queue.cc:237] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.087455   463 catalog_manager.cc:5697] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 reported cstate change: term changed from 1 to 2, leader changed from d7f1c894cd2c4c11b36b837667423755 (127.0.107.193) to 8300451f93f44610a0778716b5890b71 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8300451f93f44610a0778716b5890b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:50.108836   787 consensus_queue.cc:1048] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:50.122108   787 consensus_queue.cc:1048] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:50.133855   785 raft_consensus.cc:993] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755: : Instructing follower 8f14acb5606549a4ac96fa14aacc9824 to start an election
I20260709 12:58:50.134388   787 raft_consensus.cc:1081] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Signalling peer 8f14acb5606549a4ac96fa14aacc9824 to start an election
I20260709 12:58:50.136375   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9043d0fea676404f9ce2edb2b42132bc"
dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
 from {username='slave'} at 127.0.0.1:33154
I20260709 12:58:50.136942   715 raft_consensus.cc:493] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:50.137291   715 raft_consensus.cc:3060] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.143849   715 raft_consensus.cc:515] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.146219   715 leader_election.cc:290] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 2 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:50.147692   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9043d0fea676404f9ce2edb2b42132bc" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:50.148233   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9043d0fea676404f9ce2edb2b42132bc" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:50.148442   564 raft_consensus.cc:3055] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:50.148847   564 raft_consensus.cc:740] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:50.148893   640 raft_consensus.cc:3060] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.150025   564 consensus_queue.cc:260] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.151549   564 raft_consensus.cc:3060] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.158558   640 raft_consensus.cc:2468] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 2.
I20260709 12:58:50.161036   564 raft_consensus.cc:2468] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 2.
I20260709 12:58:50.161104   681 leader_election.cc:304] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: 
I20260709 12:58:50.162226   788 raft_consensus.cc:2804] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:50.162746   788 raft_consensus.cc:697] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 2 LEADER]: Becoming Leader. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Running, Role: LEADER
I20260709 12:58:50.163487   788 consensus_queue.cc:237] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.171820   462 catalog_manager.cc:5697] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 reported cstate change: term changed from 1 to 2, leader changed from d7f1c894cd2c4c11b36b837667423755 (127.0.107.193) to 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8f14acb5606549a4ac96fa14aacc9824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:50.255108   814 raft_consensus.cc:993] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824: : Instructing follower 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:50.255620   814 raft_consensus.cc:1081] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 1 LEADER]: Signalling peer 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:50.257563   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba"
dest_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:46234
I20260709 12:58:50.258183   640 raft_consensus.cc:493] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:50.258572   640 raft_consensus.cc:3060] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.267549   640 raft_consensus.cc:515] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.271127   640 leader_election.cc:290] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 2 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:50.272173   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:50.272176   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:50.273061   715 raft_consensus.cc:3055] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:50.273175   564 raft_consensus.cc:3060] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.273662   715 raft_consensus.cc:740] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Running, Role: LEADER
I20260709 12:58:50.274678   715 consensus_queue.cc:260] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.276221   715 raft_consensus.cc:3060] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:50.281313   564 raft_consensus.cc:2468] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 2.
I20260709 12:58:50.283900   607 leader_election.cc:304] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:50.284826   766 raft_consensus.cc:2804] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:50.285343   715 raft_consensus.cc:2468] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 2.
I20260709 12:58:50.285362   766 raft_consensus.cc:697] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Becoming Leader. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:50.287031   766 consensus_queue.cc:237] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:50.295027   462 catalog_manager.cc:5697] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 reported cstate change: term changed from 1 to 2, leader changed from 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195) to 8300451f93f44610a0778716b5890b71 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8300451f93f44610a0778716b5890b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:50.383163   715 raft_consensus.cc:1275] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.384747   787 consensus_queue.cc:1048] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:50.404299   640 raft_consensus.cc:1275] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.407649   787 consensus_queue.cc:1048] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:50.432461   715 raft_consensus.cc:1275] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.434063   787 consensus_queue.cc:1048] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:50.442545   640 raft_consensus.cc:1275] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.447295   787 consensus_queue.cc:1048] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:50.505927   715 raft_consensus.cc:1275] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.507548   766 consensus_queue.cc:1048] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:50.517550   564 raft_consensus.cc:1275] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.519266   824 consensus_queue.cc:1048] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:50.527103   640 raft_consensus.cc:1275] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.528905   787 consensus_queue.cc:1048] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:50.544376   715 raft_consensus.cc:1275] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.546842   820 consensus_queue.cc:1048] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:58:50.556960   564 raft_consensus.cc:1275] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Refusing update from remote peer 8f14acb5606549a4ac96fa14aacc9824: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.564906   814 consensus_queue.cc:1048] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:50.576216   640 raft_consensus.cc:1275] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Refusing update from remote peer 8f14acb5606549a4ac96fa14aacc9824: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.578236   788 consensus_queue.cc:1048] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:58:50.768610   564 raft_consensus.cc:1275] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.770030   804 consensus_queue.cc:1048] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:50.783951   714 raft_consensus.cc:1275] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:50.786455   766 consensus_queue.cc:1048] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:52.946900   431 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 12:58:52.947947   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:52.975183   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f"
dest_uuid: "8300451f93f44610a0778716b5890b71"
mode: GRACEFUL
new_leader_uuid: "8f14acb5606549a4ac96fa14aacc9824"
 from {username='slave'} at 127.0.0.1:54188
I20260709 12:58:52.975840   641 raft_consensus.cc:606] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Received request to transfer leadership to TS 8f14acb5606549a4ac96fa14aacc9824
I20260709 12:58:52.990845   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
mode: GRACEFUL
new_leader_uuid: "8f14acb5606549a4ac96fa14aacc9824"
 from {username='slave'} at 127.0.0.1:38734
I20260709 12:58:52.991479   565 raft_consensus.cc:606] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Received request to transfer leadership to TS 8f14acb5606549a4ac96fa14aacc9824
I20260709 12:58:52.992861   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:58:52.993342   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:58:53.110651   858 raft_consensus.cc:993] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71: : Instructing follower 8f14acb5606549a4ac96fa14aacc9824 to start an election
I20260709 12:58:53.111140   864 raft_consensus.cc:1081] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Signalling peer 8f14acb5606549a4ac96fa14aacc9824 to start an election
I20260709 12:58:53.112812   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f"
dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
 from {username='slave'} at 127.0.0.1:33144
I20260709 12:58:53.113371   715 raft_consensus.cc:493] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:53.113698   715 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:53.118479   715 raft_consensus.cc:515] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:53.120373   715 leader_election.cc:290] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 2 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:53.121292   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:53.121557   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:53.121940   565 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:53.122161   641 raft_consensus.cc:3055] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:53.122519   641 raft_consensus.cc:740] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:53.123623   641 consensus_queue.cc:260] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:53.124771   641 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:53.128955   859 raft_consensus.cc:993] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755: : Instructing follower 8f14acb5606549a4ac96fa14aacc9824 to start an election
I20260709 12:58:53.129562   860 raft_consensus.cc:1081] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Signalling peer 8f14acb5606549a4ac96fa14aacc9824 to start an election
I20260709 12:58:53.129570   565 raft_consensus.cc:2468] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 2.
I20260709 12:58:53.131455   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86"
dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
 from {username='slave'} at 127.0.0.1:33154
I20260709 12:58:53.131953   715 raft_consensus.cc:493] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:53.131810   681 leader_election.cc:304] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:53.132037   641 raft_consensus.cc:2468] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 2.
I20260709 12:58:53.132340   715 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:53.132732   857 raft_consensus.cc:2804] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:53.134208   857 raft_consensus.cc:697] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 2 LEADER]: Becoming Leader. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Running, Role: LEADER
I20260709 12:58:53.135087   857 consensus_queue.cc:237] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:53.139674   715 raft_consensus.cc:515] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:53.141309   715 leader_election.cc:290] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 3 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:53.142324   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:53.142657   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:53.143311   565 raft_consensus.cc:3055] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:58:53.143427   641 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:53.143710   565 raft_consensus.cc:740] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:53.144549   565 consensus_queue.cc:260] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:53.145722   565 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:53.148340   463 catalog_manager.cc:5697] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 reported cstate change: term changed from 1 to 2, leader changed from 8300451f93f44610a0778716b5890b71 (127.0.107.194) to 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8f14acb5606549a4ac96fa14aacc9824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:53.149551   641 raft_consensus.cc:2468] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 3.
I20260709 12:58:53.150923   681 leader_election.cc:304] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: 
I20260709 12:58:53.151685   857 raft_consensus.cc:2804] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:58:53.152158   857 raft_consensus.cc:697] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 3 LEADER]: Becoming Leader. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Running, Role: LEADER
I20260709 12:58:53.153167   565 raft_consensus.cc:2468] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 3.
I20260709 12:58:53.153108   857 consensus_queue.cc:237] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:53.170022   463 catalog_manager.cc:5697] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 reported cstate change: term changed from 2 to 3, leader changed from d7f1c894cd2c4c11b36b837667423755 (127.0.107.193) to 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "8f14acb5606549a4ac96fa14aacc9824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:53.612427   641 raft_consensus.cc:1275] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Refusing update from remote peer 8f14acb5606549a4ac96fa14aacc9824: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:53.613916   857 consensus_queue.cc:1048] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:58:53.626659   565 raft_consensus.cc:1275] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Refusing update from remote peer 8f14acb5606549a4ac96fa14aacc9824: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:53.628296   857 consensus_queue.cc:1048] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:58:53.691031   565 raft_consensus.cc:1275] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Refusing update from remote peer 8f14acb5606549a4ac96fa14aacc9824: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:53.692623   857 consensus_queue.cc:1048] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:58:53.695889   641 raft_consensus.cc:1275] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Refusing update from remote peer 8f14acb5606549a4ac96fa14aacc9824: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:53.697521   874 consensus_queue.cc:1048] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:58:53.994448   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:54.005688   715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86"
dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
mode: GRACEFUL
new_leader_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:33158
I20260709 12:58:54.006304   715 raft_consensus.cc:606] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 3 LEADER]: Received request to transfer leadership to TS 8300451f93f44610a0778716b5890b71
I20260709 12:58:54.011971   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "caa377949276484ea703e81b235ee2cf"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
mode: GRACEFUL
new_leader_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:48584
I20260709 12:58:54.012583   565 raft_consensus.cc:606] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Received request to transfer leadership to TS 8300451f93f44610a0778716b5890b71
I20260709 12:58:54.019110   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
mode: GRACEFUL
new_leader_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:48584
I20260709 12:58:54.019630   565 raft_consensus.cc:606] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Received request to transfer leadership to TS 8300451f93f44610a0778716b5890b71
I20260709 12:58:54.023734   874 raft_consensus.cc:993] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824: : Instructing follower 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:54.024333   865 raft_consensus.cc:1081] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 3 LEADER]: Signalling peer 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:54.025458   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b10c7df448e04f738e1571969c80925d"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
mode: GRACEFUL
new_leader_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:48584
I20260709 12:58:54.026139   565 raft_consensus.cc:606] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Received request to transfer leadership to TS 8300451f93f44610a0778716b5890b71
I20260709 12:58:54.026046   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86"
dest_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:46234
I20260709 12:58:54.027561   641 raft_consensus.cc:493] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:54.027966   641 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:58:54.035022   641 raft_consensus.cc:515] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:54.038059   641 leader_election.cc:290] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 4 election: Requested vote from peers 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727), d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333)
I20260709 12:58:54.038707   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:54.039083   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:54.039417   715 raft_consensus.cc:3055] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:58:54.039666   715 raft_consensus.cc:740] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Running, Role: LEADER
I20260709 12:58:54.039708   565 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:58:54.040315   715 consensus_queue.cc:260] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:54.041169   715 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:58:54.044903   565 raft_consensus.cc:2468] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 4.
I20260709 12:58:54.046248   715 raft_consensus.cc:2468] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 4.
I20260709 12:58:54.046274   607 leader_election.cc:304] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:54.047439   864 raft_consensus.cc:2804] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:58:54.048243   715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f"
dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
mode: GRACEFUL
new_leader_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:33158
I20260709 12:58:54.048825   715 raft_consensus.cc:606] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 2 LEADER]: Received request to transfer leadership to TS 8300451f93f44610a0778716b5890b71
I20260709 12:58:54.048979   864 raft_consensus.cc:697] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 4 LEADER]: Becoming Leader. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:54.050841   864 consensus_queue.cc:237] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:54.057384   461 catalog_manager.cc:5697] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 reported cstate change: term changed from 3 to 4, leader changed from 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195) to 8300451f93f44610a0778716b5890b71 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "8300451f93f44610a0778716b5890b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:54.152549   874 raft_consensus.cc:993] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824: : Instructing follower 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:54.153057   865 raft_consensus.cc:1081] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 2 LEADER]: Signalling peer 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:54.155355   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f"
dest_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:46234
I20260709 12:58:54.155889   641 raft_consensus.cc:493] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:54.156154   641 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:54.161393   641 raft_consensus.cc:515] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.163197   641 leader_election.cc:290] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 3 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:54.164180   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:54.164192   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:54.164971   565 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:54.165079   715 raft_consensus.cc:3055] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:58:54.165463   715 raft_consensus.cc:740] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Running, Role: LEADER
I20260709 12:58:54.166170   715 consensus_queue.cc:260] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.167184   715 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:54.171231   565 raft_consensus.cc:2468] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 3.
I20260709 12:58:54.171669   715 raft_consensus.cc:2468] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 3.
I20260709 12:58:54.172524   607 leader_election.cc:304] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:54.173455   864 raft_consensus.cc:2804] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:58:54.173928   864 raft_consensus.cc:697] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 3 LEADER]: Becoming Leader. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:54.174700   864 consensus_queue.cc:237] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.181348   461 catalog_manager.cc:5697] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 reported cstate change: term changed from 2 to 3, leader changed from 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195) to 8300451f93f44610a0778716b5890b71 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8300451f93f44610a0778716b5890b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:54.219558   880 raft_consensus.cc:993] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755: : Instructing follower 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:54.220078   860 raft_consensus.cc:1081] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Signalling peer 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:54.221848   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b10c7df448e04f738e1571969c80925d"
dest_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:46220
I20260709 12:58:54.222350   641 raft_consensus.cc:493] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:54.222625   641 raft_consensus.cc:3060] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:54.228760   641 raft_consensus.cc:515] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.230621   641 leader_election.cc:290] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 3 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:54.231901   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10c7df448e04f738e1571969c80925d" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:54.231887   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10c7df448e04f738e1571969c80925d" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:54.232614   565 raft_consensus.cc:3055] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:58:54.232779   715 raft_consensus.cc:3060] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:54.232933   565 raft_consensus.cc:740] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:54.234021   565 consensus_queue.cc:260] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.235095   565 raft_consensus.cc:3060] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:54.238066   715 raft_consensus.cc:2468] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 3.
I20260709 12:58:54.239413   605 leader_election.cc:304] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: 
I20260709 12:58:54.240339   864 raft_consensus.cc:2804] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:58:54.240814   864 raft_consensus.cc:697] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 3 LEADER]: Becoming Leader. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:54.241812   864 consensus_queue.cc:237] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.242328   565 raft_consensus.cc:2468] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 3.
I20260709 12:58:54.250103   461 catalog_manager.cc:5697] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 reported cstate change: term changed from 2 to 3, leader changed from d7f1c894cd2c4c11b36b837667423755 (127.0.107.193) to 8300451f93f44610a0778716b5890b71 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8300451f93f44610a0778716b5890b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:54.450604   880 raft_consensus.cc:993] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755: : Instructing follower 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:54.451103   860 raft_consensus.cc:1081] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Signalling peer 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:54.452849   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a"
dest_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:46220
I20260709 12:58:54.453347   641 raft_consensus.cc:493] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:54.453640   641 raft_consensus.cc:3060] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:54.459730   641 raft_consensus.cc:515] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.462265   641 leader_election.cc:290] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 2 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:54.463147   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:54.463493   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:54.463600   715 raft_consensus.cc:1275] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 4 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:58:54.463842   565 raft_consensus.cc:3055] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:54.464144   714 raft_consensus.cc:3060] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:54.464288   565 raft_consensus.cc:740] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:54.466615   864 consensus_queue.cc:1048] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:58:54.465235   565 consensus_queue.cc:260] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.468012   565 raft_consensus.cc:3060] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:54.473240   714 raft_consensus.cc:2468] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 2.
I20260709 12:58:54.474975   605 leader_election.cc:304] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: 
I20260709 12:58:54.476526   881 raft_consensus.cc:2804] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:54.477150   881 raft_consensus.cc:697] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Becoming Leader. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:54.477823   564 raft_consensus.cc:1275] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 4 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:58:54.478245   881 consensus_queue.cc:237] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.479802   888 consensus_queue.cc:1048] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:58:54.487746   565 raft_consensus.cc:2468] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 2.
I20260709 12:58:54.490698   461 catalog_manager.cc:5697] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 reported cstate change: term changed from 1 to 2, leader changed from d7f1c894cd2c4c11b36b837667423755 (127.0.107.193) to 8300451f93f44610a0778716b5890b71 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8300451f93f44610a0778716b5890b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:54.510108   880 raft_consensus.cc:993] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755: : Instructing follower 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:54.510576   860 raft_consensus.cc:1081] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Signalling peer 8300451f93f44610a0778716b5890b71 to start an election
I20260709 12:58:54.512039   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "caa377949276484ea703e81b235ee2cf"
dest_uuid: "8300451f93f44610a0778716b5890b71"
 from {username='slave'} at 127.0.0.1:46220
I20260709 12:58:54.512606   641 raft_consensus.cc:493] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:54.512943   641 raft_consensus.cc:3060] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:54.519022   641 raft_consensus.cc:515] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.520954   641 leader_election.cc:290] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 3 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:54.521948   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa377949276484ea703e81b235ee2cf" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:54.522156   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caa377949276484ea703e81b235ee2cf" candidate_uuid: "8300451f93f44610a0778716b5890b71" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:54.522580   565 raft_consensus.cc:3055] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:58:54.522621   715 raft_consensus.cc:3060] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:54.523773   565 raft_consensus.cc:740] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:54.524587   565 consensus_queue.cc:260] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.525780   565 raft_consensus.cc:3060] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:54.529728   715 raft_consensus.cc:2468] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 3.
I20260709 12:58:54.530925   605 leader_election.cc:304] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300451f93f44610a0778716b5890b71, 8f14acb5606549a4ac96fa14aacc9824; no voters: 
I20260709 12:58:54.531662   888 raft_consensus.cc:2804] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:58:54.531822   565 raft_consensus.cc:2468] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300451f93f44610a0778716b5890b71 in term 3.
I20260709 12:58:54.532092   888 raft_consensus.cc:697] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 3 LEADER]: Becoming Leader. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:54.533058   888 consensus_queue.cc:237] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:54.541692   461 catalog_manager.cc:5697] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 reported cstate change: term changed from 2 to 3, leader changed from d7f1c894cd2c4c11b36b837667423755 (127.0.107.193) to 8300451f93f44610a0778716b5890b71 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8300451f93f44610a0778716b5890b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:54.629652   715 raft_consensus.cc:1275] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:54.631156   888 consensus_queue.cc:1048] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:58:54.641595   565 raft_consensus.cc:1275] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:54.643059   888 consensus_queue.cc:1048] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:58:54.664947   715 raft_consensus.cc:1275] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:54.666409   888 consensus_queue.cc:1048] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:58:54.675392   565 raft_consensus.cc:1275] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:54.676895   864 consensus_queue.cc:1048] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:58:55.029691   564 raft_consensus.cc:1275] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:55.031111   864 consensus_queue.cc:1048] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:55.040333   715 raft_consensus.cc:1275] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:55.042490   864 consensus_queue.cc:1048] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:55.090523   564 raft_consensus.cc:1275] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:55.091876   864 consensus_queue.cc:1048] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:58:55.101048   715 raft_consensus.cc:1275] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Refusing update from remote peer 8300451f93f44610a0778716b5890b71: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:55.102753   881 consensus_queue.cc:1048] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:58:57.054694   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:57.063583   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f"
dest_uuid: "8300451f93f44610a0778716b5890b71"
mode: GRACEFUL
new_leader_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:54188
I20260709 12:58:57.064378   641 raft_consensus.cc:606] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 3 LEADER]: Received request to transfer leadership to TS d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:57.067837   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86"
dest_uuid: "8300451f93f44610a0778716b5890b71"
mode: GRACEFUL
new_leader_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:54188
I20260709 12:58:57.068423   641 raft_consensus.cc:606] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 4 LEADER]: Received request to transfer leadership to TS d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:57.071077   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a"
dest_uuid: "8300451f93f44610a0778716b5890b71"
mode: GRACEFUL
new_leader_uuid: "8f14acb5606549a4ac96fa14aacc9824"
 from {username='slave'} at 127.0.0.1:54188
I20260709 12:58:57.071754   641 raft_consensus.cc:606] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Received request to transfer leadership to TS 8f14acb5606549a4ac96fa14aacc9824
I20260709 12:58:57.074304   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b495693a89fb4dddbd68886b3f62f833"
dest_uuid: "8300451f93f44610a0778716b5890b71"
mode: GRACEFUL
new_leader_uuid: "8f14acb5606549a4ac96fa14aacc9824"
 from {username='slave'} at 127.0.0.1:54188
I20260709 12:58:57.074862   641 raft_consensus.cc:606] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Received request to transfer leadership to TS 8f14acb5606549a4ac96fa14aacc9824
I20260709 12:58:57.077225   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba"
dest_uuid: "8300451f93f44610a0778716b5890b71"
mode: GRACEFUL
new_leader_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:54188
I20260709 12:58:57.077841   641 raft_consensus.cc:606] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Received request to transfer leadership to TS d7f1c894cd2c4c11b36b837667423755
I20260709 12:58:57.079012   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:58:57.079569   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:58:57.140779   901 raft_consensus.cc:993] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71: : Instructing follower d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:57.141425   902 raft_consensus.cc:1081] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 3 LEADER]: Signalling peer d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:57.143450   564 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:48576
I20260709 12:58:57.144110   564 raft_consensus.cc:493] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:57.144481   564 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:58:57.150645   564 raft_consensus.cc:515] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.152814   564 leader_election.cc:290] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 4 election: Requested vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:57.154325   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:57.154210   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106212907bb64e75a6a5204fae36c61f" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:57.155107   715 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:58:57.155156   641 raft_consensus.cc:3055] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:58:57.155628   641 raft_consensus.cc:740] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:57.156597   641 consensus_queue.cc:260] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.158025   641 raft_consensus.cc:3060] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:58:57.163242   715 raft_consensus.cc:2468] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 4.
I20260709 12:58:57.164911   530 leader_election.cc:304] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:57.165120   641 raft_consensus.cc:2468] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 4.
I20260709 12:58:57.166754   903 raft_consensus.cc:2804] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:58:57.167745   903 raft_consensus.cc:697] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 4 LEADER]: Becoming Leader. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:57.168692   903 consensus_queue.cc:237] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.177057   461 catalog_manager.cc:5697] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 reported cstate change: term changed from 3 to 4, leader changed from 8300451f93f44610a0778716b5890b71 (127.0.107.194) to d7f1c894cd2c4c11b36b837667423755 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "d7f1c894cd2c4c11b36b837667423755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:57.234972   901 raft_consensus.cc:993] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71: : Instructing follower 8f14acb5606549a4ac96fa14aacc9824 to start an election
I20260709 12:58:57.235473   901 raft_consensus.cc:1081] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Signalling peer 8f14acb5606549a4ac96fa14aacc9824 to start an election
I20260709 12:58:57.237079   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a"
dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
 from {username='slave'} at 127.0.0.1:33144
I20260709 12:58:57.237612   715 raft_consensus.cc:493] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:57.238010   715 raft_consensus.cc:3060] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:57.243263   715 raft_consensus.cc:515] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.244992   715 leader_election.cc:290] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 3 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:57.246218   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:57.246446   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9f83b7307c45fda8df8c9fbc844c4a" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:57.246830   564 raft_consensus.cc:3060] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:57.247036   641 raft_consensus.cc:3055] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:58:57.247422   641 raft_consensus.cc:740] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:57.248181   641 consensus_queue.cc:260] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.249156   641 raft_consensus.cc:3060] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:57.252717   564 raft_consensus.cc:2468] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 3.
I20260709 12:58:57.254163   641 raft_consensus.cc:2468] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 3.
I20260709 12:58:57.254227   681 leader_election.cc:304] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:57.255245   900 raft_consensus.cc:2804] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:58:57.256525   900 raft_consensus.cc:697] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 3 LEADER]: Becoming Leader. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Running, Role: LEADER
I20260709 12:58:57.257318   900 consensus_queue.cc:237] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.265430   461 catalog_manager.cc:5697] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 reported cstate change: term changed from 2 to 3, leader changed from 8300451f93f44610a0778716b5890b71 (127.0.107.194) to 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "8f14acb5606549a4ac96fa14aacc9824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:57.490305   902 raft_consensus.cc:993] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71: : Instructing follower 8f14acb5606549a4ac96fa14aacc9824 to start an election
I20260709 12:58:57.490831   901 raft_consensus.cc:1081] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Signalling peer 8f14acb5606549a4ac96fa14aacc9824 to start an election
I20260709 12:58:57.492580   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b495693a89fb4dddbd68886b3f62f833"
dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
 from {username='slave'} at 127.0.0.1:33144
I20260709 12:58:57.493124   715 raft_consensus.cc:493] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:57.493435   715 raft_consensus.cc:3060] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:57.498700   715 raft_consensus.cc:515] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.500432   715 leader_election.cc:290] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 2 election: Requested vote from peers d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:57.501737   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b495693a89fb4dddbd68886b3f62f833" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
I20260709 12:58:57.501873   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b495693a89fb4dddbd68886b3f62f833" candidate_uuid: "8f14acb5606549a4ac96fa14aacc9824" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:57.502444   564 raft_consensus.cc:3060] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:57.502549   641 raft_consensus.cc:3055] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:57.502913   641 raft_consensus.cc:740] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:57.503598   641 consensus_queue.cc:260] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.504527   641 raft_consensus.cc:3060] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:57.507916   564 raft_consensus.cc:2468] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 2.
I20260709 12:58:57.509435   681 leader_election.cc:304] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:57.510381   900 raft_consensus.cc:2804] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:57.510840   900 raft_consensus.cc:697] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 2 LEADER]: Becoming Leader. State: Replica: 8f14acb5606549a4ac96fa14aacc9824, State: Running, Role: LEADER
I20260709 12:58:57.512036   641 raft_consensus.cc:2468] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f14acb5606549a4ac96fa14aacc9824 in term 2.
I20260709 12:58:57.511801   900 consensus_queue.cc:237] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.512645   902 raft_consensus.cc:993] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71: : Instructing follower d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:57.513988   901 raft_consensus.cc:1081] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 4 LEADER]: Signalling peer d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:57.516566   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:48576
I20260709 12:58:57.517145   565 raft_consensus.cc:493] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:57.517567   565 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:58:57.520937   461 catalog_manager.cc:5697] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 reported cstate change: term changed from 1 to 2, leader changed from 8300451f93f44610a0778716b5890b71 (127.0.107.194) to 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8f14acb5606549a4ac96fa14aacc9824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:57.523061   565 raft_consensus.cc:515] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:57.525214   565 leader_election.cc:290] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 5 election: Requested vote from peers 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727), 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795)
I20260709 12:58:57.526417   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:57.527033   641 raft_consensus.cc:3055] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:58:57.527371   641 raft_consensus.cc:740] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:57.527576   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228000099d3449dcb36878d71bcdbe86" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:57.528167   715 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:58:57.528198   641 consensus_queue.cc:260] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:57.529649   641 raft_consensus.cc:3060] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:58:57.533757   715 raft_consensus.cc:2468] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 5.
I20260709 12:58:57.534901   530 leader_election.cc:304] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:57.535723   903 raft_consensus.cc:2804] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:58:57.535799   641 raft_consensus.cc:2468] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 5.
I20260709 12:58:57.536197   903 raft_consensus.cc:697] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 5 LEADER]: Becoming Leader. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:57.537211   903 consensus_queue.cc:237] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } }
I20260709 12:58:57.545447   461 catalog_manager.cc:5697] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 reported cstate change: term changed from 4 to 5, leader changed from 8300451f93f44610a0778716b5890b71 (127.0.107.194) to d7f1c894cd2c4c11b36b837667423755 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "d7f1c894cd2c4c11b36b837667423755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:57.593139   912 raft_consensus.cc:993] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71: : Instructing follower d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:57.593871   902 raft_consensus.cc:1081] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Signalling peer d7f1c894cd2c4c11b36b837667423755 to start an election
I20260709 12:58:57.595862   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba"
dest_uuid: "d7f1c894cd2c4c11b36b837667423755"
 from {username='slave'} at 127.0.0.1:48576
I20260709 12:58:57.596441   565 raft_consensus.cc:493] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:57.596773   565 raft_consensus.cc:3060] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:57.601944   565 raft_consensus.cc:515] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.603834   565 leader_election.cc:290] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 3 election: Requested vote from peers 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795), 8f14acb5606549a4ac96fa14aacc9824 (127.0.107.195:35727)
I20260709 12:58:57.604944   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8f14acb5606549a4ac96fa14aacc9824"
I20260709 12:58:57.604873   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f13212b54b914d63b0f1852edf096eba" candidate_uuid: "d7f1c894cd2c4c11b36b837667423755" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8300451f93f44610a0778716b5890b71"
I20260709 12:58:57.605613   715 raft_consensus.cc:3060] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:57.605649   641 raft_consensus.cc:3055] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:58:57.606055   641 raft_consensus.cc:740] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8300451f93f44610a0778716b5890b71, State: Running, Role: LEADER
I20260709 12:58:57.606814   641 consensus_queue.cc:260] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.607836   641 raft_consensus.cc:3060] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:58:57.610605   715 raft_consensus.cc:2468] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 3.
I20260709 12:58:57.611771   530 leader_election.cc:304] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f14acb5606549a4ac96fa14aacc9824, d7f1c894cd2c4c11b36b837667423755; no voters: 
I20260709 12:58:57.612560   903 raft_consensus.cc:2804] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:58:57.612993   903 raft_consensus.cc:697] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 3 LEADER]: Becoming Leader. State: Replica: d7f1c894cd2c4c11b36b837667423755, State: Running, Role: LEADER
I20260709 12:58:57.613662   641 raft_consensus.cc:2468] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f1c894cd2c4c11b36b837667423755 in term 3.
I20260709 12:58:57.613899   903 consensus_queue.cc:237] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } }
I20260709 12:58:57.622032   461 catalog_manager.cc:5697] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 reported cstate change: term changed from 2 to 3, leader changed from 8300451f93f44610a0778716b5890b71 (127.0.107.194) to d7f1c894cd2c4c11b36b837667423755 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "d7f1c894cd2c4c11b36b837667423755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:57.623402   641 raft_consensus.cc:1275] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 4 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:58:57.624917   903 consensus_queue.cc:1048] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:58:57.641034   715 raft_consensus.cc:1275] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 4 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:58:57.647480   920 consensus_queue.cc:1048] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:58:57.736634   565 raft_consensus.cc:1275] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Refusing update from remote peer 8f14acb5606549a4ac96fa14aacc9824: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:57.738060   900 consensus_queue.cc:1048] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:58:57.748132   641 raft_consensus.cc:1275] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Refusing update from remote peer 8f14acb5606549a4ac96fa14aacc9824: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:57.750393   900 consensus_queue.cc:1048] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:58:57.969704   641 raft_consensus.cc:1275] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Refusing update from remote peer 8f14acb5606549a4ac96fa14aacc9824: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:57.971396   900 consensus_queue.cc:1048] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:58:57.981155   565 raft_consensus.cc:1275] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Refusing update from remote peer 8f14acb5606549a4ac96fa14aacc9824: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:58:57.983858   924 consensus_queue.cc:1048] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f1c894cd2c4c11b36b837667423755" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:58:57.994345   715 raft_consensus.cc:1275] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 5 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:58:57.995971   920 consensus_queue.cc:1048] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:58:58.006840   641 raft_consensus.cc:1275] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 5 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:58:58.009294   904 consensus_queue.cc:1048] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:58:58.043006   641 raft_consensus.cc:1275] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:58.044384   904 consensus_queue.cc:1048] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300451f93f44610a0778716b5890b71" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:58:58.054813   715 raft_consensus.cc:1275] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Refusing update from remote peer d7f1c894cd2c4c11b36b837667423755: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:58:58.057049   903 consensus_queue.cc:1048] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f14acb5606549a4ac96fa14aacc9824" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:58:58.080534   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:58.086946   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 12:58:58.116181   607 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:58:58.132790   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:58:58.132141   607 consensus_peers.cc:597] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 -> Peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Couldn't send request to peer d7f1c894cd2c4c11b36b837667423755. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:58.133945   431 tablet_replica.cc:333] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755: stopping tablet replica
I20260709 12:58:58.135072   431 raft_consensus.cc:2243] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.136044   431 raft_consensus.cc:2272] T b10c7df448e04f738e1571969c80925d P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.138970   431 tablet_replica.cc:333] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755: stopping tablet replica
I20260709 12:58:58.139617   431 raft_consensus.cc:2243] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.140132   431 raft_consensus.cc:2272] T caa377949276484ea703e81b235ee2cf P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.142249   431 tablet_replica.cc:333] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755: stopping tablet replica
I20260709 12:58:58.142838   431 raft_consensus.cc:2243] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.143344   431 raft_consensus.cc:2272] T 9043d0fea676404f9ce2edb2b42132bc P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.145361   431 tablet_replica.cc:333] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755: stopping tablet replica
I20260709 12:58:58.146101   431 raft_consensus.cc:2243] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.146593   431 raft_consensus.cc:2272] T b495693a89fb4dddbd68886b3f62f833 P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.149201   431 tablet_replica.cc:333] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755: stopping tablet replica
I20260709 12:58:58.149901   431 raft_consensus.cc:2243] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:58:58.150966   431 raft_consensus.cc:2272] T f13212b54b914d63b0f1852edf096eba P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.153326   431 tablet_replica.cc:333] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755: stopping tablet replica
I20260709 12:58:58.153954   431 raft_consensus.cc:2243] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.154465   431 raft_consensus.cc:2272] T 3b383c50beef4a54b0efa339dd0ff48e P d7f1c894cd2c4c11b36b837667423755 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.156404   431 tablet_replica.cc:333] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755: stopping tablet replica
I20260709 12:58:58.156982   431 raft_consensus.cc:2243] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:58:58.157932   431 raft_consensus.cc:2272] T 228000099d3449dcb36878d71bcdbe86 P d7f1c894cd2c4c11b36b837667423755 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.160200   431 tablet_replica.cc:333] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755: stopping tablet replica
I20260709 12:58:58.160946   431 raft_consensus.cc:2243] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:58:58.161864   431 raft_consensus.cc:2272] T 106212907bb64e75a6a5204fae36c61f P d7f1c894cd2c4c11b36b837667423755 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.164057   431 tablet_replica.cc:333] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755: stopping tablet replica
I20260709 12:58:58.164573   431 raft_consensus.cc:2243] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.165079   431 raft_consensus.cc:2272] T 5e9f83b7307c45fda8df8c9fbc844c4a P d7f1c894cd2c4c11b36b837667423755 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.289191   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 12:58:58.312157   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 12:58:58.341683   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:58:58.342638   431 tablet_replica.cc:333] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71: stopping tablet replica
I20260709 12:58:58.343411   431 raft_consensus.cc:2243] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.344008   431 raft_consensus.cc:2272] T 9043d0fea676404f9ce2edb2b42132bc P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.346367   431 tablet_replica.cc:333] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71: stopping tablet replica
I20260709 12:58:58.346982   431 raft_consensus.cc:2243] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.347457   431 raft_consensus.cc:2272] T b495693a89fb4dddbd68886b3f62f833 P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:58:58.348747   681 consensus_peers.cc:597] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 -> Peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Couldn't send request to peer d7f1c894cd2c4c11b36b837667423755. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:58.350306   431 tablet_replica.cc:333] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71: stopping tablet replica
I20260709 12:58:58.350894   431 raft_consensus.cc:2243] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:58:58.351722   431 raft_consensus.cc:2272] T b10c7df448e04f738e1571969c80925d P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.353597   431 tablet_replica.cc:333] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71: stopping tablet replica
I20260709 12:58:58.354138   431 raft_consensus.cc:2243] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:58:58.355019   431 raft_consensus.cc:2272] T caa377949276484ea703e81b235ee2cf P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.357092   431 tablet_replica.cc:333] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71: stopping tablet replica
I20260709 12:58:58.357676   431 raft_consensus.cc:2243] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.358114   431 raft_consensus.cc:2272] T 106212907bb64e75a6a5204fae36c61f P 8300451f93f44610a0778716b5890b71 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.360692   431 tablet_replica.cc:333] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71: stopping tablet replica
I20260709 12:58:58.361366   431 raft_consensus.cc:2243] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.361919   431 raft_consensus.cc:2272] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.364437   431 tablet_replica.cc:333] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71: stopping tablet replica
I20260709 12:58:58.365061   431 raft_consensus.cc:2243] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.365581   431 raft_consensus.cc:2272] T f13212b54b914d63b0f1852edf096eba P 8300451f93f44610a0778716b5890b71 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.368197   431 tablet_replica.cc:333] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71: stopping tablet replica
I20260709 12:58:58.368782   431 raft_consensus.cc:2243] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:58:58.369654   431 raft_consensus.cc:2272] T 3b383c50beef4a54b0efa339dd0ff48e P 8300451f93f44610a0778716b5890b71 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.371694   431 tablet_replica.cc:333] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71: stopping tablet replica
I20260709 12:58:58.372225   431 raft_consensus.cc:2243] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.372720   431 raft_consensus.cc:2272] T 228000099d3449dcb36878d71bcdbe86 P 8300451f93f44610a0778716b5890b71 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:58:58.381357   681 consensus_peers.cc:597] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 -> Peer 8300451f93f44610a0778716b5890b71 (127.0.107.194:41795): Couldn't send request to peer 8300451f93f44610a0778716b5890b71. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:58.406177   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 12:58:58.432142   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 12:58:58.431973   681 consensus_peers.cc:597] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 -> Peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Couldn't send request to peer d7f1c894cd2c4c11b36b837667423755. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:58.460139   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:58:58.460922   431 tablet_replica.cc:333] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824: stopping tablet replica
I20260709 12:58:58.461670   431 raft_consensus.cc:2243] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.462206   431 raft_consensus.cc:2272] T 106212907bb64e75a6a5204fae36c61f P 8f14acb5606549a4ac96fa14aacc9824 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.464505   431 tablet_replica.cc:333] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824: stopping tablet replica
I20260709 12:58:58.465044   431 raft_consensus.cc:2243] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:58:58.465909   431 raft_consensus.cc:2272] T 5e9f83b7307c45fda8df8c9fbc844c4a P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.468801   431 tablet_replica.cc:333] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824: stopping tablet replica
I20260709 12:58:58.469569   431 raft_consensus.cc:2243] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.470114   431 raft_consensus.cc:2272] T 3b383c50beef4a54b0efa339dd0ff48e P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:58:58.471316   681 consensus_peers.cc:597] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 -> Peer d7f1c894cd2c4c11b36b837667423755 (127.0.107.193:45333): Couldn't send request to peer d7f1c894cd2c4c11b36b837667423755. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:58.472043   431 tablet_replica.cc:333] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824: stopping tablet replica
I20260709 12:58:58.472579   431 raft_consensus.cc:2243] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.473744   431 raft_consensus.cc:2272] T f13212b54b914d63b0f1852edf096eba P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.475809   431 tablet_replica.cc:333] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824: stopping tablet replica
I20260709 12:58:58.476339   431 raft_consensus.cc:2243] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.476783   431 raft_consensus.cc:2272] T 228000099d3449dcb36878d71bcdbe86 P 8f14acb5606549a4ac96fa14aacc9824 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.478945   431 tablet_replica.cc:333] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824: stopping tablet replica
I20260709 12:58:58.479444   431 raft_consensus.cc:2243] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:58:58.480221   431 raft_consensus.cc:2272] T 9043d0fea676404f9ce2edb2b42132bc P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.482542   431 tablet_replica.cc:333] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824: stopping tablet replica
I20260709 12:58:58.483094   431 raft_consensus.cc:2243] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:58:58.483942   431 raft_consensus.cc:2272] T b495693a89fb4dddbd68886b3f62f833 P 8f14acb5606549a4ac96fa14aacc9824 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.485873   431 tablet_replica.cc:333] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824: stopping tablet replica
I20260709 12:58:58.486373   431 raft_consensus.cc:2243] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.486815   431 raft_consensus.cc:2272] T caa377949276484ea703e81b235ee2cf P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.488547   431 tablet_replica.cc:333] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824: stopping tablet replica
I20260709 12:58:58.488998   431 raft_consensus.cc:2243] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:58.489428   431 raft_consensus.cc:2272] T b10c7df448e04f738e1571969c80925d P 8f14acb5606549a4ac96fa14aacc9824 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.517959   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 12:58:58.538285   431 master.cc:562] Master@127.0.107.254:38159 shutting down...
I20260709 12:58:58.728842   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:58:58.729646   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:58.730027   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P faeef2d50afb4f5a8745a7c78b38b9f1: stopping tablet replica
I20260709 12:58:58.750850   431 master.cc:584] Master@127.0.107.254:38159 shutdown complete.
8300451f93f44610a0778716b5890b71  4
8f14acb5606549a4ac96fa14aacc9824  1
d7f1c894cd2c4c11b36b837667423755  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (13673 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:58:58.796717   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:45121
I20260709 12:58:58.798097   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:58.804567   935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:58.804942   936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:58.806186   938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:58.806438   431 server_base.cc:1061] running on GCE node
I20260709 12:58:58.807479   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:58.807690   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:58.807860   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601938807842 us; error 0 us; skew 500 ppm
I20260709 12:58:58.808461   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:58.811196   431 webserver.cc:533] Webserver started at http://127.0.107.254:43561/ using document root <none> and password file <none>
I20260709 12:58:58.811792   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:58.812016   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:58.812417   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:58.814656   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/master-0-root/instance:
uuid: "9d54dd2dafae4820b5d3b89770f30c2a"
format_stamp: "Formatted at 2026-07-09 12:58:58 on dist-test-slave-l8g5"
I20260709 12:58:58.821661   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.003s
I20260709 12:58:58.825682   943 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:58.826627   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:58:58.826963   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/master-0-root
uuid: "9d54dd2dafae4820b5d3b89770f30c2a"
format_stamp: "Formatted at 2026-07-09 12:58:58 on dist-test-slave-l8g5"
I20260709 12:58:58.827360   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:58.848738   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:58.850555   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:58.899923   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:45121
I20260709 12:58:58.900025   994 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:45121 every 8 connection(s)
I20260709 12:58:58.904557   995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.915428   995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a: Bootstrap starting.
I20260709 12:58:58.924808   995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.929939   995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a: No bootstrap required, opened a new log
I20260709 12:58:58.932360   995 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d54dd2dafae4820b5d3b89770f30c2a" member_type: VOTER }
I20260709 12:58:58.932830   995 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.933028   995 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d54dd2dafae4820b5d3b89770f30c2a, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.933687   995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d54dd2dafae4820b5d3b89770f30c2a" member_type: VOTER }
I20260709 12:58:58.934131   995 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:58:58.934343   995 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:58:58.934566   995 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.008339   995 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d54dd2dafae4820b5d3b89770f30c2a" member_type: VOTER }
I20260709 12:58:59.009285   995 leader_election.cc:304] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d54dd2dafae4820b5d3b89770f30c2a; no voters: 
I20260709 12:58:59.011361   995 leader_election.cc:290] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:58:59.011536   998 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.013942   998 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [term 1 LEADER]: Becoming Leader. State: Replica: 9d54dd2dafae4820b5d3b89770f30c2a, State: Running, Role: LEADER
I20260709 12:58:59.014919   998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d54dd2dafae4820b5d3b89770f30c2a" member_type: VOTER }
I20260709 12:58:59.015334   995 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:58:59.021159   999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d54dd2dafae4820b5d3b89770f30c2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d54dd2dafae4820b5d3b89770f30c2a" member_type: VOTER } }
I20260709 12:58:59.021275  1000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d54dd2dafae4820b5d3b89770f30c2a. Latest consensus state: current_term: 1 leader_uuid: "9d54dd2dafae4820b5d3b89770f30c2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d54dd2dafae4820b5d3b89770f30c2a" member_type: VOTER } }
I20260709 12:58:59.022208   999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [sys.catalog]: This master's current role is: LEADER
I20260709 12:58:59.022262  1000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [sys.catalog]: This master's current role is: LEADER
I20260709 12:58:59.024935  1005 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:58:59.030493  1005 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:58:59.036477   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:58:59.040452  1005 catalog_manager.cc:1383] Generated new cluster ID: 17a42b38718f4057b7185565c861b65f
I20260709 12:58:59.040786  1005 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:58:59.065124  1005 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:58:59.066803  1005 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:58:59.086766  1005 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a: Generated new TSK 0
I20260709 12:58:59.087539  1005 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:58:59.104117   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:59.111013  1016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:59.112154  1017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:59.115285  1019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:59.115440   431 server_base.cc:1061] running on GCE node
I20260709 12:58:59.116706   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:59.117753   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:59.117940   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601939117902 us; error 0 us; skew 500 ppm
I20260709 12:58:59.118534   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:59.121148   431 webserver.cc:533] Webserver started at http://127.0.107.193:35939/ using document root <none> and password file <none>
I20260709 12:58:59.121731   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:59.121963   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:59.122247   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:59.123425   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-0-root/instance:
uuid: "bc53c40ea20b40dda6d9516fa180772d"
format_stamp: "Formatted at 2026-07-09 12:58:59 on dist-test-slave-l8g5"
I20260709 12:58:59.128379   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:58:59.132207  1024 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:59.133093   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:58:59.133417   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-0-root
uuid: "bc53c40ea20b40dda6d9516fa180772d"
format_stamp: "Formatted at 2026-07-09 12:58:59 on dist-test-slave-l8g5"
I20260709 12:58:59.133764   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:59.159055   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:59.160845   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:59.162770   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:59.165402   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:59.165632   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:58:59.165984   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:59.166147   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:59.217505   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:40871
I20260709 12:58:59.217633  1086 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:40871 every 8 connection(s)
I20260709 12:58:59.226778   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:59.235564  1091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:59.236649  1092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:59.238904  1094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:59.240563   431 server_base.cc:1061] running on GCE node
I20260709 12:58:59.241648   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:59.241865   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:59.242048   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601939242010 us; error 0 us; skew 500 ppm
I20260709 12:58:59.242647   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:59.245983   431 webserver.cc:533] Webserver started at http://127.0.107.194:36555/ using document root <none> and password file <none>
I20260709 12:58:59.246855   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:59.247202   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:59.247586   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:59.248159  1087 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45121
I20260709 12:58:59.248569  1087 heartbeater.cc:461] Registering TS with master...
I20260709 12:58:59.249255   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-1-root/instance:
uuid: "9afc3caa620240e7a5a061700c100ea3"
format_stamp: "Formatted at 2026-07-09 12:58:59 on dist-test-slave-l8g5"
I20260709 12:58:59.249347  1087 heartbeater.cc:507] Master 127.0.107.254:45121 requested a full tablet report, sending...
I20260709 12:58:59.252759   960 ts_manager.cc:194] Registered new tserver with Master: bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871)
I20260709 12:58:59.255499   960 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48888
I20260709 12:58:59.255962   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:58:59.260721  1099 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:59.261726   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:58:59.262076   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-1-root
uuid: "9afc3caa620240e7a5a061700c100ea3"
format_stamp: "Formatted at 2026-07-09 12:58:59 on dist-test-slave-l8g5"
I20260709 12:58:59.262454   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:59.274420   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:59.275925   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:59.277812   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:59.280992   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:59.281263   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:59.281586   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:59.281790   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:59.337604   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:39791
I20260709 12:58:59.337704  1161 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:39791 every 8 connection(s)
I20260709 12:58:59.344818   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:59.353293  1165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:59.354120  1166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:59.361522  1162 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45121
W20260709 12:58:59.362057  1168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:59.361235   431 server_base.cc:1061] running on GCE node
I20260709 12:58:59.366238  1162 heartbeater.cc:461] Registering TS with master...
I20260709 12:58:59.367242   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:59.367583   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:59.367524  1162 heartbeater.cc:507] Master 127.0.107.254:45121 requested a full tablet report, sending...
I20260709 12:58:59.367949   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601939367923 us; error 0 us; skew 500 ppm
I20260709 12:58:59.368884   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:59.370525   960 ts_manager.cc:194] Registered new tserver with Master: 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791)
I20260709 12:58:59.372053   960 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48904
I20260709 12:58:59.372448   431 webserver.cc:533] Webserver started at http://127.0.107.195:34607/ using document root <none> and password file <none>
I20260709 12:58:59.373059   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:59.373327   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:59.373835   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:59.375334   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-2-root/instance:
uuid: "629e1bd745fb4bce886bc01856587fa3"
format_stamp: "Formatted at 2026-07-09 12:58:59 on dist-test-slave-l8g5"
I20260709 12:58:59.380389   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:58:59.384248  1173 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:59.385283   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:58:59.385663   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-2-root
uuid: "629e1bd745fb4bce886bc01856587fa3"
format_stamp: "Formatted at 2026-07-09 12:58:59 on dist-test-slave-l8g5"
I20260709 12:58:59.386008   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601924925497-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:59.398682   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:59.400210   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:59.402063   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:59.404655   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:59.404922   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:59.405212   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:59.405424   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:59.456774   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:33047
I20260709 12:58:59.456874  1235 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:33047 every 8 connection(s)
I20260709 12:58:59.473862  1236 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45121
I20260709 12:58:59.474373  1236 heartbeater.cc:461] Registering TS with master...
I20260709 12:58:59.475342  1236 heartbeater.cc:507] Master 127.0.107.254:45121 requested a full tablet report, sending...
I20260709 12:58:59.478133   960 ts_manager.cc:194] Registered new tserver with Master: 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:58:59.478415   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017624895s
I20260709 12:58:59.479750   960 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48916
I20260709 12:58:59.510056   960 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48920:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:58:59.513375   960 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:58:59.594945  1200 tablet_service.cc:1511] Processing CreateTablet for tablet daa76bc7ba174327b42ce50718d0123d (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:58:59.597031  1200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daa76bc7ba174327b42ce50718d0123d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.599982  1201 tablet_service.cc:1511] Processing CreateTablet for tablet 7ed84eb1a8eb4ab08fb0353f201560c3 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:58:59.604204  1201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ed84eb1a8eb4ab08fb0353f201560c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.603892  1198 tablet_service.cc:1511] Processing CreateTablet for tablet 5058095638664e428f3cf97e53848db2 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:58:59.605373  1127 tablet_service.cc:1511] Processing CreateTablet for tablet 7ed84eb1a8eb4ab08fb0353f201560c3 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:58:59.603894  1199 tablet_service.cc:1511] Processing CreateTablet for tablet f5c183e9d3a64539b7ac6915ae583a4b (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:58:59.606902  1127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ed84eb1a8eb4ab08fb0353f201560c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.607452  1198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5058095638664e428f3cf97e53848db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.610780  1126 tablet_service.cc:1511] Processing CreateTablet for tablet daa76bc7ba174327b42ce50718d0123d (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:58:59.612051  1199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c183e9d3a64539b7ac6915ae583a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.612342  1126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daa76bc7ba174327b42ce50718d0123d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.614810  1125 tablet_service.cc:1511] Processing CreateTablet for tablet f5c183e9d3a64539b7ac6915ae583a4b (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:58:59.616370  1125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c183e9d3a64539b7ac6915ae583a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.624015  1124 tablet_service.cc:1511] Processing CreateTablet for tablet 5058095638664e428f3cf97e53848db2 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:58:59.625662  1124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5058095638664e428f3cf97e53848db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.637607  1052 tablet_service.cc:1511] Processing CreateTablet for tablet 7ed84eb1a8eb4ab08fb0353f201560c3 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:58:59.639194  1052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ed84eb1a8eb4ab08fb0353f201560c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.647586  1051 tablet_service.cc:1511] Processing CreateTablet for tablet daa76bc7ba174327b42ce50718d0123d (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:58:59.649129  1051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daa76bc7ba174327b42ce50718d0123d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.651796  1251 tablet_bootstrap.cc:492] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3: Bootstrap starting.
I20260709 12:58:59.657583  1200 tablet_service.cc:1511] Processing CreateTablet for tablet 81567809e6c1412d88a5becc05f305ab (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:58:59.659790  1050 tablet_service.cc:1511] Processing CreateTablet for tablet f5c183e9d3a64539b7ac6915ae583a4b (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:58:59.661389  1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c183e9d3a64539b7ac6915ae583a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.661386  1200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81567809e6c1412d88a5becc05f305ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.668574  1251 tablet_bootstrap.cc:654] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.673723  1198 tablet_service.cc:1511] Processing CreateTablet for tablet 262d0445f78c4bc5a1da87074cfd1101 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:58:59.680126  1127 tablet_service.cc:1511] Processing CreateTablet for tablet 81567809e6c1412d88a5becc05f305ab (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:58:59.681057  1198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 262d0445f78c4bc5a1da87074cfd1101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.681506  1127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81567809e6c1412d88a5becc05f305ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.685380  1047 tablet_service.cc:1511] Processing CreateTablet for tablet 262d0445f78c4bc5a1da87074cfd1101 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:58:59.687119  1047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 262d0445f78c4bc5a1da87074cfd1101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.688651  1196 tablet_service.cc:1511] Processing CreateTablet for tablet 4bb1cda199a84d5bafed13f1a1d44b7b (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:58:59.690135  1196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bb1cda199a84d5bafed13f1a1d44b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.694280  1126 tablet_service.cc:1511] Processing CreateTablet for tablet 262d0445f78c4bc5a1da87074cfd1101 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:58:59.695092  1195 tablet_service.cc:1511] Processing CreateTablet for tablet 93594eee86c84ebd8bba2c1d2a934ed6 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:58:59.695840  1126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 262d0445f78c4bc5a1da87074cfd1101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.696528  1195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93594eee86c84ebd8bba2c1d2a934ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.678162  1197 tablet_service.cc:1511] Processing CreateTablet for tablet 362f1af397b74a879245d9616bacf029 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:58:59.698356  1197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362f1af397b74a879245d9616bacf029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.678162  1048 tablet_service.cc:1511] Processing CreateTablet for tablet 81567809e6c1412d88a5becc05f305ab (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:58:59.695431  1194 tablet_service.cc:1511] Processing CreateTablet for tablet df3d601732374c73ad6f1b5a15956de9 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:58:59.700016  1194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df3d601732374c73ad6f1b5a15956de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.703171  1125 tablet_service.cc:1511] Processing CreateTablet for tablet 4bb1cda199a84d5bafed13f1a1d44b7b (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:58:59.703739  1124 tablet_service.cc:1511] Processing CreateTablet for tablet 362f1af397b74a879245d9616bacf029 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:58:59.704648  1125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bb1cda199a84d5bafed13f1a1d44b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.675688  1049 tablet_service.cc:1511] Processing CreateTablet for tablet 5058095638664e428f3cf97e53848db2 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:58:59.706225  1124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362f1af397b74a879245d9616bacf029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.707978  1122 tablet_service.cc:1511] Processing CreateTablet for tablet df3d601732374c73ad6f1b5a15956de9 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:58:59.708379  1123 tablet_service.cc:1511] Processing CreateTablet for tablet 93594eee86c84ebd8bba2c1d2a934ed6 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:58:59.711638  1045 tablet_service.cc:1511] Processing CreateTablet for tablet 4bb1cda199a84d5bafed13f1a1d44b7b (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:58:59.706473  1048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81567809e6c1412d88a5becc05f305ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.716388  1043 tablet_service.cc:1511] Processing CreateTablet for tablet df3d601732374c73ad6f1b5a15956de9 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:58:59.719152  1123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93594eee86c84ebd8bba2c1d2a934ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.719739  1044 tablet_service.cc:1511] Processing CreateTablet for tablet 93594eee86c84ebd8bba2c1d2a934ed6 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:58:59.725639  1046 tablet_service.cc:1511] Processing CreateTablet for tablet 362f1af397b74a879245d9616bacf029 (DEFAULT_TABLE table=test-workload [id=6ee529e9fbb94fdba098cd99803c17ed]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:58:59.730046  1044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93594eee86c84ebd8bba2c1d2a934ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.733873  1046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362f1af397b74a879245d9616bacf029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.734622  1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df3d601732374c73ad6f1b5a15956de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.735455  1122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df3d601732374c73ad6f1b5a15956de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.735814  1045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bb1cda199a84d5bafed13f1a1d44b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.737831  1049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5058095638664e428f3cf97e53848db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:59.786898  1251 tablet_bootstrap.cc:492] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3: No bootstrap required, opened a new log
I20260709 12:58:59.787670  1251 ts_tablet_manager.cc:1403] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3: Time spent bootstrapping tablet: real 0.138s	user 0.010s	sys 0.017s
I20260709 12:58:59.791157  1251 raft_consensus.cc:359] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:58:59.792073  1251 raft_consensus.cc:385] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.792378  1251 raft_consensus.cc:740] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.786895  1252 tablet_bootstrap.cc:492] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3: Bootstrap starting.
I20260709 12:58:59.800343  1252 tablet_bootstrap.cc:654] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.801862  1251 consensus_queue.cc:260] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:58:59.850754  1254 tablet_bootstrap.cc:492] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d: Bootstrap starting.
I20260709 12:58:59.866812  1162 heartbeater.cc:499] Master 127.0.107.254:45121 was elected leader, sending a full tablet report...
I20260709 12:58:59.868258  1251 ts_tablet_manager.cc:1434] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3: Time spent starting tablet: real 0.080s	user 0.017s	sys 0.011s
I20260709 12:58:59.907437  1252 tablet_bootstrap.cc:492] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3: No bootstrap required, opened a new log
I20260709 12:58:59.907552  1251 tablet_bootstrap.cc:492] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3: Bootstrap starting.
I20260709 12:58:59.908116  1252 ts_tablet_manager.cc:1403] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3: Time spent bootstrapping tablet: real 0.123s	user 0.030s	sys 0.028s
I20260709 12:58:59.911435  1252 raft_consensus.cc:359] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:58:59.912220  1252 raft_consensus.cc:385] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.912530  1252 raft_consensus.cc:740] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.915755  1254 tablet_bootstrap.cc:654] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.921214  1251 tablet_bootstrap.cc:654] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.922501  1252 consensus_queue.cc:260] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:58:59.929323  1236 heartbeater.cc:499] Master 127.0.107.254:45121 was elected leader, sending a full tablet report...
I20260709 12:58:59.931381  1252 ts_tablet_manager.cc:1434] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.006s
I20260709 12:58:59.937011  1251 tablet_bootstrap.cc:492] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3: No bootstrap required, opened a new log
I20260709 12:58:59.937512  1252 tablet_bootstrap.cc:492] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3: Bootstrap starting.
I20260709 12:58:59.937637  1251 ts_tablet_manager.cc:1403] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.007s
I20260709 12:58:59.940987  1251 raft_consensus.cc:359] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:58:59.937029  1254 tablet_bootstrap.cc:492] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d: No bootstrap required, opened a new log
I20260709 12:58:59.941785  1251 raft_consensus.cc:385] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.942147  1251 raft_consensus.cc:740] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.942337  1254 ts_tablet_manager.cc:1403] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d: Time spent bootstrapping tablet: real 0.092s	user 0.009s	sys 0.008s
I20260709 12:58:59.942998  1251 consensus_queue.cc:260] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:58:59.945652  1252 tablet_bootstrap.cc:654] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.945755  1251 ts_tablet_manager.cc:1434] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:58:59.946942  1251 tablet_bootstrap.cc:492] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3: Bootstrap starting.
I20260709 12:58:59.951411  1254 raft_consensus.cc:359] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } }
I20260709 12:58:59.952435  1254 raft_consensus.cc:385] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.952804  1254 raft_consensus.cc:740] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.953861  1251 tablet_bootstrap.cc:654] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.953802  1254 consensus_queue.cc:260] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } }
I20260709 12:58:59.974021  1087 heartbeater.cc:499] Master 127.0.107.254:45121 was elected leader, sending a full tablet report...
I20260709 12:58:59.974695  1252 tablet_bootstrap.cc:492] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3: No bootstrap required, opened a new log
I20260709 12:58:59.974716  1251 tablet_bootstrap.cc:492] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3: No bootstrap required, opened a new log
I20260709 12:58:59.975473  1252 ts_tablet_manager.cc:1403] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.020s
I20260709 12:58:59.975529  1251 ts_tablet_manager.cc:1403] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.012s
I20260709 12:58:59.978996  1252 raft_consensus.cc:359] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:58:59.979766  1252 raft_consensus.cc:385] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.980276  1252 raft_consensus.cc:740] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.979735  1251 raft_consensus.cc:359] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:58:59.980683  1251 raft_consensus.cc:385] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.981030  1251 raft_consensus.cc:740] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.981084  1252 consensus_queue.cc:260] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:58:59.982205  1255 raft_consensus.cc:493] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.981962  1251 consensus_queue.cc:260] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:58:59.982695  1255 raft_consensus.cc:515] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:58:59.987828  1252 ts_tablet_manager.cc:1434] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 12:58:59.989003  1252 tablet_bootstrap.cc:492] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3: Bootstrap starting.
I20260709 12:58:59.993954  1254 ts_tablet_manager.cc:1434] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.016s
I20260709 12:58:59.995551  1254 tablet_bootstrap.cc:492] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d: Bootstrap starting.
I20260709 12:58:59.997156  1252 tablet_bootstrap.cc:654] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.997632  1255 leader_election.cc:290] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.014632  1254 tablet_bootstrap.cc:654] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.023108  1252 tablet_bootstrap.cc:492] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3: No bootstrap required, opened a new log
I20260709 12:59:00.023712  1252 ts_tablet_manager.cc:1403] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.005s
I20260709 12:59:00.025020  1254 tablet_bootstrap.cc:492] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d: No bootstrap required, opened a new log
I20260709 12:59:00.025750  1254 ts_tablet_manager.cc:1403] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.001s
I20260709 12:59:00.027405  1252 raft_consensus.cc:359] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.028247  1252 raft_consensus.cc:385] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:00.031083  1008 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:00.031096  1254 raft_consensus.cc:359] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.028795  1252 raft_consensus.cc:740] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.036034  1252 consensus_queue.cc:260] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.033212  1254 raft_consensus.cc:385] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.039263  1254 raft_consensus.cc:740] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.040194  1254 consensus_queue.cc:260] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.042760  1252 ts_tablet_manager.cc:1434] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.000s
I20260709 12:59:00.043987  1252 tablet_bootstrap.cc:492] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3: Bootstrap starting.
I20260709 12:59:00.049288  1254 ts_tablet_manager.cc:1434] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 12:59:00.049789  1251 ts_tablet_manager.cc:1434] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3: Time spent starting tablet: real 0.074s	user 0.026s	sys 0.046s
I20260709 12:59:00.050420  1254 tablet_bootstrap.cc:492] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d: Bootstrap starting.
I20260709 12:59:00.050882  1251 tablet_bootstrap.cc:492] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3: Bootstrap starting.
I20260709 12:59:00.057610  1251 tablet_bootstrap.cc:654] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.063858  1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.064235  1252 tablet_bootstrap.cc:654] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.064749  1211 raft_consensus.cc:2468] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 0.
I20260709 12:59:00.065784  1254 tablet_bootstrap.cc:654] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.066596  1101 leader_election.cc:304] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: 
I20260709 12:59:00.068686  1263 raft_consensus.cc:2804] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:00.072968  1263 raft_consensus.cc:493] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:00.073836  1263 raft_consensus.cc:3060] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.074398  1251 tablet_bootstrap.cc:492] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3: No bootstrap required, opened a new log
I20260709 12:59:00.075145  1251 ts_tablet_manager.cc:1403] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.010s
I20260709 12:59:00.079520  1251 raft_consensus.cc:359] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } }
I20260709 12:59:00.080598  1251 raft_consensus.cc:385] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.081056  1251 raft_consensus.cc:740] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.083307  1263 raft_consensus.cc:515] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.084214  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d" is_pre_election: true
I20260709 12:59:00.084255  1251 consensus_queue.cc:260] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } }
W20260709 12:59:00.086638  1100 leader_election.cc:343] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:00.089136  1263 leader_election.cc:290] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.089139  1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.090404  1211 raft_consensus.cc:3060] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.091203  1251 ts_tablet_manager.cc:1434] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:59:00.092378  1251 tablet_bootstrap.cc:492] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3: Bootstrap starting.
I20260709 12:59:00.093524  1252 tablet_bootstrap.cc:492] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3: No bootstrap required, opened a new log
I20260709 12:59:00.096092  1252 ts_tablet_manager.cc:1403] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.004s
I20260709 12:59:00.096658  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
W20260709 12:59:00.098366  1100 leader_election.cc:343] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:00.099473  1251 tablet_bootstrap.cc:654] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.099687  1252 raft_consensus.cc:359] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } }
I20260709 12:59:00.100512  1252 raft_consensus.cc:385] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.100819  1252 raft_consensus.cc:740] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.101644  1252 consensus_queue.cc:260] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } }
I20260709 12:59:00.104025  1252 ts_tablet_manager.cc:1434] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:59:00.105075  1252 tablet_bootstrap.cc:492] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3: Bootstrap starting.
I20260709 12:59:00.105957  1211 raft_consensus.cc:2468] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 1.
I20260709 12:59:00.107311  1101 leader_election.cc:304] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:00.108175  1263 raft_consensus.cc:2804] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:00.109531  1263 raft_consensus.cc:697] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Leader. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:00.110240  1263 consensus_queue.cc:237] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.112839  1251 tablet_bootstrap.cc:492] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3: No bootstrap required, opened a new log
I20260709 12:59:00.113659  1251 ts_tablet_manager.cc:1403] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 12:59:00.117678  1251 raft_consensus.cc:359] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.118796  1251 raft_consensus.cc:385] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.119295  1251 raft_consensus.cc:740] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.120306  1251 consensus_queue.cc:260] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.121170  1252 tablet_bootstrap.cc:654] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.123152  1254 tablet_bootstrap.cc:492] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d: No bootstrap required, opened a new log
I20260709 12:59:00.123572  1251 ts_tablet_manager.cc:1434] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:59:00.123770  1254 ts_tablet_manager.cc:1403] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d: Time spent bootstrapping tablet: real 0.074s	user 0.009s	sys 0.025s
I20260709 12:59:00.124783  1251 tablet_bootstrap.cc:492] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3: Bootstrap starting.
I20260709 12:59:00.126962  1254 raft_consensus.cc:359] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.127789  1254 raft_consensus.cc:385] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.128125  1254 raft_consensus.cc:740] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.129380  1254 consensus_queue.cc:260] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.132711  1251 tablet_bootstrap.cc:654] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.132782   960 catalog_manager.cc:5697] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "9afc3caa620240e7a5a061700c100ea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.135244  1254 ts_tablet_manager.cc:1434] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:59:00.136473  1254 tablet_bootstrap.cc:492] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d: Bootstrap starting.
I20260709 12:59:00.160714  1254 tablet_bootstrap.cc:654] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.161510  1252 tablet_bootstrap.cc:492] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3: No bootstrap required, opened a new log
I20260709 12:59:00.162074  1252 ts_tablet_manager.cc:1403] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3: Time spent bootstrapping tablet: real 0.057s	user 0.019s	sys 0.027s
I20260709 12:59:00.162729  1251 tablet_bootstrap.cc:492] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3: No bootstrap required, opened a new log
I20260709 12:59:00.163403  1251 ts_tablet_manager.cc:1403] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3: Time spent bootstrapping tablet: real 0.039s	user 0.024s	sys 0.010s
I20260709 12:59:00.165620  1252 raft_consensus.cc:359] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.166437  1252 raft_consensus.cc:385] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.166736  1252 raft_consensus.cc:740] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.167814  1252 consensus_queue.cc:260] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.168264  1251 raft_consensus.cc:359] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.169337  1251 raft_consensus.cc:385] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.169736  1251 raft_consensus.cc:740] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.170591  1252 ts_tablet_manager.cc:1434] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:59:00.170771  1251 consensus_queue.cc:260] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.171651  1252 tablet_bootstrap.cc:492] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3: Bootstrap starting.
I20260709 12:59:00.173625  1251 ts_tablet_manager.cc:1434] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:59:00.174782  1251 tablet_bootstrap.cc:492] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3: Bootstrap starting.
I20260709 12:59:00.178030  1252 tablet_bootstrap.cc:654] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.182337  1251 tablet_bootstrap.cc:654] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.187873  1263 raft_consensus.cc:493] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.188464  1263 raft_consensus.cc:515] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.189602  1254 tablet_bootstrap.cc:492] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d: No bootstrap required, opened a new log
I20260709 12:59:00.190407  1254 ts_tablet_manager.cc:1403] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d: Time spent bootstrapping tablet: real 0.054s	user 0.011s	sys 0.012s
I20260709 12:59:00.192167  1263 leader_election.cc:290] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.192453  1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93594eee86c84ebd8bba2c1d2a934ed6" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.193248  1211 raft_consensus.cc:2468] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 0.
I20260709 12:59:00.194307  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93594eee86c84ebd8bba2c1d2a934ed6" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d" is_pre_election: true
I20260709 12:59:00.194860  1254 raft_consensus.cc:359] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.195631  1254 raft_consensus.cc:385] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:00.195849  1100 leader_election.cc:343] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:00.196003  1254 raft_consensus.cc:740] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.197117  1101 leader_election.cc:304] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:00.196893  1254 consensus_queue.cc:260] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.198540  1263 raft_consensus.cc:2804] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:00.198989  1263 raft_consensus.cc:493] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:00.199355  1263 raft_consensus.cc:3060] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.199717  1254 ts_tablet_manager.cc:1434] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:00.200682  1251 tablet_bootstrap.cc:492] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3: No bootstrap required, opened a new log
I20260709 12:59:00.200868  1254 tablet_bootstrap.cc:492] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d: Bootstrap starting.
I20260709 12:59:00.201243  1251 ts_tablet_manager.cc:1403] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.002s
I20260709 12:59:00.204811  1251 raft_consensus.cc:359] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.205752  1251 raft_consensus.cc:385] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.206054  1251 raft_consensus.cc:740] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.206943  1251 consensus_queue.cc:260] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.207639  1263 raft_consensus.cc:515] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.208962  1254 tablet_bootstrap.cc:654] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.210626  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93594eee86c84ebd8bba2c1d2a934ed6" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:00.211897  1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93594eee86c84ebd8bba2c1d2a934ed6" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.212507  1211 raft_consensus.cc:3060] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:00.212777  1100 leader_election.cc:343] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:00.213836  1251 ts_tablet_manager.cc:1434] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 12:59:00.215044  1251 tablet_bootstrap.cc:492] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3: Bootstrap starting.
I20260709 12:59:00.221805  1211 raft_consensus.cc:2468] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 1.
I20260709 12:59:00.223198  1101 leader_election.cc:304] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:00.224089  1255 raft_consensus.cc:2804] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:00.224786  1254 tablet_bootstrap.cc:492] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d: No bootstrap required, opened a new log
I20260709 12:59:00.225363  1254 ts_tablet_manager.cc:1403] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.003s
I20260709 12:59:00.225714  1252 tablet_bootstrap.cc:492] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3: No bootstrap required, opened a new log
I20260709 12:59:00.226253  1252 ts_tablet_manager.cc:1403] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.006s
I20260709 12:59:00.228763  1254 raft_consensus.cc:359] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.229743  1254 raft_consensus.cc:385] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.230019  1263 leader_election.cc:290] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.230139  1254 raft_consensus.cc:740] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.230482  1251 tablet_bootstrap.cc:654] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.231137  1263 raft_consensus.cc:493] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.231271  1254 consensus_queue.cc:260] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.231655  1263 raft_consensus.cc:515] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.234741  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daa76bc7ba174327b42ce50718d0123d" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d" is_pre_election: true
I20260709 12:59:00.235620  1062 raft_consensus.cc:2468] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 0.
I20260709 12:59:00.229166  1252 raft_consensus.cc:359] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.236905  1252 raft_consensus.cc:385] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.237221  1252 raft_consensus.cc:740] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.236527  1254 ts_tablet_manager.cc:1434] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.005s
I20260709 12:59:00.237344  1100 leader_election.cc:304] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.238808  1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daa76bc7ba174327b42ce50718d0123d" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.239161  1252 consensus_queue.cc:260] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.238808  1254 tablet_bootstrap.cc:492] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d: Bootstrap starting.
I20260709 12:59:00.241914  1251 tablet_bootstrap.cc:492] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3: No bootstrap required, opened a new log
I20260709 12:59:00.240231  1263 leader_election.cc:290] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.242409  1251 ts_tablet_manager.cc:1403] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.004s
I20260709 12:59:00.239563  1211 raft_consensus.cc:2468] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 0.
I20260709 12:59:00.242475  1263 raft_consensus.cc:493] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.244567  1263 raft_consensus.cc:515] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.248667  1255 raft_consensus.cc:697] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Leader. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:00.249588  1252 ts_tablet_manager.cc:1434] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:59:00.249678  1255 consensus_queue.cc:237] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.251277  1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362f1af397b74a879245d9616bacf029" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.251610  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362f1af397b74a879245d9616bacf029" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d" is_pre_election: true
I20260709 12:59:00.252276  1062 raft_consensus.cc:2468] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 0.
W20260709 12:59:00.252724  1101 leader_election.cc:343] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:00.253393  1100 leader_election.cc:304] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 629e1bd745fb4bce886bc01856587fa3
I20260709 12:59:00.247586  1283 raft_consensus.cc:2804] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:00.253558  1263 leader_election.cc:290] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.254043  1283 raft_consensus.cc:493] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:00.254281  1263 raft_consensus.cc:2804] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:00.254859  1263 raft_consensus.cc:493] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:00.245348  1251 raft_consensus.cc:359] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.254426  1283 raft_consensus.cc:3060] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.255676  1251 raft_consensus.cc:385] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.256124  1251 raft_consensus.cc:740] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.257040  1251 consensus_queue.cc:260] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.255354  1263 raft_consensus.cc:3060] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.258093  1252 tablet_bootstrap.cc:492] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3: Bootstrap starting.
I20260709 12:59:00.259759  1251 ts_tablet_manager.cc:1434] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:59:00.261077  1251 tablet_bootstrap.cc:492] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3: Bootstrap starting.
I20260709 12:59:00.265650  1252 tablet_bootstrap.cc:654] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.269239  1254 tablet_bootstrap.cc:654] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.269285  1251 tablet_bootstrap.cc:654] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.278414  1283 raft_consensus.cc:515] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.278112   958 catalog_manager.cc:5697] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "9afc3caa620240e7a5a061700c100ea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.283432  1251 tablet_bootstrap.cc:492] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3: No bootstrap required, opened a new log
I20260709 12:59:00.284184  1251 ts_tablet_manager.cc:1403] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 12:59:00.285141  1252 tablet_bootstrap.cc:492] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3: No bootstrap required, opened a new log
I20260709 12:59:00.285742  1252 ts_tablet_manager.cc:1403] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.000s
I20260709 12:59:00.286373  1254 tablet_bootstrap.cc:492] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d: No bootstrap required, opened a new log
I20260709 12:59:00.286837  1254 ts_tablet_manager.cc:1403] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.005s
I20260709 12:59:00.287592  1251 raft_consensus.cc:359] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.288477  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daa76bc7ba174327b42ce50718d0123d" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:00.289552  1251 raft_consensus.cc:385] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.290201  1251 raft_consensus.cc:740] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.289938  1254 raft_consensus.cc:359] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.290832  1254 raft_consensus.cc:385] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.289273  1062 raft_consensus.cc:3060] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.291270  1254 raft_consensus.cc:740] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.291364  1251 consensus_queue.cc:260] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.292683  1254 consensus_queue.cc:260] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.295620  1254 ts_tablet_manager.cc:1434] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:59:00.291008  1252 raft_consensus.cc:359] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.298058  1252 raft_consensus.cc:385] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.298280  1251 ts_tablet_manager.cc:1434] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:59:00.299172  1255 raft_consensus.cc:493] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.299917  1261 raft_consensus.cc:493] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.299721  1255 raft_consensus.cc:515] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.300418  1261 raft_consensus.cc:515] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } }
I20260709 12:59:00.303630  1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.304360  1211 raft_consensus.cc:2468] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 0.
I20260709 12:59:00.305316  1255 leader_election.cc:290] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.306416  1254 tablet_bootstrap.cc:492] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d: Bootstrap starting.
I20260709 12:59:00.306485  1101 leader_election.cc:304] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: 
I20260709 12:59:00.308753  1061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d" is_pre_election: true
W20260709 12:59:00.310609  1100 leader_election.cc:343] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:00.311594  1255 raft_consensus.cc:2804] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:00.311990  1255 raft_consensus.cc:493] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:00.312321  1255 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.314217  1254 tablet_bootstrap.cc:654] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.316170  1261 leader_election.cc:290] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047), 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791)
I20260709 12:59:00.303880  1251 tablet_bootstrap.cc:492] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3: Bootstrap starting.
I20260709 12:59:00.282657  1263 raft_consensus.cc:515] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.331060  1061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362f1af397b74a879245d9616bacf029" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:00.331880  1061 raft_consensus.cc:3060] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.298615  1252 raft_consensus.cc:740] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.335875  1252 consensus_queue.cc:260] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.339177  1263 leader_election.cc:290] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.341209  1283 leader_election.cc:290] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.347039  1252 ts_tablet_manager.cc:1434] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 12:59:00.348182  1252 tablet_bootstrap.cc:492] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3: Bootstrap starting.
I20260709 12:59:00.349699  1254 tablet_bootstrap.cc:492] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d: No bootstrap required, opened a new log
I20260709 12:59:00.350380  1254 ts_tablet_manager.cc:1403] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d: Time spent bootstrapping tablet: real 0.044s	user 0.031s	sys 0.012s
I20260709 12:59:00.351548  1061 raft_consensus.cc:2468] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 1.
I20260709 12:59:00.353034  1100 leader_election.cc:304] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.354007  1254 raft_consensus.cc:359] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.354827  1254 raft_consensus.cc:385] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.355178  1254 raft_consensus.cc:740] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.356335  1258 raft_consensus.cc:493] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.356058  1254 consensus_queue.cc:260] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.356833  1258 raft_consensus.cc:515] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.357833  1283 raft_consensus.cc:2804] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:00.358349  1283 raft_consensus.cc:697] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Leader. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:00.359404  1254 ts_tablet_manager.cc:1434] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 12:59:00.359277  1283 consensus_queue.cc:237] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.360570  1254 tablet_bootstrap.cc:492] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d: Bootstrap starting.
I20260709 12:59:00.359735  1258 leader_election.cc:290] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791)
I20260709 12:59:00.345729  1062 raft_consensus.cc:2468] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 1.
I20260709 12:59:00.369033  1254 tablet_bootstrap.cc:654] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.380787  1254 tablet_bootstrap.cc:492] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d: No bootstrap required, opened a new log
I20260709 12:59:00.381459  1254 ts_tablet_manager.cc:1403] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.002s
I20260709 12:59:00.383105  1255 raft_consensus.cc:515] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.384794  1254 raft_consensus.cc:359] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.385569  1254 raft_consensus.cc:385] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.385905  1254 raft_consensus.cc:740] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.385990  1100 leader_election.cc:304] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.386881  1254 consensus_queue.cc:260] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.389219  1251 tablet_bootstrap.cc:654] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.389333  1254 ts_tablet_manager.cc:1434] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:00.390651  1263 raft_consensus.cc:493] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.391341  1263 raft_consensus.cc:515] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.391731  1254 tablet_bootstrap.cc:492] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d: Bootstrap starting.
I20260709 12:59:00.394017  1255 leader_election.cc:290] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.394680  1255 raft_consensus.cc:2804] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:00.395180  1255 raft_consensus.cc:697] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Leader. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:00.393417  1263 leader_election.cc:290] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.398921  1252 tablet_bootstrap.cc:654] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.400357  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:00.401031  1062 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.401216  1061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "262d0445f78c4bc5a1da87074cfd1101" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d" is_pre_election: true
I20260709 12:59:00.401871  1061 raft_consensus.cc:2468] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 0.
I20260709 12:59:00.403162  1100 leader_election.cc:304] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.409780  1263 raft_consensus.cc:2804] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:00.412657  1263 raft_consensus.cc:493] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:00.413084  1263 raft_consensus.cc:3060] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.418885  1254 tablet_bootstrap.cc:654] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.396070  1255 consensus_queue.cc:237] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.389022   960 catalog_manager.cc:5697] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "9afc3caa620240e7a5a061700c100ea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.446442  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed84eb1a8eb4ab08fb0353f201560c3" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3" is_pre_election: true
I20260709 12:59:00.447260  1137 raft_consensus.cc:2468] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 0.
I20260709 12:59:00.448747  1025 leader_election.cc:304] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.449800  1261 raft_consensus.cc:2804] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:00.435137  1062 raft_consensus.cc:2468] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 1.
I20260709 12:59:00.450188  1261 raft_consensus.cc:493] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:00.450547  1261 raft_consensus.cc:3060] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.450996  1263 raft_consensus.cc:515] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.453295  1263 leader_election.cc:290] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.461733  1100 leader_election.cc:304] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.462445  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "262d0445f78c4bc5a1da87074cfd1101" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:00.462781  1263 raft_consensus.cc:2804] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:00.463791  1263 raft_consensus.cc:697] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Leader. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:00.463373  1062 raft_consensus.cc:3060] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.465068  1263 consensus_queue.cc:237] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.478055   960 catalog_manager.cc:5697] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "9afc3caa620240e7a5a061700c100ea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.486680  1303 raft_consensus.cc:493] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.487396  1303 raft_consensus.cc:515] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.490433  1255 raft_consensus.cc:493] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.491411  1251 tablet_bootstrap.cc:492] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3: No bootstrap required, opened a new log
I20260709 12:59:00.491882  1251 ts_tablet_manager.cc:1403] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3: Time spent bootstrapping tablet: real 0.188s	user 0.020s	sys 0.007s
I20260709 12:59:00.491192  1255 raft_consensus.cc:515] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.491174  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3" is_pre_election: true
I20260709 12:59:00.493173  1303 leader_election.cc:290] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.497016  1210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daa76bc7ba174327b42ce50718d0123d" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.497785  1210 raft_consensus.cc:3060] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.494582  1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362f1af397b74a879245d9616bacf029" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.499068  1255 leader_election.cc:290] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.499383  1211 raft_consensus.cc:3060] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.495072  1137 raft_consensus.cc:2468] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 0.
I20260709 12:59:00.503500  1061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d" is_pre_election: true
I20260709 12:59:00.503351  1261 raft_consensus.cc:515] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } }
I20260709 12:59:00.504158  1061 raft_consensus.cc:2468] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 0.
I20260709 12:59:00.504565  1251 raft_consensus.cc:359] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.505324  1251 raft_consensus.cc:385] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.505623  1251 raft_consensus.cc:740] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.507125  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.508641  1251 consensus_queue.cc:260] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.511812  1025 leader_election.cc:304] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.512213  1208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "262d0445f78c4bc5a1da87074cfd1101" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.512955  1208 raft_consensus.cc:2468] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 0.
I20260709 12:59:00.511952  1100 leader_election.cc:304] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.507810  1209 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.514266  1255 raft_consensus.cc:2804] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:00.515789  1254 tablet_bootstrap.cc:492] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d: No bootstrap required, opened a new log
I20260709 12:59:00.516215  1254 ts_tablet_manager.cc:1403] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d: Time spent bootstrapping tablet: real 0.125s	user 0.015s	sys 0.016s
I20260709 12:59:00.514628  1255 raft_consensus.cc:493] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:00.520357  1251 ts_tablet_manager.cc:1434] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:59:00.520493  1255 raft_consensus.cc:3060] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.522557  1254 raft_consensus.cc:359] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.516855  1303 raft_consensus.cc:2804] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:00.523378  1254 raft_consensus.cc:385] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.523751  1254 raft_consensus.cc:740] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.523635  1303 raft_consensus.cc:493] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:00.516628  1261 leader_election.cc:290] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 election: Requested vote from peers 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047), 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791)
I20260709 12:59:00.518551  1061 raft_consensus.cc:3060] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.517020  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed84eb1a8eb4ab08fb0353f201560c3" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:00.528074  1254 consensus_queue.cc:260] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.525148  1303 raft_consensus.cc:3060] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.526418  1062 raft_consensus.cc:2468] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 1.
I20260709 12:59:00.526983  1208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "262d0445f78c4bc5a1da87074cfd1101" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.530550  1207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.531281  1137 raft_consensus.cc:3060] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.531512  1208 raft_consensus.cc:3060] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.532958  1206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed84eb1a8eb4ab08fb0353f201560c3" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.536363  1254 ts_tablet_manager.cc:1434] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.002s
I20260709 12:59:00.537305  1206 raft_consensus.cc:2468] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 0.
I20260709 12:59:00.537511  1254 tablet_bootstrap.cc:492] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d: Bootstrap starting.
I20260709 12:59:00.538626  1252 tablet_bootstrap.cc:492] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3: No bootstrap required, opened a new log
I20260709 12:59:00.539094  1252 ts_tablet_manager.cc:1403] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3: Time spent bootstrapping tablet: real 0.191s	user 0.015s	sys 0.011s
I20260709 12:59:00.540685  1100 leader_election.cc:304] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.541662  1263 raft_consensus.cc:2804] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:00.542155  1263 raft_consensus.cc:697] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Leader. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:00.543284  1263 consensus_queue.cc:237] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.549394   958 catalog_manager.cc:5697] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "9afc3caa620240e7a5a061700c100ea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.554003  1206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.554903  1210 raft_consensus.cc:2468] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 1.
I20260709 12:59:00.555130  1208 raft_consensus.cc:2468] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 1.
I20260709 12:59:00.554656  1252 raft_consensus.cc:359] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.556344  1252 raft_consensus.cc:385] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.556695  1252 raft_consensus.cc:740] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.559849  1207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed84eb1a8eb4ab08fb0353f201560c3" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.560732  1207 raft_consensus.cc:3060] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.558766  1252 consensus_queue.cc:260] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.566782  1206 raft_consensus.cc:2468] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 0.
W20260709 12:59:00.569562  1101 leader_election.cc:343] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:00.559345  1211 raft_consensus.cc:2468] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 1.
I20260709 12:59:00.578675  1137 raft_consensus.cc:2468] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 1.
I20260709 12:59:00.588353  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362f1af397b74a879245d9616bacf029" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d" is_pre_election: true
I20260709 12:59:00.584095  1304 raft_consensus.cc:493] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.589253  1062 raft_consensus.cc:2393] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 629e1bd745fb4bce886bc01856587fa3 in current term 1: Already voted for candidate 9afc3caa620240e7a5a061700c100ea3 in this term.
I20260709 12:59:00.589110  1304 raft_consensus.cc:515] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.591954  1304 leader_election.cc:290] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.589098  1025 leader_election.cc:304] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.593632  1304 raft_consensus.cc:2804] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:00.595351  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3" is_pre_election: true
I20260709 12:59:00.596330  1137 raft_consensus.cc:2468] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 0.
I20260709 12:59:00.596587  1207 raft_consensus.cc:2468] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 1.
I20260709 12:59:00.599567  1254 tablet_bootstrap.cc:654] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.602300  1304 raft_consensus.cc:697] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Becoming Leader. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:00.603458  1304 consensus_queue.cc:237] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } }
I20260709 12:59:00.598680  1025 leader_election.cc:304] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.604478  1207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.605149  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362f1af397b74a879245d9616bacf029" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3" is_pre_election: true
W20260709 12:59:00.613047  1026 leader_election.cc:343] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:00.614626  1209 raft_consensus.cc:2468] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 1.
I20260709 12:59:00.616497  1307 raft_consensus.cc:2804] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:00.619493  1307 raft_consensus.cc:493] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:00.619803  1307 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.614355  1174 leader_election.cc:304] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3; no voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:00.624239  1258 raft_consensus.cc:2749] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:00.624825  1261 raft_consensus.cc:493] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.625327  1261 raft_consensus.cc:515] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.628192  1263 consensus_queue.cc:1048] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:00.632401  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3" is_pre_election: true
I20260709 12:59:00.636785  1255 raft_consensus.cc:515] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.641650  1252 ts_tablet_manager.cc:1434] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.004s
I20260709 12:59:00.642535  1252 tablet_bootstrap.cc:492] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3: Bootstrap starting.
I20260709 12:59:00.644868  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:00.647234  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.648188  1209 raft_consensus.cc:2468] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 1.
I20260709 12:59:00.647118  1261 leader_election.cc:290] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.648705  1254 tablet_bootstrap.cc:492] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d: No bootstrap required, opened a new log
I20260709 12:59:00.650173  1254 ts_tablet_manager.cc:1403] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d: Time spent bootstrapping tablet: real 0.113s	user 0.020s	sys 0.006s
I20260709 12:59:00.649300  1026 leader_election.cc:304] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, bc53c40ea20b40dda6d9516fa180772d; no voters: 9afc3caa620240e7a5a061700c100ea3
I20260709 12:59:00.649964  1303 raft_consensus.cc:515] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.650422  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.651006  1209 raft_consensus.cc:3060] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.651487  1062 raft_consensus.cc:2393] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9afc3caa620240e7a5a061700c100ea3 in current term 1: Already voted for candidate bc53c40ea20b40dda6d9516fa180772d in this term.
I20260709 12:59:00.639730   958 catalog_manager.cc:5697] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "9afc3caa620240e7a5a061700c100ea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.649080  1255 leader_election.cc:290] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.653863  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:00.654625  1137 raft_consensus.cc:2393] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bc53c40ea20b40dda6d9516fa180772d in current term 1: Already voted for candidate 9afc3caa620240e7a5a061700c100ea3 in this term.
I20260709 12:59:00.655700  1254 raft_consensus.cc:359] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.656366  1254 raft_consensus.cc:385] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.656639  1254 raft_consensus.cc:740] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.657509  1254 consensus_queue.cc:260] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.659652  1254 ts_tablet_manager.cc:1434] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:00.661521  1261 raft_consensus.cc:2804] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:00.662449  1306 raft_consensus.cc:493] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.661602  1307 raft_consensus.cc:515] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.663021  1306 raft_consensus.cc:515] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.667132  1261 raft_consensus.cc:493] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9afc3caa620240e7a5a061700c100ea3)
I20260709 12:59:00.667794  1303 leader_election.cc:290] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 election: Requested vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.667901  1261 raft_consensus.cc:3060] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:00.670001  1207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.671393  1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.671681  1209 raft_consensus.cc:2468] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 1.
I20260709 12:59:00.674001  1101 leader_election.cc:304] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:00.675989  1026 leader_election.cc:304] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc53c40ea20b40dda6d9516fa180772d; no voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3
I20260709 12:59:00.675755  1261 raft_consensus.cc:515] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.676121  1283 raft_consensus.cc:2804] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:00.676317  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3" is_pre_election: true
I20260709 12:59:00.676811  1283 raft_consensus.cc:697] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Leader. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:00.676980  1137 raft_consensus.cc:2468] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 0.
W20260709 12:59:00.677237  1026 leader_election.cc:343] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:00.677910  1283 consensus_queue.cc:237] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.680101  1025 leader_election.cc:304] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 629e1bd745fb4bce886bc01856587fa3
I20260709 12:59:00.681022  1304 raft_consensus.cc:2749] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:00.681742  1304 raft_consensus.cc:2804] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:00.682070  1304 raft_consensus.cc:493] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:00.682408  1304 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.683459  1306 leader_election.cc:290] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.685109   960 catalog_manager.cc:5697] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d reported cstate change: term changed from 0 to 1, leader changed from <none> to bc53c40ea20b40dda6d9516fa180772d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bc53c40ea20b40dda6d9516fa180772d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:00.690303  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:00.690728  1061 raft_consensus.cc:1240] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Rejecting Update request from peer 9afc3caa620240e7a5a061700c100ea3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:00.691033  1137 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.692327  1304 raft_consensus.cc:515] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.693066  1307 leader_election.cc:290] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 election: Requested vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.695070  1282 consensus_queue.cc:1059] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:00.695860  1136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:00.696543  1136 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.697364  1252 tablet_bootstrap.cc:654] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.697330  1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.698091  1211 raft_consensus.cc:3060] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:00.701998  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.703251  1135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
W20260709 12:59:00.703985  1026 leader_election.cc:343] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:00.701428  1137 raft_consensus.cc:2468] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 1.
I20260709 12:59:00.705660  1307 raft_consensus.cc:493] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.706238  1307 raft_consensus.cc:515] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.706732  1025 leader_election.cc:304] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 629e1bd745fb4bce886bc01856587fa3
I20260709 12:59:00.707626  1303 raft_consensus.cc:2804] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:00.708127  1303 raft_consensus.cc:697] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Becoming Leader. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:00.708505  1307 leader_election.cc:290] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.709012  1303 consensus_queue.cc:237] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.709726  1211 raft_consensus.cc:2468] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 2.
I20260709 12:59:00.711181  1026 leader_election.cc:304] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, bc53c40ea20b40dda6d9516fa180772d; no voters: 9afc3caa620240e7a5a061700c100ea3
I20260709 12:59:00.710156  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93594eee86c84ebd8bba2c1d2a934ed6" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3" is_pre_election: true
I20260709 12:59:00.712646  1306 raft_consensus.cc:2804] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:00.713181  1306 raft_consensus.cc:697] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Becoming Leader. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:00.714246  1306 consensus_queue.cc:237] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.717756  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93594eee86c84ebd8bba2c1d2a934ed6" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3" is_pre_election: true
I20260709 12:59:00.717209  1261 leader_election.cc:290] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 election: Requested vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.722182  1209 raft_consensus.cc:2393] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc53c40ea20b40dda6d9516fa180772d in current term 1: Already voted for candidate 9afc3caa620240e7a5a061700c100ea3 in this term.
I20260709 12:59:00.723925  1026 leader_election.cc:304] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc53c40ea20b40dda6d9516fa180772d; no voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3
I20260709 12:59:00.725620  1306 raft_consensus.cc:3060] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.728191  1136 raft_consensus.cc:2468] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 1.
I20260709 12:59:00.729966  1025 leader_election.cc:304] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:00.732486  1261 raft_consensus.cc:2804] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:00.733428  1261 raft_consensus.cc:697] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Becoming Leader. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:00.736188  1255 raft_consensus.cc:3055] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:00.736415  1209 raft_consensus.cc:1240] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Rejecting Update request from peer 9afc3caa620240e7a5a061700c100ea3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:00.736616  1255 raft_consensus.cc:740] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:00.737548  1255 consensus_queue.cc:260] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.751169  1304 leader_election.cc:290] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 election: Requested vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:00.751992  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:00.752303   960 catalog_manager.cc:5697] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "9afc3caa620240e7a5a061700c100ea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.753705  1306 raft_consensus.cc:2749] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:00.753896  1026 leader_election.cc:343] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:00.756490  1252 tablet_bootstrap.cc:492] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3: No bootstrap required, opened a new log
I20260709 12:59:00.756299  1261 consensus_queue.cc:237] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.757035  1252 ts_tablet_manager.cc:1403] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3: Time spent bootstrapping tablet: real 0.115s	user 0.021s	sys 0.019s
I20260709 12:59:00.760424  1283 consensus_queue.cc:1048] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:00.764828  1255 raft_consensus.cc:3060] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:00.765904  1252 raft_consensus.cc:359] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.766629  1252 raft_consensus.cc:385] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.766965  1252 raft_consensus.cc:740] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.767802  1252 consensus_queue.cc:260] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.770254  1252 ts_tablet_manager.cc:1434] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:59:00.771252  1252 tablet_bootstrap.cc:492] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3: Bootstrap starting.
I20260709 12:59:00.786013  1252 tablet_bootstrap.cc:654] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:00.791884  1283 consensus_queue.cc:1048] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:00.793864   958 catalog_manager.cc:5697] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d reported cstate change: term changed from 0 to 1, leader changed from <none> to bc53c40ea20b40dda6d9516fa180772d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bc53c40ea20b40dda6d9516fa180772d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.795938   958 catalog_manager.cc:5697] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d reported cstate change: term changed from 0 to 1, leader changed from <none> to bc53c40ea20b40dda6d9516fa180772d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bc53c40ea20b40dda6d9516fa180772d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.797708   958 catalog_manager.cc:5697] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d reported cstate change: term changed from 1 to 2, leader changed from 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194) to bc53c40ea20b40dda6d9516fa180772d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bc53c40ea20b40dda6d9516fa180772d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.803225  1282 consensus_queue.cc:1048] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:00.807217  1252 tablet_bootstrap.cc:492] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3: No bootstrap required, opened a new log
I20260709 12:59:00.808077  1252 ts_tablet_manager.cc:1403] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.014s
I20260709 12:59:00.816906  1252 raft_consensus.cc:359] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.826148  1252 raft_consensus.cc:385] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:00.826777  1252 raft_consensus.cc:740] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Initialized, Role: FOLLOWER
I20260709 12:59:00.827960  1252 consensus_queue.cc:260] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.831435  1252 ts_tablet_manager.cc:1434] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:59:00.836275  1255 consensus_queue.cc:1048] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:00.854039  1283 consensus_queue.cc:1048] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:00.871694  1255 consensus_queue.cc:1048] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:00.924502  1324 raft_consensus.cc:493] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:00.925117  1324 raft_consensus.cc:515] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:00.927382  1324 leader_election.cc:290] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791)
I20260709 12:59:00.928388  1061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d" is_pre_election: true
I20260709 12:59:00.928581  1136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3" is_pre_election: true
I20260709 12:59:00.929593  1136 raft_consensus.cc:2393] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 629e1bd745fb4bce886bc01856587fa3 in current term 1: Already voted for candidate bc53c40ea20b40dda6d9516fa180772d in this term.
I20260709 12:59:00.930656  1174 leader_election.cc:304] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3; no voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:00.931342  1324 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:00.937541  1324 raft_consensus.cc:2749] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:00.962604  1255 consensus_queue.cc:1048] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:00.974471  1283 consensus_queue.cc:1048] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:00.999498  1324 raft_consensus.cc:493] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:01.000054  1324 raft_consensus.cc:515] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.002146  1324 leader_election.cc:290] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791)
I20260709 12:59:01.003078  1061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc53c40ea20b40dda6d9516fa180772d" is_pre_election: true
I20260709 12:59:01.003429  1136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9afc3caa620240e7a5a061700c100ea3" is_pre_election: true
I20260709 12:59:01.004300  1136 raft_consensus.cc:2393] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 629e1bd745fb4bce886bc01856587fa3 in current term 1: Already voted for candidate bc53c40ea20b40dda6d9516fa180772d in this term.
I20260709 12:59:01.005425  1174 leader_election.cc:304] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3; no voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:01.006325  1324 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:01.011564  1324 raft_consensus.cc:2749] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:01.022168  1136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "daa76bc7ba174327b42ce50718d0123d"
dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
mode: GRACEFUL
new_leader_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:47738
I20260709 12:59:01.022876  1136 raft_consensus.cc:606] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Received request to transfer leadership to TS bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:01.031231  1136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5058095638664e428f3cf97e53848db2"
dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
mode: GRACEFUL
new_leader_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:47738
I20260709 12:59:01.031739  1136 raft_consensus.cc:606] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Received request to transfer leadership to TS bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:01.056084  1061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
mode: GRACEFUL
new_leader_uuid: "9afc3caa620240e7a5a061700c100ea3"
 from {username='slave'} at 127.0.0.1:53532
I20260709 12:59:01.056855  1061 raft_consensus.cc:606] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Received request to transfer leadership to TS 9afc3caa620240e7a5a061700c100ea3
I20260709 12:59:01.057144  1283 consensus_queue.cc:1048] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:01.067448  1136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9"
dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
mode: GRACEFUL
new_leader_uuid: "629e1bd745fb4bce886bc01856587fa3"
 from {username='slave'} at 127.0.0.1:47738
I20260709 12:59:01.068133  1136 raft_consensus.cc:606] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Received request to transfer leadership to TS 629e1bd745fb4bce886bc01856587fa3
I20260709 12:59:01.070055  1263 consensus_queue.cc:1048] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:01.073117  1261 consensus_queue.cc:1048] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:01.092042  1304 consensus_queue.cc:1048] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:01.114176  1263 consensus_queue.cc:1048] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:01.120349  1263 consensus_queue.cc:1048] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:01.121987  1282 raft_consensus.cc:993] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3: : Instructing follower 629e1bd745fb4bce886bc01856587fa3 to start an election
I20260709 12:59:01.122524  1282 raft_consensus.cc:1081] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Signalling peer 629e1bd745fb4bce886bc01856587fa3 to start an election
I20260709 12:59:01.125903  1209 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9"
dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
 from {username='slave'} at 127.0.0.1:44224
I20260709 12:59:01.126629  1209 raft_consensus.cc:493] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:01.127049  1209 raft_consensus.cc:3060] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.139496  1209 raft_consensus.cc:515] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.144690  1061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:01.145458  1061 raft_consensus.cc:3060] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.150099  1209 leader_election.cc:290] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 2 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791)
I20260709 12:59:01.151387  1136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3d601732374c73ad6f1b5a15956de9" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:01.152105  1136 raft_consensus.cc:3055] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:01.152421  1136 raft_consensus.cc:740] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:01.153141  1136 consensus_queue.cc:260] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.154240  1136 raft_consensus.cc:3060] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.154309  1061 raft_consensus.cc:2468] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e1bd745fb4bce886bc01856587fa3 in term 2.
I20260709 12:59:01.155553  1174 leader_election.cc:304] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:01.156375  1324 raft_consensus.cc:2804] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:01.156848  1324 raft_consensus.cc:697] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 2 LEADER]: Becoming Leader. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Running, Role: LEADER
I20260709 12:59:01.157717  1324 consensus_queue.cc:237] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.160682  1136 raft_consensus.cc:2468] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e1bd745fb4bce886bc01856587fa3 in term 2.
I20260709 12:59:01.165283   960 catalog_manager.cc:5697] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 reported cstate change: term changed from 1 to 2, leader changed from 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194) to 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "629e1bd745fb4bce886bc01856587fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:01.226243  1136 raft_consensus.cc:1275] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:01.227730  1304 consensus_queue.cc:1048] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:01.232287  1304 consensus_queue.cc:1048] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:01.233106  1306 consensus_queue.cc:1048] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:01.240499  1211 raft_consensus.cc:1275] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:01.242941  1304 consensus_queue.cc:1048] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:01.251021  1306 consensus_queue.cc:1048] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:01.252772  1306 raft_consensus.cc:993] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d: : Instructing follower 9afc3caa620240e7a5a061700c100ea3 to start an election
I20260709 12:59:01.253268  1306 raft_consensus.cc:1081] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Signalling peer 9afc3caa620240e7a5a061700c100ea3 to start an election
I20260709 12:59:01.256750  1137 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b"
dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
 from {username='slave'} at 127.0.0.1:47730
I20260709 12:59:01.257370  1137 raft_consensus.cc:493] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:01.257833  1137 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.269130  1137 raft_consensus.cc:515] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.283154  1061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:01.284219  1061 raft_consensus.cc:3055] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:01.284720  1061 raft_consensus.cc:740] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:01.288071  1061 consensus_queue.cc:260] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.290577  1136 raft_consensus.cc:1240] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Rejecting Update request from peer bc53c40ea20b40dda6d9516fa180772d for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:01.291093  1061 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.293370  1304 consensus_queue.cc:1048] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:01.296541  1137 leader_election.cc:290] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 2 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:01.298161  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:01.298882  1209 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.302374  1061 raft_consensus.cc:2468] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 2.
I20260709 12:59:01.310562  1100 leader_election.cc:304] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:01.310567  1209 raft_consensus.cc:2468] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 2.
I20260709 12:59:01.311903  1283 raft_consensus.cc:2804] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:01.312472  1283 raft_consensus.cc:697] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 2 LEADER]: Becoming Leader. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:01.313349  1283 consensus_queue.cc:237] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.325623   957 catalog_manager.cc:5697] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 reported cstate change: term changed from 1 to 2, leader changed from bc53c40ea20b40dda6d9516fa180772d (127.0.107.193) to 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "9afc3caa620240e7a5a061700c100ea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:01.330435  1263 raft_consensus.cc:993] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3: : Instructing follower bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:01.330950  1283 raft_consensus.cc:1081] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Signalling peer bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:01.332715  1061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "daa76bc7ba174327b42ce50718d0123d"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:53518
I20260709 12:59:01.333338  1061 raft_consensus.cc:493] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:01.333856  1061 raft_consensus.cc:3060] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.342429  1061 raft_consensus.cc:515] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.345660  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daa76bc7ba174327b42ce50718d0123d" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:01.345963  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daa76bc7ba174327b42ce50718d0123d" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:01.346530  1209 raft_consensus.cc:3060] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.346292  1137 raft_consensus.cc:3055] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:01.347046  1137 raft_consensus.cc:740] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:01.347893  1137 consensus_queue.cc:260] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.348707  1137 raft_consensus.cc:3060] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.349640  1061 leader_election.cc:290] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 election: Requested vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:01.353075  1209 raft_consensus.cc:2468] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 2.
I20260709 12:59:01.353606  1137 raft_consensus.cc:2468] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 2.
I20260709 12:59:01.354475  1026 leader_election.cc:304] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:01.355312  1304 raft_consensus.cc:2804] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:01.355796  1304 raft_consensus.cc:697] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Becoming Leader. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:01.356495  1304 consensus_queue.cc:237] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.363606   957 catalog_manager.cc:5697] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d reported cstate change: term changed from 1 to 2, leader changed from 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194) to bc53c40ea20b40dda6d9516fa180772d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bc53c40ea20b40dda6d9516fa180772d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:01.557067  1137 raft_consensus.cc:1275] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Refusing update from remote peer 629e1bd745fb4bce886bc01856587fa3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:01.558578  1258 consensus_queue.cc:1048] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:01.568413  1061 raft_consensus.cc:1275] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Refusing update from remote peer 629e1bd745fb4bce886bc01856587fa3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:01.570449  1258 consensus_queue.cc:1048] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:01.581346  1263 raft_consensus.cc:993] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3: : Instructing follower bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:01.581882  1283 raft_consensus.cc:1081] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Signalling peer bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:01.583627  1061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5058095638664e428f3cf97e53848db2"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:53518
I20260709 12:59:01.584187  1061 raft_consensus.cc:493] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:01.584501  1061 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.590512  1061 raft_consensus.cc:515] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.592288  1061 leader_election.cc:290] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 election: Requested vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:01.593254  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:01.593461  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:01.593995  1137 raft_consensus.cc:3055] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:01.594003  1209 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.594519  1137 raft_consensus.cc:740] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:01.595402  1137 consensus_queue.cc:260] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.596684  1137 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:01.602214  1209 raft_consensus.cc:2468] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 2.
I20260709 12:59:01.603467  1026 leader_election.cc:304] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:01.604209  1304 raft_consensus.cc:2804] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:01.604251  1137 raft_consensus.cc:2468] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 2.
I20260709 12:59:01.604862  1304 raft_consensus.cc:697] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Becoming Leader. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:01.605808  1304 consensus_queue.cc:237] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:01.612531   960 catalog_manager.cc:5697] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d reported cstate change: term changed from 1 to 2, leader changed from 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194) to bc53c40ea20b40dda6d9516fa180772d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bc53c40ea20b40dda6d9516fa180772d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:01.751210  1209 raft_consensus.cc:1275] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:01.753679  1304 consensus_queue.cc:1048] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:01.765043  1137 raft_consensus.cc:1275] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:01.766678  1261 consensus_queue.cc:1048] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:01.846238  1061 raft_consensus.cc:1275] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Refusing update from remote peer 9afc3caa620240e7a5a061700c100ea3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:01.847904  1283 consensus_queue.cc:1048] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:01.856997  1207 raft_consensus.cc:1275] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Refusing update from remote peer 9afc3caa620240e7a5a061700c100ea3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:01.858809  1344 consensus_queue.cc:1048] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:02.107857  1207 raft_consensus.cc:1275] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:02.109282  1304 consensus_queue.cc:1048] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:02.118925  1137 raft_consensus.cc:1275] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:02.121346  1304 consensus_queue.cc:1048] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:04.110725   431 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 12:59:04.111552   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:04.129909  1062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5058095638664e428f3cf97e53848db2"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
mode: GRACEFUL
new_leader_uuid: "629e1bd745fb4bce886bc01856587fa3"
 from {username='slave'} at 127.0.0.1:56322
I20260709 12:59:04.130734  1062 raft_consensus.cc:606] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Received request to transfer leadership to TS 629e1bd745fb4bce886bc01856587fa3
I20260709 12:59:04.133455  1062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
mode: GRACEFUL
new_leader_uuid: "629e1bd745fb4bce886bc01856587fa3"
 from {username='slave'} at 127.0.0.1:56322
I20260709 12:59:04.134063  1062 raft_consensus.cc:606] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Received request to transfer leadership to TS 629e1bd745fb4bce886bc01856587fa3
I20260709 12:59:04.135350   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:59:04.135918   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:04.179770  1352 raft_consensus.cc:993] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d: : Instructing follower 629e1bd745fb4bce886bc01856587fa3 to start an election
I20260709 12:59:04.180361  1304 raft_consensus.cc:1081] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Signalling peer 629e1bd745fb4bce886bc01856587fa3 to start an election
I20260709 12:59:04.182257  1209 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab"
dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
 from {username='slave'} at 127.0.0.1:44236
I20260709 12:59:04.182864  1209 raft_consensus.cc:493] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:04.183226  1209 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:04.189672  1209 raft_consensus.cc:515] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:04.191459  1209 leader_election.cc:290] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 2 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791)
I20260709 12:59:04.192696  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:04.192937  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:04.193406  1062 raft_consensus.cc:3055] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:04.193531  1137 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:04.193823  1062 raft_consensus.cc:740] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:04.194677  1062 consensus_queue.cc:260] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:04.195771  1062 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:04.199245  1137 raft_consensus.cc:2468] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e1bd745fb4bce886bc01856587fa3 in term 2.
I20260709 12:59:04.200675  1174 leader_election.cc:304] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: 
I20260709 12:59:04.201449  1348 raft_consensus.cc:2804] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:04.202050  1062 raft_consensus.cc:2468] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e1bd745fb4bce886bc01856587fa3 in term 2.
I20260709 12:59:04.202699  1348 raft_consensus.cc:697] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 2 LEADER]: Becoming Leader. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Running, Role: LEADER
I20260709 12:59:04.203639  1348 consensus_queue.cc:237] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:04.210879   960 catalog_manager.cc:5697] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 reported cstate change: term changed from 1 to 2, leader changed from bc53c40ea20b40dda6d9516fa180772d (127.0.107.193) to 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "629e1bd745fb4bce886bc01856587fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:04.292083  1352 raft_consensus.cc:993] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d: : Instructing follower 629e1bd745fb4bce886bc01856587fa3 to start an election
I20260709 12:59:04.292685  1352 raft_consensus.cc:1081] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Signalling peer 629e1bd745fb4bce886bc01856587fa3 to start an election
I20260709 12:59:04.294462  1209 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5058095638664e428f3cf97e53848db2"
dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
 from {username='slave'} at 127.0.0.1:44236
I20260709 12:59:04.295156  1209 raft_consensus.cc:493] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:04.295629  1209 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:04.301935  1209 raft_consensus.cc:515] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:04.303886  1209 leader_election.cc:290] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 3 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791)
I20260709 12:59:04.305037  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:04.305251  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:04.305733  1062 raft_consensus.cc:3055] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:04.305869  1137 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:04.306079  1062 raft_consensus.cc:740] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:04.306941  1062 consensus_queue.cc:260] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:04.307950  1062 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:04.311581  1137 raft_consensus.cc:2468] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e1bd745fb4bce886bc01856587fa3 in term 3.
I20260709 12:59:04.312803  1174 leader_election.cc:304] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: 
I20260709 12:59:04.313668  1062 raft_consensus.cc:2468] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e1bd745fb4bce886bc01856587fa3 in term 3.
I20260709 12:59:04.313724  1348 raft_consensus.cc:2804] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:04.314309  1348 raft_consensus.cc:697] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 3 LEADER]: Becoming Leader. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Running, Role: LEADER
I20260709 12:59:04.315212  1348 consensus_queue.cc:237] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:04.322634   960 catalog_manager.cc:5697] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 reported cstate change: term changed from 2 to 3, leader changed from bc53c40ea20b40dda6d9516fa180772d (127.0.107.193) to 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "629e1bd745fb4bce886bc01856587fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:04.633736  1137 raft_consensus.cc:1275] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Refusing update from remote peer 629e1bd745fb4bce886bc01856587fa3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:04.635143  1348 consensus_queue.cc:1048] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:04.645373  1062 raft_consensus.cc:1275] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Refusing update from remote peer 629e1bd745fb4bce886bc01856587fa3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:04.647310  1348 consensus_queue.cc:1048] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:04.742286  1137 raft_consensus.cc:1275] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Refusing update from remote peer 629e1bd745fb4bce886bc01856587fa3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.743723  1348 consensus_queue.cc:1048] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.754868  1062 raft_consensus.cc:1275] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Refusing update from remote peer 629e1bd745fb4bce886bc01856587fa3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.756446  1367 consensus_queue.cc:1048] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:05.136955   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:05.179867  1209 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5058095638664e428f3cf97e53848db2"
dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
mode: GRACEFUL
new_leader_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:48248
I20260709 12:59:05.180570  1209 raft_consensus.cc:606] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 3 LEADER]: Received request to transfer leadership to TS bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:05.189060  1209 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab"
dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
mode: GRACEFUL
new_leader_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:48248
I20260709 12:59:05.189704  1209 raft_consensus.cc:606] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 2 LEADER]: Received request to transfer leadership to TS bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:05.195828  1136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "262d0445f78c4bc5a1da87074cfd1101"
dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
mode: GRACEFUL
new_leader_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:47738
I20260709 12:59:05.196501  1136 raft_consensus.cc:606] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Received request to transfer leadership to TS bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:05.202049  1136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "362f1af397b74a879245d9616bacf029"
dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
mode: GRACEFUL
new_leader_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:47738
I20260709 12:59:05.202595  1136 raft_consensus.cc:606] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Received request to transfer leadership to TS bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:05.208554  1136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b"
dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
mode: GRACEFUL
new_leader_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:47738
I20260709 12:59:05.209252  1136 raft_consensus.cc:606] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 2 LEADER]: Received request to transfer leadership to TS bc53c40ea20b40dda6d9516fa180772d
I20260709 12:59:05.363566  1371 raft_consensus.cc:993] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3: : Instructing follower bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:05.364389  1371 raft_consensus.cc:1081] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Signalling peer bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:05.366761  1061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "262d0445f78c4bc5a1da87074cfd1101"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:53518
I20260709 12:59:05.367537  1061 raft_consensus.cc:493] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:05.367925  1061 raft_consensus.cc:3060] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:05.413642  1061 raft_consensus.cc:515] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.415845  1061 leader_election.cc:290] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 election: Requested vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:05.417420  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "262d0445f78c4bc5a1da87074cfd1101" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:05.417441  1136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "262d0445f78c4bc5a1da87074cfd1101" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:05.418437  1136 raft_consensus.cc:3055] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:05.418438  1209 raft_consensus.cc:3060] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:05.418818  1136 raft_consensus.cc:740] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:05.419838  1136 consensus_queue.cc:260] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.421173  1136 raft_consensus.cc:3060] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:05.427271  1136 raft_consensus.cc:2468] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 2.
I20260709 12:59:05.427209  1209 raft_consensus.cc:2468] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 2.
I20260709 12:59:05.429023  1025 leader_election.cc:304] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:05.430145  1352 raft_consensus.cc:2804] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:05.431272  1352 raft_consensus.cc:697] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Becoming Leader. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:05.432319  1352 consensus_queue.cc:237] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.442371   960 catalog_manager.cc:5697] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d reported cstate change: term changed from 1 to 2, leader changed from 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194) to bc53c40ea20b40dda6d9516fa180772d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bc53c40ea20b40dda6d9516fa180772d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:05.650184  1344 raft_consensus.cc:993] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3: : Instructing follower bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:05.650769  1344 raft_consensus.cc:1081] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 2 LEADER]: Signalling peer bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:05.652559  1061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:53518
I20260709 12:59:05.653085  1061 raft_consensus.cc:493] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:05.653370  1061 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:05.660521  1061 raft_consensus.cc:515] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.662482  1061 leader_election.cc:290] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 3 election: Requested vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:05.663677  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:05.663677  1136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:05.664574  1209 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:05.664659  1136 raft_consensus.cc:3055] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:05.665273  1136 raft_consensus.cc:740] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:05.666481  1136 consensus_queue.cc:260] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.668181  1136 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:05.670094  1353 raft_consensus.cc:993] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3: : Instructing follower bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:05.670804  1353 raft_consensus.cc:1081] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 2 LEADER]: Signalling peer bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:05.672932  1061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:53520
I20260709 12:59:05.673431  1061 raft_consensus.cc:493] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:05.673815  1061 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:05.676462  1209 raft_consensus.cc:2468] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 3.
I20260709 12:59:05.677125  1136 raft_consensus.cc:2468] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 3.
I20260709 12:59:05.677956  1026 leader_election.cc:304] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:05.678916  1352 raft_consensus.cc:2804] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:05.678894  1061 raft_consensus.cc:515] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.679643  1352 raft_consensus.cc:697] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 3 LEADER]: Becoming Leader. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:05.680831  1352 consensus_queue.cc:237] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.681564  1061 leader_election.cc:290] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 3 election: Requested vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:05.682294  1136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:05.682924  1136 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:05.683130  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:05.683930  1209 raft_consensus.cc:3055] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:05.684314  1209 raft_consensus.cc:740] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Running, Role: LEADER
I20260709 12:59:05.685495  1209 consensus_queue.cc:260] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.686954  1209 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:05.691473   960 catalog_manager.cc:5697] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d reported cstate change: term changed from 2 to 3, leader changed from 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194) to bc53c40ea20b40dda6d9516fa180772d (127.0.107.193). New cstate: current_term: 3 leader_uuid: "bc53c40ea20b40dda6d9516fa180772d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:05.692965  1136 raft_consensus.cc:2468] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 3.
I20260709 12:59:05.694007  1209 raft_consensus.cc:2468] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 3.
I20260709 12:59:05.694689  1344 raft_consensus.cc:993] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3: : Instructing follower bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:05.694512  1025 leader_election.cc:304] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:05.695168  1344 raft_consensus.cc:1081] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Signalling peer bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:05.695835  1352 raft_consensus.cc:2804] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:05.696467  1352 raft_consensus.cc:697] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 3 LEADER]: Becoming Leader. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:05.696978  1061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "362f1af397b74a879245d9616bacf029"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:53518
I20260709 12:59:05.697645  1061 raft_consensus.cc:493] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:05.698086  1061 raft_consensus.cc:3060] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:05.697713  1352 consensus_queue.cc:237] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.705374  1061 raft_consensus.cc:515] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.707500  1061 leader_election.cc:290] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 election: Requested vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:05.708571  1136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362f1af397b74a879245d9616bacf029" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:05.708626  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "362f1af397b74a879245d9616bacf029" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:05.709345  1209 raft_consensus.cc:3060] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:05.709340  1136 raft_consensus.cc:3055] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:05.709890  1136 raft_consensus.cc:740] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:05.711014  1136 consensus_queue.cc:260] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.711313   960 catalog_manager.cc:5697] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d reported cstate change: term changed from 2 to 3, leader changed from 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195) to bc53c40ea20b40dda6d9516fa180772d (127.0.107.193). New cstate: current_term: 3 leader_uuid: "bc53c40ea20b40dda6d9516fa180772d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:05.712338  1136 raft_consensus.cc:3060] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:05.717587  1209 raft_consensus.cc:2468] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 2.
I20260709 12:59:05.718892  1026 leader_election.cc:304] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:05.719975  1136 raft_consensus.cc:2468] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 2.
I20260709 12:59:05.720460  1352 raft_consensus.cc:2804] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:05.721125  1352 raft_consensus.cc:697] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Becoming Leader. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:05.722798  1352 consensus_queue.cc:237] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.733337   960 catalog_manager.cc:5697] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d reported cstate change: term changed from 1 to 2, leader changed from 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194) to bc53c40ea20b40dda6d9516fa180772d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bc53c40ea20b40dda6d9516fa180772d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:05.746549  1353 raft_consensus.cc:993] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3: : Instructing follower bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:05.747287  1348 raft_consensus.cc:1081] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 3 LEADER]: Signalling peer bc53c40ea20b40dda6d9516fa180772d to start an election
I20260709 12:59:05.749097  1061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5058095638664e428f3cf97e53848db2"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
 from {username='slave'} at 127.0.0.1:53520
I20260709 12:59:05.749655  1061 raft_consensus.cc:493] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:05.749990  1061 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:05.755266  1061 raft_consensus.cc:515] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.757215  1061 leader_election.cc:290] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 4 election: Requested vote from peers 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:05.758380  1209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:05.758375  1136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "bc53c40ea20b40dda6d9516fa180772d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:05.759053  1136 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:05.759048  1209 raft_consensus.cc:3055] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:05.759564  1209 raft_consensus.cc:740] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Running, Role: LEADER
I20260709 12:59:05.760282  1209 consensus_queue.cc:260] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.761219  1209 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:05.765378  1136 raft_consensus.cc:2468] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 4.
I20260709 12:59:05.766824  1025 leader_election.cc:304] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9afc3caa620240e7a5a061700c100ea3, bc53c40ea20b40dda6d9516fa180772d; no voters: 
I20260709 12:59:05.767707  1352 raft_consensus.cc:2804] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:05.767652  1209 raft_consensus.cc:2468] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc53c40ea20b40dda6d9516fa180772d in term 4.
I20260709 12:59:05.768378  1352 raft_consensus.cc:697] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 4 LEADER]: Becoming Leader. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:05.769798  1352 consensus_queue.cc:237] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:05.777050   960 catalog_manager.cc:5697] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d reported cstate change: term changed from 3 to 4, leader changed from 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195) to bc53c40ea20b40dda6d9516fa180772d (127.0.107.193). New cstate: current_term: 4 leader_uuid: "bc53c40ea20b40dda6d9516fa180772d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:05.872138  1136 raft_consensus.cc:1275] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:05.873617  1352 consensus_queue.cc:1048] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:05.884845  1209 raft_consensus.cc:1275] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:05.886881  1373 consensus_queue.cc:1048] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:06.120716  1136 raft_consensus.cc:1275] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:06.122298  1373 consensus_queue.cc:1048] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:06.133997  1209 raft_consensus.cc:1275] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:06.136737  1381 consensus_queue.cc:1048] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:06.154953  1136 raft_consensus.cc:1275] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:06.156325  1381 consensus_queue.cc:1048] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:06.168844  1209 raft_consensus.cc:1275] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:06.171039  1352 consensus_queue.cc:1048] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:06.200372  1209 raft_consensus.cc:1275] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:06.202158  1352 consensus_queue.cc:1048] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:06.214334  1136 raft_consensus.cc:1275] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:06.216251  1381 consensus_queue.cc:1048] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:06.224087  1209 raft_consensus.cc:1275] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 4 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:06.226197  1381 consensus_queue.cc:1048] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:06.238682  1136 raft_consensus.cc:1275] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 4 FOLLOWER]: Refusing update from remote peer bc53c40ea20b40dda6d9516fa180772d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:06.240190  1352 consensus_queue.cc:1048] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:59:07.451732  1232 debug-util.cc:398] Leaking SignalData structure 0x7b08000b0700 after lost signal to thread 434
W20260709 12:59:07.452744  1232 debug-util.cc:398] Leaking SignalData structure 0x7b08000a6000 after lost signal to thread 994
W20260709 12:59:07.453773  1232 debug-util.cc:398] Leaking SignalData structure 0x7b08000e31c0 after lost signal to thread 1086
W20260709 12:59:07.454599  1232 debug-util.cc:398] Leaking SignalData structure 0x7b08001b7aa0 after lost signal to thread 1161
W20260709 12:59:07.455395  1232 debug-util.cc:398] Leaking SignalData structure 0x7b0800089f60 after lost signal to thread 1235
I20260709 12:59:08.374469   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:08.389117  1061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
mode: GRACEFUL
new_leader_uuid: "629e1bd745fb4bce886bc01856587fa3"
 from {username='slave'} at 127.0.0.1:56322
I20260709 12:59:08.389879  1061 raft_consensus.cc:606] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 3 LEADER]: Received request to transfer leadership to TS 629e1bd745fb4bce886bc01856587fa3
I20260709 12:59:08.393181  1061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5058095638664e428f3cf97e53848db2"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
mode: GRACEFUL
new_leader_uuid: "629e1bd745fb4bce886bc01856587fa3"
 from {username='slave'} at 127.0.0.1:56322
I20260709 12:59:08.393776  1061 raft_consensus.cc:606] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 4 LEADER]: Received request to transfer leadership to TS 629e1bd745fb4bce886bc01856587fa3
I20260709 12:59:08.396772  1061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
mode: GRACEFUL
new_leader_uuid: "9afc3caa620240e7a5a061700c100ea3"
 from {username='slave'} at 127.0.0.1:56322
I20260709 12:59:08.397341  1061 raft_consensus.cc:606] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 3 LEADER]: Received request to transfer leadership to TS 9afc3caa620240e7a5a061700c100ea3
I20260709 12:59:08.400211  1061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b"
dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
mode: GRACEFUL
new_leader_uuid: "9afc3caa620240e7a5a061700c100ea3"
 from {username='slave'} at 127.0.0.1:56322
I20260709 12:59:08.400767  1061 raft_consensus.cc:606] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Received request to transfer leadership to TS 9afc3caa620240e7a5a061700c100ea3
I20260709 12:59:08.401895   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:59:08.402472   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:08.505403  1403 raft_consensus.cc:993] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d: : Instructing follower 9afc3caa620240e7a5a061700c100ea3 to start an election
I20260709 12:59:08.506021  1401 raft_consensus.cc:1081] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 3 LEADER]: Signalling peer 9afc3caa620240e7a5a061700c100ea3 to start an election
I20260709 12:59:08.508038  1137 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab"
dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
 from {username='slave'} at 127.0.0.1:47730
I20260709 12:59:08.508705  1137 raft_consensus.cc:493] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:08.509008  1137 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:08.514937  1137 raft_consensus.cc:515] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.517375  1137 leader_election.cc:290] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 4 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:08.518375  1061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:08.518774  1207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81567809e6c1412d88a5becc05f305ab" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:08.519032  1061 raft_consensus.cc:3055] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:08.519377  1061 raft_consensus.cc:740] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:08.519510  1207 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:08.520215  1061 consensus_queue.cc:260] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.521654  1061 raft_consensus.cc:3060] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:08.527262  1207 raft_consensus.cc:2468] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 4.
I20260709 12:59:08.528795  1101 leader_election.cc:304] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: 
I20260709 12:59:08.529663  1371 raft_consensus.cc:2804] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:08.532033  1371 raft_consensus.cc:697] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 4 LEADER]: Becoming Leader. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:08.533087  1371 consensus_queue.cc:237] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.547611  1061 raft_consensus.cc:2468] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 4.
I20260709 12:59:08.561321   957 catalog_manager.cc:5697] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 reported cstate change: term changed from 3 to 4, leader changed from bc53c40ea20b40dda6d9516fa180772d (127.0.107.193) to 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "9afc3caa620240e7a5a061700c100ea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:08.691627  1405 raft_consensus.cc:993] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d: : Instructing follower 629e1bd745fb4bce886bc01856587fa3 to start an election
I20260709 12:59:08.692241  1403 raft_consensus.cc:1081] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 3 LEADER]: Signalling peer 629e1bd745fb4bce886bc01856587fa3 to start an election
I20260709 12:59:08.694065  1207 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b"
dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
 from {username='slave'} at 127.0.0.1:44236
I20260709 12:59:08.694861  1207 raft_consensus.cc:493] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:08.695226  1207 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:08.702255  1207 raft_consensus.cc:515] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.704329  1207 leader_election.cc:290] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 4 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791)
I20260709 12:59:08.705693  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:08.705581  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb1cda199a84d5bafed13f1a1d44b7b" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:08.706353  1137 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:08.706367  1062 raft_consensus.cc:3055] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:08.707002  1062 raft_consensus.cc:740] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:08.707959  1062 consensus_queue.cc:260] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.709436  1062 raft_consensus.cc:3060] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:08.713243  1137 raft_consensus.cc:2468] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e1bd745fb4bce886bc01856587fa3 in term 4.
I20260709 12:59:08.714812  1174 leader_election.cc:304] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: 
I20260709 12:59:08.715566  1402 raft_consensus.cc:2804] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:08.717069  1402 raft_consensus.cc:697] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 4 LEADER]: Becoming Leader. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Running, Role: LEADER
I20260709 12:59:08.718362  1402 consensus_queue.cc:237] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.720088  1062 raft_consensus.cc:2468] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e1bd745fb4bce886bc01856587fa3 in term 4.
I20260709 12:59:08.728845   957 catalog_manager.cc:5697] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 reported cstate change: term changed from 3 to 4, leader changed from bc53c40ea20b40dda6d9516fa180772d (127.0.107.193) to 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "629e1bd745fb4bce886bc01856587fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:08.769922  1405 raft_consensus.cc:993] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d: : Instructing follower 9afc3caa620240e7a5a061700c100ea3 to start an election
I20260709 12:59:08.770550  1403 raft_consensus.cc:1081] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Signalling peer 9afc3caa620240e7a5a061700c100ea3 to start an election
I20260709 12:59:08.772617  1137 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b"
dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
 from {username='slave'} at 127.0.0.1:47730
I20260709 12:59:08.773164  1137 raft_consensus.cc:493] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:08.773550  1137 raft_consensus.cc:3060] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:08.778442  1137 raft_consensus.cc:515] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.780222  1137 leader_election.cc:290] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 3 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195:33047)
I20260709 12:59:08.781365  1207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
I20260709 12:59:08.781437  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c183e9d3a64539b7ac6915ae583a4b" candidate_uuid: "9afc3caa620240e7a5a061700c100ea3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:08.782178  1207 raft_consensus.cc:3060] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:08.782294  1062 raft_consensus.cc:3055] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:08.782763  1062 raft_consensus.cc:740] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:08.784107  1062 consensus_queue.cc:260] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.786006  1062 raft_consensus.cc:3060] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:08.788818  1207 raft_consensus.cc:2468] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 3.
I20260709 12:59:08.790233  1101 leader_election.cc:304] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: 
I20260709 12:59:08.791061  1371 raft_consensus.cc:2804] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:08.791646  1371 raft_consensus.cc:697] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 3 LEADER]: Becoming Leader. State: Replica: 9afc3caa620240e7a5a061700c100ea3, State: Running, Role: LEADER
I20260709 12:59:08.792675  1371 consensus_queue.cc:237] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.798728  1062 raft_consensus.cc:2468] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9afc3caa620240e7a5a061700c100ea3 in term 3.
I20260709 12:59:08.803555   957 catalog_manager.cc:5697] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 reported cstate change: term changed from 2 to 3, leader changed from bc53c40ea20b40dda6d9516fa180772d (127.0.107.193) to 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "9afc3caa620240e7a5a061700c100ea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:08.840869  1405 raft_consensus.cc:993] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d: : Instructing follower 629e1bd745fb4bce886bc01856587fa3 to start an election
I20260709 12:59:08.841465  1403 raft_consensus.cc:1081] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 4 LEADER]: Signalling peer 629e1bd745fb4bce886bc01856587fa3 to start an election
I20260709 12:59:08.843788  1207 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5058095638664e428f3cf97e53848db2"
dest_uuid: "629e1bd745fb4bce886bc01856587fa3"
 from {username='slave'} at 127.0.0.1:44236
I20260709 12:59:08.844372  1207 raft_consensus.cc:493] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:08.844761  1207 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:08.851764  1207 raft_consensus.cc:515] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.854754  1207 leader_election.cc:290] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 5 election: Requested vote from peers bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871), 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791)
I20260709 12:59:08.855765  1137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9afc3caa620240e7a5a061700c100ea3"
I20260709 12:59:08.855708  1062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5058095638664e428f3cf97e53848db2" candidate_uuid: "629e1bd745fb4bce886bc01856587fa3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "bc53c40ea20b40dda6d9516fa180772d"
I20260709 12:59:08.856459  1137 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:08.856556  1062 raft_consensus.cc:3055] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:59:08.856956  1062 raft_consensus.cc:740] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: bc53c40ea20b40dda6d9516fa180772d, State: Running, Role: LEADER
I20260709 12:59:08.857969  1062 consensus_queue.cc:260] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.859439  1062 raft_consensus.cc:3060] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:08.865352  1137 raft_consensus.cc:2468] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e1bd745fb4bce886bc01856587fa3 in term 5.
I20260709 12:59:08.866850  1174 leader_election.cc:304] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e1bd745fb4bce886bc01856587fa3, 9afc3caa620240e7a5a061700c100ea3; no voters: 
I20260709 12:59:08.867657  1402 raft_consensus.cc:2804] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:59:08.868059  1062 raft_consensus.cc:2468] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e1bd745fb4bce886bc01856587fa3 in term 5.
I20260709 12:59:08.868181  1402 raft_consensus.cc:697] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 5 LEADER]: Becoming Leader. State: Replica: 629e1bd745fb4bce886bc01856587fa3, State: Running, Role: LEADER
I20260709 12:59:08.869316  1402 consensus_queue.cc:237] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } }
I20260709 12:59:08.878065   957 catalog_manager.cc:5697] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 reported cstate change: term changed from 4 to 5, leader changed from bc53c40ea20b40dda6d9516fa180772d (127.0.107.193) to 629e1bd745fb4bce886bc01856587fa3 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "629e1bd745fb4bce886bc01856587fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:09.075085  1207 raft_consensus.cc:1275] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 4 FOLLOWER]: Refusing update from remote peer 9afc3caa620240e7a5a061700c100ea3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:09.076555  1371 consensus_queue.cc:1048] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:09.088074  1062 raft_consensus.cc:1275] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 4 FOLLOWER]: Refusing update from remote peer 9afc3caa620240e7a5a061700c100ea3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:09.090590  1371 consensus_queue.cc:1048] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:09.226591  1062 raft_consensus.cc:1275] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 4 FOLLOWER]: Refusing update from remote peer 629e1bd745fb4bce886bc01856587fa3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:09.228186  1402 consensus_queue.cc:1048] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:09.239475  1137 raft_consensus.cc:1275] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 4 FOLLOWER]: Refusing update from remote peer 629e1bd745fb4bce886bc01856587fa3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:09.242825  1402 consensus_queue.cc:1048] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:09.325083  1062 raft_consensus.cc:1275] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Refusing update from remote peer 9afc3caa620240e7a5a061700c100ea3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:09.326712  1371 consensus_queue.cc:1048] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc53c40ea20b40dda6d9516fa180772d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:09.350301  1207 raft_consensus.cc:1275] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Refusing update from remote peer 9afc3caa620240e7a5a061700c100ea3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:09.353343  1371 consensus_queue.cc:1048] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e1bd745fb4bce886bc01856587fa3" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:59:09.403451   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:09.411245   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 12:59:09.450614   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:09.451651   431 tablet_replica.cc:333] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d: stopping tablet replica
I20260709 12:59:09.452689   431 raft_consensus.cc:2243] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.453433   431 raft_consensus.cc:2272] T 93594eee86c84ebd8bba2c1d2a934ed6 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:09.454494  1174 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
I20260709 12:59:09.456755   431 tablet_replica.cc:333] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d: stopping tablet replica
I20260709 12:59:09.457798   431 raft_consensus.cc:2243] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 4 FOLLOWER]: Raft consensus shutting down.
W20260709 12:59:09.458102  1174 consensus_peers.cc:597] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 -> Peer bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871): Couldn't send request to peer bc53c40ea20b40dda6d9516fa180772d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:09.458415   431 raft_consensus.cc:2272] T 4bb1cda199a84d5bafed13f1a1d44b7b P bc53c40ea20b40dda6d9516fa180772d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.461556   431 tablet_replica.cc:333] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d: stopping tablet replica
I20260709 12:59:09.462386   431 raft_consensus.cc:2243] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.463001   431 raft_consensus.cc:2272] T df3d601732374c73ad6f1b5a15956de9 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.465901   431 tablet_replica.cc:333] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d: stopping tablet replica
I20260709 12:59:09.466702   431 raft_consensus.cc:2243] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.467291   431 raft_consensus.cc:2272] T 81567809e6c1412d88a5becc05f305ab P bc53c40ea20b40dda6d9516fa180772d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.470479   431 tablet_replica.cc:333] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d: stopping tablet replica
I20260709 12:59:09.471422   431 raft_consensus.cc:2243] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:09.472420   431 raft_consensus.cc:2272] T 362f1af397b74a879245d9616bacf029 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.474838   431 tablet_replica.cc:333] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d: stopping tablet replica
I20260709 12:59:09.475600   431 raft_consensus.cc:2243] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.476150   431 raft_consensus.cc:2272] T f5c183e9d3a64539b7ac6915ae583a4b P bc53c40ea20b40dda6d9516fa180772d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.479275   431 tablet_replica.cc:333] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d: stopping tablet replica
I20260709 12:59:09.480090   431 raft_consensus.cc:2243] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.480592   431 raft_consensus.cc:2272] T 5058095638664e428f3cf97e53848db2 P bc53c40ea20b40dda6d9516fa180772d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.483377   431 tablet_replica.cc:333] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d: stopping tablet replica
I20260709 12:59:09.484177   431 raft_consensus.cc:2243] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:09.485286   431 raft_consensus.cc:2272] T daa76bc7ba174327b42ce50718d0123d P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.487985   431 tablet_replica.cc:333] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d: stopping tablet replica
I20260709 12:59:09.488729   431 raft_consensus.cc:2243] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:59:09.489230  1100 consensus_peers.cc:597] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 -> Peer bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871): Couldn't send request to peer bc53c40ea20b40dda6d9516fa180772d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:09.489681  1137 raft_consensus.cc:1275] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 5 FOLLOWER]: Refusing update from remote peer 629e1bd745fb4bce886bc01856587fa3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:59:09.489799   431 raft_consensus.cc:2272] T 262d0445f78c4bc5a1da87074cfd1101 P bc53c40ea20b40dda6d9516fa180772d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.491355  1412 consensus_queue.cc:1048] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9afc3caa620240e7a5a061700c100ea3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:09.492789   431 tablet_replica.cc:333] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d: stopping tablet replica
I20260709 12:59:09.493662   431 raft_consensus.cc:2243] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:09.494995   431 raft_consensus.cc:2272] T 7ed84eb1a8eb4ab08fb0353f201560c3 P bc53c40ea20b40dda6d9516fa180772d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.526036   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 12:59:09.552778   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 12:59:09.587898  1100 consensus_peers.cc:597] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 -> Peer bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871): Couldn't send request to peer bc53c40ea20b40dda6d9516fa180772d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:09.591212   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:09.592231   431 tablet_replica.cc:333] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3: stopping tablet replica
I20260709 12:59:09.593281   431 raft_consensus.cc:2243] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.593987   431 raft_consensus.cc:2272] T 4bb1cda199a84d5bafed13f1a1d44b7b P 9afc3caa620240e7a5a061700c100ea3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.597337   431 tablet_replica.cc:333] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3: stopping tablet replica
I20260709 12:59:09.598253   431 raft_consensus.cc:2243] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:59:09.599500   431 raft_consensus.cc:2272] T 81567809e6c1412d88a5becc05f305ab P 9afc3caa620240e7a5a061700c100ea3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.603175   431 tablet_replica.cc:333] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3: stopping tablet replica
I20260709 12:59:09.604023   431 raft_consensus.cc:2243] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.604647   431 raft_consensus.cc:2272] T df3d601732374c73ad6f1b5a15956de9 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.607426   431 tablet_replica.cc:333] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3: stopping tablet replica
I20260709 12:59:09.608721   431 raft_consensus.cc:2243] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.609274   431 raft_consensus.cc:2272] T 262d0445f78c4bc5a1da87074cfd1101 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:09.610819  1174 consensus_peers.cc:597] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 -> Peer 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791): Couldn't send request to peer 9afc3caa620240e7a5a061700c100ea3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:09.612035   431 tablet_replica.cc:333] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3: stopping tablet replica
I20260709 12:59:09.612895   431 raft_consensus.cc:2243] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:09.614154   431 raft_consensus.cc:2272] T 93594eee86c84ebd8bba2c1d2a934ed6 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.616874   431 tablet_replica.cc:333] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3: stopping tablet replica
I20260709 12:59:09.617725   431 raft_consensus.cc:2243] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:59:09.618825   431 raft_consensus.cc:2272] T f5c183e9d3a64539b7ac6915ae583a4b P 9afc3caa620240e7a5a061700c100ea3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.621963   431 tablet_replica.cc:333] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3: stopping tablet replica
I20260709 12:59:09.622809   431 raft_consensus.cc:2243] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.623400   431 raft_consensus.cc:2272] T 362f1af397b74a879245d9616bacf029 P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.626168   431 tablet_replica.cc:333] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3: stopping tablet replica
I20260709 12:59:09.626847   431 raft_consensus.cc:2243] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.627388   431 raft_consensus.cc:2272] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 9afc3caa620240e7a5a061700c100ea3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.630149   431 tablet_replica.cc:333] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3: stopping tablet replica
I20260709 12:59:09.630846   431 raft_consensus.cc:2243] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.631424   431 raft_consensus.cc:2272] T daa76bc7ba174327b42ce50718d0123d P 9afc3caa620240e7a5a061700c100ea3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.634351   431 tablet_replica.cc:333] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3: stopping tablet replica
I20260709 12:59:09.635169   431 raft_consensus.cc:2243] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.635876   431 raft_consensus.cc:2272] T 5058095638664e428f3cf97e53848db2 P 9afc3caa620240e7a5a061700c100ea3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.677443   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 12:59:09.714357   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 12:59:09.725366  1174 consensus_peers.cc:597] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 -> Peer 9afc3caa620240e7a5a061700c100ea3 (127.0.107.194:39791): Couldn't send request to peer 9afc3caa620240e7a5a061700c100ea3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39791: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:09.746387  1174 consensus_peers.cc:597] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 -> Peer bc53c40ea20b40dda6d9516fa180772d (127.0.107.193:40871): Couldn't send request to peer bc53c40ea20b40dda6d9516fa180772d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:09.763442   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:09.764346   431 tablet_replica.cc:333] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3: stopping tablet replica
I20260709 12:59:09.765116   431 raft_consensus.cc:2243] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.765756   431 raft_consensus.cc:2272] T 81567809e6c1412d88a5becc05f305ab P 629e1bd745fb4bce886bc01856587fa3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.768424   431 tablet_replica.cc:333] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3: stopping tablet replica
I20260709 12:59:09.769124   431 raft_consensus.cc:2243] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:59:09.770085   431 raft_consensus.cc:2272] T 4bb1cda199a84d5bafed13f1a1d44b7b P 629e1bd745fb4bce886bc01856587fa3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.772995   431 tablet_replica.cc:333] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3: stopping tablet replica
I20260709 12:59:09.773715   431 raft_consensus.cc:2243] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:09.774623   431 raft_consensus.cc:2272] T df3d601732374c73ad6f1b5a15956de9 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.776743   431 tablet_replica.cc:333] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3: stopping tablet replica
I20260709 12:59:09.777303   431 raft_consensus.cc:2243] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.777846   431 raft_consensus.cc:2272] T 262d0445f78c4bc5a1da87074cfd1101 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.779814   431 tablet_replica.cc:333] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3: stopping tablet replica
I20260709 12:59:09.780493   431 raft_consensus.cc:2243] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.780958   431 raft_consensus.cc:2272] T 7ed84eb1a8eb4ab08fb0353f201560c3 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.783198   431 tablet_replica.cc:333] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3: stopping tablet replica
I20260709 12:59:09.783856   431 raft_consensus.cc:2243] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:59:09.784870   431 raft_consensus.cc:2272] T 5058095638664e428f3cf97e53848db2 P 629e1bd745fb4bce886bc01856587fa3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.787302   431 tablet_replica.cc:333] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3: stopping tablet replica
I20260709 12:59:09.787925   431 raft_consensus.cc:2243] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.788424   431 raft_consensus.cc:2272] T daa76bc7ba174327b42ce50718d0123d P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.790469   431 tablet_replica.cc:333] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3: stopping tablet replica
I20260709 12:59:09.791041   431 raft_consensus.cc:2243] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.791636   431 raft_consensus.cc:2272] T 93594eee86c84ebd8bba2c1d2a934ed6 P 629e1bd745fb4bce886bc01856587fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.793681   431 tablet_replica.cc:333] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3: stopping tablet replica
I20260709 12:59:09.794221   431 raft_consensus.cc:2243] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.794673   431 raft_consensus.cc:2272] T 362f1af397b74a879245d9616bacf029 P 629e1bd745fb4bce886bc01856587fa3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:09.796474   431 tablet_replica.cc:333] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3: stopping tablet replica
I20260709 12:59:09.796972   431 raft_consensus.cc:2243] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:09.797420   431 raft_consensus.cc:2272] T f5c183e9d3a64539b7ac6915ae583a4b P 629e1bd745fb4bce886bc01856587fa3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:10.310374   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 12:59:10.332979   431 master.cc:562] Master@127.0.107.254:45121 shutting down...
I20260709 12:59:10.360282   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:10.360965   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:10.361336   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9d54dd2dafae4820b5d3b89770f30c2a: stopping tablet replica
I20260709 12:59:10.382877   431 master.cc:584] Master@127.0.107.254:45121 shutdown complete.
629e1bd745fb4bce886bc01856587fa3  1
9afc3caa620240e7a5a061700c100ea3  4
bc53c40ea20b40dda6d9516fa180772d  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (11633 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:59:10.430016   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:43857
I20260709 12:59:10.431318   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:10.437947  1435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:10.438153  1434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:10.438719  1437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:10.439880   431 server_base.cc:1061] running on GCE node
I20260709 12:59:10.440905   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:10.441188   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:10.441390   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601950441365 us; error 0 us; skew 500 ppm
I20260709 12:59:10.442162   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:10.445036   431 webserver.cc:533] Webserver started at http://127.0.107.254:36965/ using document root <none> and password file <none>
I20260709 12:59:10.445648   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:10.445873   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:10.446185   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:10.447419   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/master-0-root/instance:
uuid: "8f622dabf84e4c0595473d2c01f108d0"
format_stamp: "Formatted at 2026-07-09 12:59:10 on dist-test-slave-l8g5"
I20260709 12:59:10.452469   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:59:10.456485  1442 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:10.457458   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:59:10.457798   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/master-0-root
uuid: "8f622dabf84e4c0595473d2c01f108d0"
format_stamp: "Formatted at 2026-07-09 12:59:10 on dist-test-slave-l8g5"
I20260709 12:59:10.458073   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:10.477334   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:10.479429   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:10.540413   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:43857
I20260709 12:59:10.540649  1493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:43857 every 8 connection(s)
I20260709 12:59:10.545104  1494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:10.556547  1494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Bootstrap starting.
I20260709 12:59:10.563306  1494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:10.568663  1494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: No bootstrap required, opened a new log
I20260709 12:59:10.571530  1494 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f622dabf84e4c0595473d2c01f108d0" member_type: VOTER }
I20260709 12:59:10.571995  1494 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:10.572201  1494 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f622dabf84e4c0595473d2c01f108d0, State: Initialized, Role: FOLLOWER
I20260709 12:59:10.572871  1494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f622dabf84e4c0595473d2c01f108d0" member_type: VOTER }
I20260709 12:59:10.573359  1494 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:59:10.573621  1494 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:59:10.573871  1494 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.578972  1494 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f622dabf84e4c0595473d2c01f108d0" member_type: VOTER }
I20260709 12:59:10.579633  1494 leader_election.cc:304] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f622dabf84e4c0595473d2c01f108d0; no voters: 
I20260709 12:59:10.581013  1494 leader_election.cc:290] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:59:10.581413  1497 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:10.583039  1497 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8f622dabf84e4c0595473d2c01f108d0, State: Running, Role: LEADER
I20260709 12:59:10.583793  1497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f622dabf84e4c0595473d2c01f108d0" member_type: VOTER }
I20260709 12:59:10.584674  1494 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:59:10.588904  1499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f622dabf84e4c0595473d2c01f108d0. Latest consensus state: current_term: 1 leader_uuid: "8f622dabf84e4c0595473d2c01f108d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f622dabf84e4c0595473d2c01f108d0" member_type: VOTER } }
I20260709 12:59:10.588860  1498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f622dabf84e4c0595473d2c01f108d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f622dabf84e4c0595473d2c01f108d0" member_type: VOTER } }
I20260709 12:59:10.590157  1499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:10.590190  1498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:10.596580  1503 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:59:10.606081  1503 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:59:10.608371   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:59:10.616894  1503 catalog_manager.cc:1383] Generated new cluster ID: 83656cb1f01240be90c4869c8fe2c0f1
I20260709 12:59:10.617321  1503 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:59:10.655855  1503 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:59:10.657338  1503 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:59:10.672982  1503 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Generated new TSK 0
I20260709 12:59:10.673873  1503 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:59:10.740473   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:10.747910  1515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:10.749202  1516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:10.751384  1518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:10.752303   431 server_base.cc:1061] running on GCE node
I20260709 12:59:10.755319   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:10.755551   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:10.755802   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601950755759 us; error 0 us; skew 500 ppm
I20260709 12:59:10.756404   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:10.759470   431 webserver.cc:533] Webserver started at http://127.0.107.193:41331/ using document root <none> and password file <none>
I20260709 12:59:10.760111   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:10.760291   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:10.760633   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:10.761961   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/instance:
uuid: "d62ce354d09a4be59c29094425d09bc1"
format_stamp: "Formatted at 2026-07-09 12:59:10 on dist-test-slave-l8g5"
I20260709 12:59:10.767127   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:59:10.771132  1523 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:10.772140   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:59:10.772404   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root
uuid: "d62ce354d09a4be59c29094425d09bc1"
format_stamp: "Formatted at 2026-07-09 12:59:10 on dist-test-slave-l8g5"
I20260709 12:59:10.772785   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:10.793737   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:10.795179   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:10.796880   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:10.801227   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:10.801450   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:10.801685   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:10.801836   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:10.854132  1585 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:33149 every 8 connection(s)
I20260709 12:59:10.854815   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:33149
I20260709 12:59:10.861152   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:10.869884  1590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:10.871541  1591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:10.875572   431 server_base.cc:1061] running on GCE node
W20260709 12:59:10.877204  1594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:10.877424  1586 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43857
I20260709 12:59:10.877945  1586 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:10.878412   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:10.878692   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:10.878789  1586 heartbeater.cc:507] Master 127.0.107.254:43857 requested a full tablet report, sending...
I20260709 12:59:10.878906   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601950878883 us; error 0 us; skew 500 ppm
I20260709 12:59:10.879657   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:10.881394  1459 ts_manager.cc:194] Registered new tserver with Master: d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:10.882642   431 webserver.cc:533] Webserver started at http://127.0.107.194:43959/ using document root <none> and password file <none>
I20260709 12:59:10.883193   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:10.883435   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:10.883612  1459 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40384
I20260709 12:59:10.883725   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:10.885156   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/instance:
uuid: "ba0548b7d2a94e8598475a46c279fe0e"
format_stamp: "Formatted at 2026-07-09 12:59:10 on dist-test-slave-l8g5"
I20260709 12:59:10.890558   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:59:10.894536  1598 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:10.895524   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:59:10.895903   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root
uuid: "ba0548b7d2a94e8598475a46c279fe0e"
format_stamp: "Formatted at 2026-07-09 12:59:10 on dist-test-slave-l8g5"
I20260709 12:59:10.896239   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:10.930675   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:10.932075   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:10.933911   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:10.936426   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:10.936671   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:10.936929   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:10.937115   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:10.987376   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:37097
I20260709 12:59:10.987509  1660 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:37097 every 8 connection(s)
I20260709 12:59:10.993279   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:11.002277  1664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:11.005452  1665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:11.007977   431 server_base.cc:1061] running on GCE node
W20260709 12:59:11.009416  1667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:11.010638   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:11.010934   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:11.011092  1661 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43857
I20260709 12:59:11.011173   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601951011147 us; error 0 us; skew 500 ppm
I20260709 12:59:11.011533  1661 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:11.012069   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:11.012596  1661 heartbeater.cc:507] Master 127.0.107.254:43857 requested a full tablet report, sending...
I20260709 12:59:11.015401   431 webserver.cc:533] Webserver started at http://127.0.107.195:35717/ using document root <none> and password file <none>
I20260709 12:59:11.015463  1459 ts_manager.cc:194] Registered new tserver with Master: ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:11.016072   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:11.016324   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:11.016628   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:11.017234  1459 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40386
I20260709 12:59:11.017902   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/instance:
uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
format_stamp: "Formatted at 2026-07-09 12:59:11 on dist-test-slave-l8g5"
I20260709 12:59:11.023487   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:59:11.027493  1672 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:11.028498   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 12:59:11.028855   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root
uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
format_stamp: "Formatted at 2026-07-09 12:59:11 on dist-test-slave-l8g5"
I20260709 12:59:11.029196   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:11.041152   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:11.042589   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:11.044261   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:11.047405   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:11.047669   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:11.047930   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:11.048113   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:11.101253   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:45431
I20260709 12:59:11.101377  1734 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:45431 every 8 connection(s)
I20260709 12:59:11.117748  1735 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43857
I20260709 12:59:11.118213  1735 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:11.119980  1735 heartbeater.cc:507] Master 127.0.107.254:43857 requested a full tablet report, sending...
I20260709 12:59:11.122920  1459 ts_manager.cc:194] Registered new tserver with Master: a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:11.123781   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018787446s
I20260709 12:59:11.124529  1459 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40398
I20260709 12:59:11.156309  1459 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40410:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:59:11.163599  1459 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:59:11.395973  1700 tablet_service.cc:1511] Processing CreateTablet for tablet 8785cfce0d0042d3855cf9a7edaf1f7d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:11.397699  1700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8785cfce0d0042d3855cf9a7edaf1f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.405144  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 0cb744c0184549918fb0bb586614bb7f (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:11.405618  1698 tablet_service.cc:1511] Processing CreateTablet for tablet 26a92c141f564f638497aa02dd5c7c79 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:11.406912  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cb744c0184549918fb0bb586614bb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.408195  1623 tablet_service.cc:1511] Processing CreateTablet for tablet 61a6c336f56b4c248aac90993fb477cf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:11.408638  1697 tablet_service.cc:1511] Processing CreateTablet for tablet 61a6c336f56b4c248aac90993fb477cf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:11.409747  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61a6c336f56b4c248aac90993fb477cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.410058  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61a6c336f56b4c248aac90993fb477cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.409826  1551 tablet_service.cc:1511] Processing CreateTablet for tablet 8785cfce0d0042d3855cf9a7edaf1f7d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:11.411194  1551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8785cfce0d0042d3855cf9a7edaf1f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.413920  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a92c141f564f638497aa02dd5c7c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.414705  1626 tablet_service.cc:1511] Processing CreateTablet for tablet 8785cfce0d0042d3855cf9a7edaf1f7d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:11.416136  1626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8785cfce0d0042d3855cf9a7edaf1f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.416895  1550 tablet_service.cc:1511] Processing CreateTablet for tablet 0cb744c0184549918fb0bb586614bb7f (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:11.418290  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cb744c0184549918fb0bb586614bb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.418812  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 0cb744c0184549918fb0bb586614bb7f (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:11.421217  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cb744c0184549918fb0bb586614bb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.422811  1696 tablet_service.cc:1511] Processing CreateTablet for tablet bc25e0c8bb834899a98a40efe1e66bba (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:11.424325  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc25e0c8bb834899a98a40efe1e66bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.426798  1624 tablet_service.cc:1511] Processing CreateTablet for tablet 26a92c141f564f638497aa02dd5c7c79 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:11.428371  1549 tablet_service.cc:1511] Processing CreateTablet for tablet 26a92c141f564f638497aa02dd5c7c79 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:11.430320  1622 tablet_service.cc:1511] Processing CreateTablet for tablet bc25e0c8bb834899a98a40efe1e66bba (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:11.430621  1545 tablet_service.cc:1511] Processing CreateTablet for tablet 74f866a699e74063990bd566c7fb1ebf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:11.432147  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74f866a699e74063990bd566c7fb1ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.431804  1547 tablet_service.cc:1511] Processing CreateTablet for tablet bc25e0c8bb834899a98a40efe1e66bba (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:11.433172  1547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc25e0c8bb834899a98a40efe1e66bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.434703  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 2f47909c83d54b699ec2b1c2402084b4 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:11.435503  1544 tablet_service.cc:1511] Processing CreateTablet for tablet c96b268f1f734ec793355ab247e28a75 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:11.430657  1548 tablet_service.cc:1511] Processing CreateTablet for tablet 61a6c336f56b4c248aac90993fb477cf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:11.437830  1694 tablet_service.cc:1511] Processing CreateTablet for tablet 74f866a699e74063990bd566c7fb1ebf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:11.439749  1544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c96b268f1f734ec793355ab247e28a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.441530  1620 tablet_service.cc:1511] Processing CreateTablet for tablet 74f866a699e74063990bd566c7fb1ebf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:11.443632  1695 tablet_service.cc:1511] Processing CreateTablet for tablet 2f47909c83d54b699ec2b1c2402084b4 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:11.435261  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 2f47909c83d54b699ec2b1c2402084b4 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:11.445879  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61a6c336f56b4c248aac90993fb477cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.439779  1694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74f866a699e74063990bd566c7fb1ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.450906  1695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f47909c83d54b699ec2b1c2402084b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.449836  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f47909c83d54b699ec2b1c2402084b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.454017  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a92c141f564f638497aa02dd5c7c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.461866  1624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a92c141f564f638497aa02dd5c7c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.465994  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f47909c83d54b699ec2b1c2402084b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.470001  1620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74f866a699e74063990bd566c7fb1ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.474056  1622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc25e0c8bb834899a98a40efe1e66bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.518358  1754 tablet_bootstrap.cc:492] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:11.530201  1753 tablet_bootstrap.cc:492] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:11.533650  1754 tablet_bootstrap.cc:654] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:11.539935  1699 tablet_service.cc:1511] Processing CreateTablet for tablet c96b268f1f734ec793355ab247e28a75 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:11.541155  1544 tablet_service.cc:1511] Processing CreateTablet for tablet e6c08bb7c6294cf9b33ad3b85e34831d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:11.542779  1544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6c08bb7c6294cf9b33ad3b85e34831d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.542444  1618 tablet_service.cc:1511] Processing CreateTablet for tablet e6c08bb7c6294cf9b33ad3b85e34831d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:11.544056  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6c08bb7c6294cf9b33ad3b85e34831d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.545567  1619 tablet_service.cc:1511] Processing CreateTablet for tablet c96b268f1f734ec793355ab247e28a75 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:11.547077  1619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c96b268f1f734ec793355ab247e28a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.549675  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c96b268f1f734ec793355ab247e28a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.558146  1693 tablet_service.cc:1511] Processing CreateTablet for tablet e6c08bb7c6294cf9b33ad3b85e34831d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:11.559965  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6c08bb7c6294cf9b33ad3b85e34831d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.566102  1551 tablet_service.cc:1511] Processing CreateTablet for tablet 1457f31daeb646a9824ac8068949eddd (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:11.568405  1551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1457f31daeb646a9824ac8068949eddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.580647  1698 tablet_service.cc:1511] Processing CreateTablet for tablet 1457f31daeb646a9824ac8068949eddd (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:11.582222  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1457f31daeb646a9824ac8068949eddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.587239  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 1457f31daeb646a9824ac8068949eddd (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:11.589164  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1457f31daeb646a9824ac8068949eddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.596998  1755 tablet_bootstrap.cc:492] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:11.598883  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 7db3e87d77f44ba19ad756277b557c45 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:11.600288  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7db3e87d77f44ba19ad756277b557c45. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:11.603050  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:11.620118  1754 tablet_bootstrap.cc:492] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:11.620707  1754 ts_tablet_manager.cc:1403] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.103s	user 0.012s	sys 0.003s
I20260709 12:59:11.621223  1549 tablet_service.cc:1511] Processing CreateTablet for tablet 29b074786b944dd19df72b17c4fbb001 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:11.622289  1700 tablet_service.cc:1511] Processing CreateTablet for tablet 7db3e87d77f44ba19ad756277b557c45 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:11.623607  1700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7db3e87d77f44ba19ad756277b557c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.622715  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29b074786b944dd19df72b17c4fbb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.626489  1623 tablet_service.cc:1511] Processing CreateTablet for tablet 29b074786b944dd19df72b17c4fbb001 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:11.632165  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29b074786b944dd19df72b17c4fbb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.632200  1692 tablet_service.cc:1511] Processing CreateTablet for tablet 29b074786b944dd19df72b17c4fbb001 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:11.636512  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29b074786b944dd19df72b17c4fbb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.642732  1617 tablet_service.cc:1511] Processing CreateTablet for tablet 43dce0aaab204cb084a73a74a80bd7a3 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:11.644299  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43dce0aaab204cb084a73a74a80bd7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.655386  1691 tablet_service.cc:1511] Processing CreateTablet for tablet 43dce0aaab204cb084a73a74a80bd7a3 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:11.656961  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43dce0aaab204cb084a73a74a80bd7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.657261  1548 tablet_service.cc:1511] Processing CreateTablet for tablet 43dce0aaab204cb084a73a74a80bd7a3 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:11.657269  1550 tablet_service.cc:1511] Processing CreateTablet for tablet 6b73223ab67f4b5aab700c65a70e42cf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:11.660384  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b73223ab67f4b5aab700c65a70e42cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.666251  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43dce0aaab204cb084a73a74a80bd7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.674173  1624 tablet_service.cc:1511] Processing CreateTablet for tablet 7db3e87d77f44ba19ad756277b557c45 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:11.675609  1624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7db3e87d77f44ba19ad756277b557c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.651856  1753 tablet_bootstrap.cc:654] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:11.678926  1754 raft_consensus.cc:359] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 12:59:11.679714  1754 raft_consensus.cc:385] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:11.680043  1754 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:11.680924  1754 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 12:59:11.684978  1755 tablet_bootstrap.cc:654] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:11.688051  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 6b73223ab67f4b5aab700c65a70e42cf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:11.693387  1693 tablet_service.cc:1511] Processing CreateTablet for tablet 6b73223ab67f4b5aab700c65a70e42cf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:11.695081  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b73223ab67f4b5aab700c65a70e42cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.699460  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 9a353430d15945ee9524da3d2b667c90 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:11.700901  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a353430d15945ee9524da3d2b667c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.716692  1753 tablet_bootstrap.cc:492] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:11.717250  1753 ts_tablet_manager.cc:1403] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.187s	user 0.003s	sys 0.012s
I20260709 12:59:11.716331  1623 tablet_service.cc:1511] Processing CreateTablet for tablet 9a353430d15945ee9524da3d2b667c90 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:11.720552  1753 raft_consensus.cc:359] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:11.721463  1753 raft_consensus.cc:385] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:11.721817  1753 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:11.722648  1754 ts_tablet_manager.cc:1434] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.102s	user 0.009s	sys 0.003s
I20260709 12:59:11.722766  1753 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:11.725587  1586 heartbeater.cc:499] Master 127.0.107.254:43857 was elected leader, sending a full tablet report...
I20260709 12:59:11.733949  1754 tablet_bootstrap.cc:492] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:11.749418  1551 tablet_service.cc:1511] Processing CreateTablet for tablet 69b9dbe96d28487fa6d8a381ac6fc383 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:11.730798  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b73223ab67f4b5aab700c65a70e42cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.758455  1753 ts_tablet_manager.cc:1434] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.041s	user 0.023s	sys 0.012s
I20260709 12:59:11.758404  1735 heartbeater.cc:499] Master 127.0.107.254:43857 was elected leader, sending a full tablet report...
I20260709 12:59:11.761102  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 69b9dbe96d28487fa6d8a381ac6fc383 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:11.762660  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69b9dbe96d28487fa6d8a381ac6fc383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.762827  1551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69b9dbe96d28487fa6d8a381ac6fc383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.762411  1694 tablet_service.cc:1511] Processing CreateTablet for tablet 696fb3c290c54f14a67f61bf78588eda (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:11.763872  1694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 696fb3c290c54f14a67f61bf78588eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.758214  1697 tablet_service.cc:1511] Processing CreateTablet for tablet 9a353430d15945ee9524da3d2b667c90 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:11.766316  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a353430d15945ee9524da3d2b667c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.766582  1544 tablet_service.cc:1511] Processing CreateTablet for tablet 696fb3c290c54f14a67f61bf78588eda (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:11.767900  1544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 696fb3c290c54f14a67f61bf78588eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.764914  1626 tablet_service.cc:1511] Processing CreateTablet for tablet 69b9dbe96d28487fa6d8a381ac6fc383 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:11.773898  1759 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:11.774515  1759 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:11.774749  1617 tablet_service.cc:1511] Processing CreateTablet for tablet 696fb3c290c54f14a67f61bf78588eda (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:11.759821  1753 tablet_bootstrap.cc:492] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:11.784935  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 696fb3c290c54f14a67f61bf78588eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.799813  1759 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:11.814754  1758 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:11.815510  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a353430d15945ee9524da3d2b667c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.815373  1758 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 12:59:11.821838  1758 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:11.836407  1626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69b9dbe96d28487fa6d8a381ac6fc383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.871078  1754 tablet_bootstrap.cc:654] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:11.874754  1549 tablet_service.cc:1511] Processing CreateTablet for tablet c2ba61e8020a4800b9e7fdae5533800b (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:11.882345  1755 tablet_bootstrap.cc:492] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:11.883234  1755 ts_tablet_manager.cc:1403] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.287s	user 0.018s	sys 0.018s
I20260709 12:59:11.896914  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2ba61e8020a4800b9e7fdae5533800b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.925977  1545 tablet_service.cc:1511] Processing CreateTablet for tablet e6c211dcb5c947719413e384c6a72769 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:11.927610  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6c211dcb5c947719413e384c6a72769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.941282  1692 tablet_service.cc:1511] Processing CreateTablet for tablet c2ba61e8020a4800b9e7fdae5533800b (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:11.942817  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2ba61e8020a4800b9e7fdae5533800b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.946609  1691 tablet_service.cc:1511] Processing CreateTablet for tablet e6c211dcb5c947719413e384c6a72769 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:11.947990  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6c211dcb5c947719413e384c6a72769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.953536  1755 raft_consensus.cc:359] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:11.954527  1755 raft_consensus.cc:385] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:11.954890  1755 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:11.955750  1755 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:11.963119  1625 tablet_service.cc:1511] Processing CreateTablet for tablet c2ba61e8020a4800b9e7fdae5533800b (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:11.965284  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2ba61e8020a4800b9e7fdae5533800b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.971591  1621 tablet_service.cc:1511] Processing CreateTablet for tablet e6c211dcb5c947719413e384c6a72769 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:11.972112  1620 tablet_service.cc:1511] Processing CreateTablet for tablet 8b022b45a9b7462bb4b1a2a30ae36aa9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:11.973348  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6c211dcb5c947719413e384c6a72769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:11.976598  1753 tablet_bootstrap.cc:654] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:12.003854  1620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b022b45a9b7462bb4b1a2a30ae36aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.019826  1661 heartbeater.cc:499] Master 127.0.107.254:43857 was elected leader, sending a full tablet report...
I20260709 12:59:12.028846  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 8b022b45a9b7462bb4b1a2a30ae36aa9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:12.036981  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b022b45a9b7462bb4b1a2a30ae36aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.037770  1755 ts_tablet_manager.cc:1434] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.154s	user 0.007s	sys 0.012s
I20260709 12:59:12.039042  1755 tablet_bootstrap.cc:492] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:12.046280  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:12.048429  1526 leader_election.cc:343] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.049280  1769 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:12.050205  1769 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.055823  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 12:59:12.057883  1674 leader_election.cc:343] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.058851  1623 tablet_service.cc:1511] Processing CreateTablet for tablet 0adeaf85877a4103bbd070927413d267 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:12.059929  1693 tablet_service.cc:1511] Processing CreateTablet for tablet 8b022b45a9b7462bb4b1a2a30ae36aa9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:12.061429  1551 tablet_service.cc:1511] Processing CreateTablet for tablet 0adeaf85877a4103bbd070927413d267 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:12.063051  1551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0adeaf85877a4103bbd070927413d267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.062776  1696 tablet_service.cc:1511] Processing CreateTablet for tablet 0adeaf85877a4103bbd070927413d267 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:12.071724  1618 tablet_service.cc:1511] Processing CreateTablet for tablet eb99ad78ebb2479fbf4715bb9d33258e (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:12.072419  1545 tablet_service.cc:1511] Processing CreateTablet for tablet eb99ad78ebb2479fbf4715bb9d33258e (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:12.073035  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:12.074771  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0adeaf85877a4103bbd070927413d267. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:12.075177  1525 leader_election.cc:343] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.075739  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:12.076195  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb99ad78ebb2479fbf4715bb9d33258e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.077291  1763 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:12.071300  1699 tablet_service.cc:1511] Processing CreateTablet for tablet eb99ad78ebb2479fbf4715bb9d33258e (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:12.081596  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b022b45a9b7462bb4b1a2a30ae36aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.087266  1549 tablet_service.cc:1511] Processing CreateTablet for tablet 169e9f6ed9684060b36e93c778925461 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:12.089012  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 169e9f6ed9684060b36e93c778925461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.085229  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0adeaf85877a4103bbd070927413d267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.090260  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb99ad78ebb2479fbf4715bb9d33258e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.090741  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb99ad78ebb2479fbf4715bb9d33258e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.107884  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 12:59:12.110966  1675 leader_election.cc:343] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.111546  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:12.112933  1759 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:12.122871  1755 tablet_bootstrap.cc:654] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:12.127550  1754 tablet_bootstrap.cc:492] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:12.130507  1753 tablet_bootstrap.cc:492] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:12.148413  1753 ts_tablet_manager.cc:1403] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.389s	user 0.023s	sys 0.021s
I20260709 12:59:12.164628  1754 ts_tablet_manager.cc:1403] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.431s	user 0.013s	sys 0.014s
I20260709 12:59:12.165830  1769 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:12.168146  1754 raft_consensus.cc:359] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.168985  1754 raft_consensus.cc:385] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:12.172082  1691 tablet_service.cc:1511] Processing CreateTablet for tablet 169e9f6ed9684060b36e93c778925461 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:12.175575  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 169e9f6ed9684060b36e93c778925461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.152035  1753 raft_consensus.cc:359] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.178532  1753 raft_consensus.cc:385] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:12.178956  1753 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:12.180019  1753 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.169286  1754 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:12.190526  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 169e9f6ed9684060b36e93c778925461 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:12.194566  1697 tablet_service.cc:1511] Processing CreateTablet for tablet 70981cbf9c1143309f0f78706fe5931d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:12.195717  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 169e9f6ed9684060b36e93c778925461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.196223  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70981cbf9c1143309f0f78706fe5931d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.192246  1544 tablet_service.cc:1511] Processing CreateTablet for tablet b5838267321348eb83df088988da6449 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:12.197970  1551 tablet_service.cc:1511] Processing CreateTablet for tablet 70981cbf9c1143309f0f78706fe5931d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:12.198503  1544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5838267321348eb83df088988da6449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.199398  1551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70981cbf9c1143309f0f78706fe5931d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.205844  1620 tablet_service.cc:1511] Processing CreateTablet for tablet b5838267321348eb83df088988da6449 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:12.207425  1620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5838267321348eb83df088988da6449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.210682  1753 ts_tablet_manager.cc:1434] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.062s	user 0.004s	sys 0.004s
I20260709 12:59:12.211462  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 70981cbf9c1143309f0f78706fe5931d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:12.212914  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70981cbf9c1143309f0f78706fe5931d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.218321  1546 tablet_service.cc:1511] Processing CreateTablet for tablet f5aa381f739e4c71869e157bdba8aa97 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:12.220104  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5aa381f739e4c71869e157bdba8aa97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.220388  1617 tablet_service.cc:1511] Processing CreateTablet for tablet f5aa381f739e4c71869e157bdba8aa97 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:12.222007  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5aa381f739e4c71869e157bdba8aa97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.225209  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:12.225449  1754 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.230242  1753 tablet_bootstrap.cc:492] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:12.233294  1550 tablet_service.cc:1511] Processing CreateTablet for tablet e6edcffcaa5a46d2ba7c824073943b82 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:12.234681  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6edcffcaa5a46d2ba7c824073943b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.234611  1625 tablet_service.cc:1511] Processing CreateTablet for tablet e6edcffcaa5a46d2ba7c824073943b82 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:12.236008  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6edcffcaa5a46d2ba7c824073943b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.238613  1696 tablet_service.cc:1511] Processing CreateTablet for tablet f5aa381f739e4c71869e157bdba8aa97 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:12.240042  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5aa381f739e4c71869e157bdba8aa97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.244725  1693 tablet_service.cc:1511] Processing CreateTablet for tablet e6edcffcaa5a46d2ba7c824073943b82 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:12.246208  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6edcffcaa5a46d2ba7c824073943b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.247200  1754 ts_tablet_manager.cc:1434] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.082s	user 0.002s	sys 0.006s
I20260709 12:59:12.255026  1548 tablet_service.cc:1511] Processing CreateTablet for tablet 946eea23a2334d678895281941dd9926 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:12.256523  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 946eea23a2334d678895281941dd9926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.260329  1754 tablet_bootstrap.cc:492] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
W20260709 12:59:12.264451  1601 leader_election.cc:343] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.266481  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:12.268353  1601 leader_election.cc:343] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.268751  1754 tablet_bootstrap.cc:654] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:12.269043  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:12.270166  1769 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:12.275902  1623 tablet_service.cc:1511] Processing CreateTablet for tablet 946eea23a2334d678895281941dd9926 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:12.277719  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 946eea23a2334d678895281941dd9926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.278301  1692 tablet_service.cc:1511] Processing CreateTablet for tablet b5838267321348eb83df088988da6449 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:12.279654  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5838267321348eb83df088988da6449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.291612  1755 tablet_bootstrap.cc:492] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:12.292423  1755 ts_tablet_manager.cc:1403] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.254s	user 0.013s	sys 0.003s
I20260709 12:59:12.303439  1548 tablet_service.cc:1511] Processing CreateTablet for tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:12.310462  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.312137  1697 tablet_service.cc:1511] Processing CreateTablet for tablet 946eea23a2334d678895281941dd9926 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:12.317183  1626 tablet_service.cc:1511] Processing CreateTablet for tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:12.323987  1626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.329423  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 946eea23a2334d678895281941dd9926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.331570  1754 tablet_bootstrap.cc:492] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:12.332171  1754 ts_tablet_manager.cc:1403] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.072s	user 0.017s	sys 0.006s
I20260709 12:59:12.335391  1754 raft_consensus.cc:359] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.338388  1754 raft_consensus.cc:385] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:12.338723  1754 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:12.342908  1755 raft_consensus.cc:359] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.340883  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:12.347246  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.343683  1754 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.348951  1550 tablet_service.cc:1511] Processing CreateTablet for tablet 10315b15aaee40e0a1110e5ce8cf0abf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:12.351987  1622 tablet_service.cc:1511] Processing CreateTablet for tablet 10315b15aaee40e0a1110e5ce8cf0abf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:12.353237  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10315b15aaee40e0a1110e5ce8cf0abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.353384  1622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10315b15aaee40e0a1110e5ce8cf0abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.355732  1693 tablet_service.cc:1511] Processing CreateTablet for tablet 10315b15aaee40e0a1110e5ce8cf0abf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:12.357582  1545 tablet_service.cc:1511] Processing CreateTablet for tablet 05b3e259298f4617a9f83f3e7cf738ac (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:59:12.358923  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b3e259298f4617a9f83f3e7cf738ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.360489  1753 tablet_bootstrap.cc:654] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:12.343744  1755 raft_consensus.cc:385] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:12.369550  1755 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:12.377152  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10315b15aaee40e0a1110e5ce8cf0abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.378899  1754 ts_tablet_manager.cc:1434] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 12:59:12.373912  1755 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.380079  1754 tablet_bootstrap.cc:492] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:12.392843  1623 tablet_service.cc:1511] Processing CreateTablet for tablet 05b3e259298f4617a9f83f3e7cf738ac (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:59:12.394547  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b3e259298f4617a9f83f3e7cf738ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.398542  1763 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:12.399112  1763 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.401757  1763 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:12.403079  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:12.403266  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:12.404989  1525 leader_election.cc:343] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:12.405668  1526 leader_election.cc:343] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.405885  1755 ts_tablet_manager.cc:1434] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.113s	user 0.007s	sys 0.006s
I20260709 12:59:12.406173  1526 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:12.406944  1755 tablet_bootstrap.cc:492] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:12.407122  1763 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:12.412168  1754 tablet_bootstrap.cc:654] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:12.412863  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 3897029c4b5f443296641c39af287e98 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:12.413188  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 3897029c4b5f443296641c39af287e98 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:12.414474  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3897029c4b5f443296641c39af287e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.414538  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3897029c4b5f443296641c39af287e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.415369  1753 tablet_bootstrap.cc:492] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:12.415894  1753 ts_tablet_manager.cc:1403] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.186s	user 0.009s	sys 0.021s
I20260709 12:59:12.418691  1692 tablet_service.cc:1511] Processing CreateTablet for tablet 05b3e259298f4617a9f83f3e7cf738ac (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:59:12.419528  1753 raft_consensus.cc:359] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 12:59:12.420334  1753 raft_consensus.cc:385] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:12.420671  1753 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:12.430946  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b3e259298f4617a9f83f3e7cf738ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.431300  1753 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 12:59:12.433746  1753 ts_tablet_manager.cc:1434] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.004s
I20260709 12:59:12.436122  1753 tablet_bootstrap.cc:492] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:12.430894  1755 tablet_bootstrap.cc:654] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:12.455477  1696 tablet_service.cc:1511] Processing CreateTablet for tablet 3897029c4b5f443296641c39af287e98 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:12.457708  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3897029c4b5f443296641c39af287e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.458981  1753 tablet_bootstrap.cc:654] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:12.460537  1754 tablet_bootstrap.cc:492] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:12.461102  1754 ts_tablet_manager.cc:1403] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.081s	user 0.013s	sys 0.023s
I20260709 12:59:12.463107  1548 tablet_service.cc:1511] Processing CreateTablet for tablet 0835fee76231431da297ec1cf367640d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:12.471621  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0835fee76231431da297ec1cf367640d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.474450  1754 raft_consensus.cc:359] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.485869  1754 raft_consensus.cc:385] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:12.486222  1754 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:12.476257  1753 tablet_bootstrap.cc:492] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:12.477900  1623 tablet_service.cc:1511] Processing CreateTablet for tablet 0835fee76231431da297ec1cf367640d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:12.487749  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0835fee76231431da297ec1cf367640d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.488111  1755 tablet_bootstrap.cc:492] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:12.488621  1755 ts_tablet_manager.cc:1403] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.002s
I20260709 12:59:12.480072  1545 tablet_service.cc:1511] Processing CreateTablet for tablet 563a5aa85e754fe48eaf5615d947ab9a (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:12.493827  1753 ts_tablet_manager.cc:1403] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.058s	user 0.019s	sys 0.002s
I20260709 12:59:12.494351  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563a5aa85e754fe48eaf5615d947ab9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.481382  1626 tablet_service.cc:1511] Processing CreateTablet for tablet 563a5aa85e754fe48eaf5615d947ab9a (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:12.498500  1626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563a5aa85e754fe48eaf5615d947ab9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.501776  1754 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.483120  1691 tablet_service.cc:1511] Processing CreateTablet for tablet 563a5aa85e754fe48eaf5615d947ab9a (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:12.503515  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563a5aa85e754fe48eaf5615d947ab9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.479347  1697 tablet_service.cc:1511] Processing CreateTablet for tablet 0835fee76231431da297ec1cf367640d (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:12.506475  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0835fee76231431da297ec1cf367640d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.497092  1753 raft_consensus.cc:359] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.507141  1753 raft_consensus.cc:385] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:12.507464  1753 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:12.508286  1753 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.511144  1754 ts_tablet_manager.cc:1434] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.050s	user 0.013s	sys 0.005s
I20260709 12:59:12.512532  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 37bfe7ffa6604774b0b035da58f96eb9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:59:12.513989  1753 ts_tablet_manager.cc:1434] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:59:12.515131  1753 tablet_bootstrap.cc:492] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:12.522024  1753 tablet_bootstrap.cc:654] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:12.522523  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37bfe7ffa6604774b0b035da58f96eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.494066  1755 raft_consensus.cc:359] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.529834  1755 raft_consensus.cc:385] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:12.530359  1755 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:12.530074  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 37bfe7ffa6604774b0b035da58f96eb9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:59:12.531868  1754 tablet_bootstrap.cc:492] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:12.531559  1755 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.536510  1769 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:12.537094  1622 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac4670b4c614e9eb392b74198615198 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:12.531042  1696 tablet_service.cc:1511] Processing CreateTablet for tablet 37bfe7ffa6604774b0b035da58f96eb9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:59:12.536249  1550 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac4670b4c614e9eb392b74198615198 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:12.547745  1754 tablet_bootstrap.cc:654] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:12.548071  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37bfe7ffa6604774b0b035da58f96eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.531704  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37bfe7ffa6604774b0b035da58f96eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.550864  1622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac4670b4c614e9eb392b74198615198. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.549721  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac4670b4c614e9eb392b74198615198. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.555001  1755 ts_tablet_manager.cc:1434] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 12:59:12.545935  1769 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.576081  1769 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:12.556108  1551 tablet_service.cc:1511] Processing CreateTablet for tablet bb0b8480ae5646bea5cd44d499b6bcaa (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:59:12.577454  1551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0b8480ae5646bea5cd44d499b6bcaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.578506  1625 tablet_service.cc:1511] Processing CreateTablet for tablet fb91933caa7241dc9c723195dd6950dc (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:12.579921  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb91933caa7241dc9c723195dd6950dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.561144  1759 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:12.581072  1699 tablet_service.cc:1511] Processing CreateTablet for tablet fb91933caa7241dc9c723195dd6950dc (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:12.581768  1759 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.582793  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb91933caa7241dc9c723195dd6950dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.584293  1759 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:12.584815  1759 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:12.562793  1621 tablet_service.cc:1511] Processing CreateTablet for tablet bb0b8480ae5646bea5cd44d499b6bcaa (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:59:12.564224  1763 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:12.585788  1763 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.586338  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0b8480ae5646bea5cd44d499b6bcaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.589563  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:12.589982  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:12.573566  1693 tablet_service.cc:1511] Processing CreateTablet for tablet bb0b8480ae5646bea5cd44d499b6bcaa (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:59:12.590691  1635 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:12.591399  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0b8480ae5646bea5cd44d499b6bcaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.591295  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:12.592180  1674 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:12.589466  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 12:59:12.593003  1601 leader_election.cc:343] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.593405  1636 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
W20260709 12:59:12.594501  1526 leader_election.cc:343] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.595090  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:12.595340  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:12.596819  1601 leader_election.cc:343] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.597252  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:12.585224  1759 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 12:59:12.601601  1763 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:12.602192  1763 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:12.602571  1763 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:12.602906  1763 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:12.603300  1548 tablet_service.cc:1511] Processing CreateTablet for tablet fb91933caa7241dc9c723195dd6950dc (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:12.604911  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb91933caa7241dc9c723195dd6950dc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:12.605635  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:12.605691  1769 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:12.607847  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 12:59:12.609602  1674 leader_election.cc:343] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.610306  1759 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:12.610901  1759 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:12.611326  1759 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:12.611739  1759 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:12.541320  1692 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac4670b4c614e9eb392b74198615198 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:12.614116  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac4670b4c614e9eb392b74198615198. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.617779  1617 tablet_service.cc:1511] Processing CreateTablet for tablet 0fb354b7d6104034b98fab8201f2719e (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:12.619398  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fb354b7d6104034b98fab8201f2719e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.620558  1755 tablet_bootstrap.cc:492] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:12.634322  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 12:59:12.637840  1675 leader_election.cc:343] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:12.638756  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 43985us
I20260709 12:59:12.641124  1544 tablet_service.cc:1511] Processing CreateTablet for tablet 0fb354b7d6104034b98fab8201f2719e (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:12.651473  1763 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.657852  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:12.658660  1561 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:12.659329  1696 tablet_service.cc:1511] Processing CreateTablet for tablet 0fb354b7d6104034b98fab8201f2719e (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:12.660586  1544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fb354b7d6104034b98fab8201f2719e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.660665  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fb354b7d6104034b98fab8201f2719e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.663374  1549 tablet_service.cc:1511] Processing CreateTablet for tablet 316c6fe30c8f416c942b4338f4e6eab9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:12.664000  1781 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:12.664860  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 316c6fe30c8f416c942b4338f4e6eab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.664530  1781 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.666502  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:12.668035  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:12.669620  1526 leader_election.cc:343] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.669871  1780 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:12.670432  1780 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:12.670790  1781 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:12.670854  1780 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:12.674918  1622 tablet_service.cc:1511] Processing CreateTablet for tablet 316c6fe30c8f416c942b4338f4e6eab9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:12.676359  1622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 316c6fe30c8f416c942b4338f4e6eab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.684523  1694 tablet_service.cc:1511] Processing CreateTablet for tablet 316c6fe30c8f416c942b4338f4e6eab9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:12.685310  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:12.686034  1636 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:12.686447  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:12.686733  1763 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:12.687888  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 12:59:12.688133  1525 leader_election.cc:343] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.663359  1759 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.688651  1525 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 12:59:12.689440  1526 leader_election.cc:343] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.686043  1694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 316c6fe30c8f416c942b4338f4e6eab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.690524  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 6b24efe048e6499b923f1a0215713fad (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:12.692085  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b24efe048e6499b923f1a0215713fad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.698324  1763 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:12.699720  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 12:59:12.705902  1675 leader_election.cc:343] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.706960  1636 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:12.708259  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:12.709460  1763 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:12.709982  1763 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:12.710847  1763 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.711781  1620 tablet_service.cc:1511] Processing CreateTablet for tablet 6b24efe048e6499b923f1a0215713fad (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:12.713924  1620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b24efe048e6499b923f1a0215713fad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.714617  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:12.715242  1636 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:12.729378  1693 tablet_service.cc:1511] Processing CreateTablet for tablet 6b24efe048e6499b923f1a0215713fad (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:12.731518  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b24efe048e6499b923f1a0215713fad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.731288  1550 tablet_service.cc:1511] Processing CreateTablet for tablet 18511a70f641409b83d4cde6d4e56009 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:59:12.731492  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 18511a70f641409b83d4cde6d4e56009 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:59:12.732728  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18511a70f641409b83d4cde6d4e56009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.732764  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18511a70f641409b83d4cde6d4e56009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.744976  1547 tablet_service.cc:1511] Processing CreateTablet for tablet 8516bd3d9b8948f0a0a32c2e93d4673e (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:59:12.747787  1696 tablet_service.cc:1511] Processing CreateTablet for tablet 8516bd3d9b8948f0a0a32c2e93d4673e (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:59:12.749539  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8516bd3d9b8948f0a0a32c2e93d4673e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.747850  1547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8516bd3d9b8948f0a0a32c2e93d4673e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.719841  1759 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:12.769326  1543 tablet_service.cc:1511] Processing CreateTablet for tablet 22f9010f94804fcaacae8539ddfc64a7 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:59:12.770900  1543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f9010f94804fcaacae8539ddfc64a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.750020  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 8516bd3d9b8948f0a0a32c2e93d4673e (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:59:12.765918  1759 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:12.783318  1695 tablet_service.cc:1511] Processing CreateTablet for tablet 22f9010f94804fcaacae8539ddfc64a7 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:59:12.784781  1695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f9010f94804fcaacae8539ddfc64a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.781262  1759 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.797780  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8516bd3d9b8948f0a0a32c2e93d4673e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.798604  1759 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:12.798705  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:12.799995  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 12:59:12.800460  1675 leader_election.cc:343] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:12.801800  1674 leader_election.cc:343] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.802304  1674 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:12.796432  1459 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:12.790498  1755 tablet_bootstrap.cc:654] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:12.806317  1622 tablet_service.cc:1511] Processing CreateTablet for tablet 18511a70f641409b83d4cde6d4e56009 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:59:12.807946  1622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18511a70f641409b83d4cde6d4e56009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.811882  1754 tablet_bootstrap.cc:492] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:12.812534  1754 ts_tablet_manager.cc:1403] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.284s	user 0.017s	sys 0.005s
I20260709 12:59:12.816808  1759 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:12.818477  1780 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 12:59:12.818572  1623 tablet_service.cc:1511] Processing CreateTablet for tablet 22f9010f94804fcaacae8539ddfc64a7 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:59:12.819818  1753 tablet_bootstrap.cc:492] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:12.820117  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f9010f94804fcaacae8539ddfc64a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.820822  1621 tablet_service.cc:1511] Processing CreateTablet for tablet 4e2094a808984c84823c1e0dad0a5de0 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:12.822201  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:12.822604  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:12.822830  1561 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:12.825410  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 4e2094a808984c84823c1e0dad0a5de0 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:12.827468  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2094a808984c84823c1e0dad0a5de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.824498  1753 ts_tablet_manager.cc:1403] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.310s	user 0.041s	sys 0.045s
W20260709 12:59:12.827843  1674 leader_election.cc:343] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:12.816982  1754 raft_consensus.cc:359] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.847191  1544 tablet_service.cc:1511] Processing CreateTablet for tablet 472ed50e0865403c81123d4efe7108d7 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:12.848994  1544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 472ed50e0865403c81123d4efe7108d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.850097  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 472ed50e0865403c81123d4efe7108d7 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:12.854529  1753 raft_consensus.cc:359] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.854760  1551 tablet_service.cc:1511] Processing CreateTablet for tablet 13debc393e8a4121883dda403cb4badf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:12.829660  1780 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:12.855450  1753 raft_consensus.cc:385] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:12.856154  1753 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:12.858052  1551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13debc393e8a4121883dda403cb4badf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.857570  1697 tablet_service.cc:1511] Processing CreateTablet for tablet 472ed50e0865403c81123d4efe7108d7 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:12.859089  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 472ed50e0865403c81123d4efe7108d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.857165  1691 tablet_service.cc:1511] Processing CreateTablet for tablet 4e2094a808984c84823c1e0dad0a5de0 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:12.860754  1691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2094a808984c84823c1e0dad0a5de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.859104  1753 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.864668  1753 ts_tablet_manager.cc:1434] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 12:59:12.865583  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2094a808984c84823c1e0dad0a5de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.868775  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 472ed50e0865403c81123d4efe7108d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.812463  1636 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:12.875205  1561 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:12.878155  1674 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:12.889573  1759 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:12.841542  1754 raft_consensus.cc:385] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:12.890180  1759 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:12.890523  1754 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:12.891578  1759 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.893445  1754 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.897186  1755 tablet_bootstrap.cc:492] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:12.897847  1755 ts_tablet_manager.cc:1403] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.278s	user 0.010s	sys 0.015s
I20260709 12:59:12.898995  1693 tablet_service.cc:1511] Processing CreateTablet for tablet 13debc393e8a4121883dda403cb4badf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:12.900604  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13debc393e8a4121883dda403cb4badf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.902673  1753 tablet_bootstrap.cc:492] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:12.902776  1626 tablet_service.cc:1511] Processing CreateTablet for tablet 13debc393e8a4121883dda403cb4badf (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:12.904448  1626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13debc393e8a4121883dda403cb4badf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.910099  1617 tablet_service.cc:1511] Processing CreateTablet for tablet 939d7b92438a4b20bd5f2e81d2291c03 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:12.911533  1754 ts_tablet_manager.cc:1434] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.099s	user 0.010s	sys 0.000s
I20260709 12:59:12.912736  1754 tablet_bootstrap.cc:492] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:12.914624  1550 tablet_service.cc:1511] Processing CreateTablet for tablet 939d7b92438a4b20bd5f2e81d2291c03 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:12.916083  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 939d7b92438a4b20bd5f2e81d2291c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.916657  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 939d7b92438a4b20bd5f2e81d2291c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.931170  1694 tablet_service.cc:1511] Processing CreateTablet for tablet 939d7b92438a4b20bd5f2e81d2291c03 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:12.933184  1694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 939d7b92438a4b20bd5f2e81d2291c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.900943  1755 raft_consensus.cc:359] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.938531  1755 raft_consensus.cc:385] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:12.938948  1755 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:12.939994  1755 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:12.942826  1755 ts_tablet_manager.cc:1434] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.003s
I20260709 12:59:12.945708  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:12.951406  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 637e660a66be4924866287fa6150aaea (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:12.952842  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 637e660a66be4924866287fa6150aaea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.952556  1693 tablet_service.cc:1511] Processing CreateTablet for tablet 637e660a66be4924866287fa6150aaea (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:12.953776  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 637e660a66be4924866287fa6150aaea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.945784  1755 tablet_bootstrap.cc:492] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:12.961169  1759 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:12.962311  1543 tablet_service.cc:1511] Processing CreateTablet for tablet 5ba40a6e1d714431be77e8a1518ec8d6 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:12.963016  1759 raft_consensus.cc:697] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:12.963816  1543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ba40a6e1d714431be77e8a1518ec8d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.964015  1759 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 12:59:12.971174  1755 tablet_bootstrap.cc:654] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:12.979125  1546 tablet_service.cc:1511] Processing CreateTablet for tablet 637e660a66be4924866287fa6150aaea (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:12.980469  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 637e660a66be4924866287fa6150aaea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.983476  1620 tablet_service.cc:1511] Processing CreateTablet for tablet 5ba40a6e1d714431be77e8a1518ec8d6 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:12.985122  1620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ba40a6e1d714431be77e8a1518ec8d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:12.976143  1457 catalog_manager.cc:5697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:12.995784  1625 tablet_service.cc:1511] Processing CreateTablet for tablet ce507f8325ba41dda6865b9492bfd0a7 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:12.997304  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce507f8325ba41dda6865b9492bfd0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.001600  1547 tablet_service.cc:1511] Processing CreateTablet for tablet ce507f8325ba41dda6865b9492bfd0a7 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:13.003239  1547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce507f8325ba41dda6865b9492bfd0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.008098  1753 tablet_bootstrap.cc:654] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.011932  1551 tablet_service.cc:1511] Processing CreateTablet for tablet ca209b0d98c64e1da48e3027b3ea1d9a (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:13.013461  1551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca209b0d98c64e1da48e3027b3ea1d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.012877  1623 tablet_service.cc:1511] Processing CreateTablet for tablet ca209b0d98c64e1da48e3027b3ea1d9a (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:13.015025  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca209b0d98c64e1da48e3027b3ea1d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.022451  1693 tablet_service.cc:1511] Processing CreateTablet for tablet ce507f8325ba41dda6865b9492bfd0a7 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:13.023813  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce507f8325ba41dda6865b9492bfd0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.022454  1695 tablet_service.cc:1511] Processing CreateTablet for tablet 5ba40a6e1d714431be77e8a1518ec8d6 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:13.025200  1754 tablet_bootstrap.cc:654] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.025449  1695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ba40a6e1d714431be77e8a1518ec8d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.030668  1699 tablet_service.cc:1511] Processing CreateTablet for tablet ca209b0d98c64e1da48e3027b3ea1d9a (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:13.041575  1546 tablet_service.cc:1511] Processing CreateTablet for tablet ac931d9355284a168be64d7debd009ae (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:13.055469  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac931d9355284a168be64d7debd009ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.087416  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca209b0d98c64e1da48e3027b3ea1d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.094517  1697 tablet_service.cc:1511] Processing CreateTablet for tablet ac931d9355284a168be64d7debd009ae (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:13.095359  1548 tablet_service.cc:1511] Processing CreateTablet for tablet 61e7345bb41047a2896de16e89540bbb (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:13.096392  1696 tablet_service.cc:1511] Processing CreateTablet for tablet 77e952b7f0f34618a5ce435fa48e0448 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:13.096925  1548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61e7345bb41047a2896de16e89540bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.101341  1546 tablet_service.cc:1511] Processing CreateTablet for tablet d72a925fb22046e2be514a7913c613a4 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:59:13.103600  1546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d72a925fb22046e2be514a7913c613a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.105449  1544 tablet_service.cc:1511] Processing CreateTablet for tablet 77e952b7f0f34618a5ce435fa48e0448 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:13.106257  1545 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3a7d726b2f43c28355175906793af9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:13.106988  1544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77e952b7f0f34618a5ce435fa48e0448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.112098  1549 tablet_service.cc:1511] Processing CreateTablet for tablet 8cb801d86c874e1686eceacb9de1d0a6 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:13.113757  1549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cb801d86c874e1686eceacb9de1d0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.113359  1700 tablet_service.cc:1511] Processing CreateTablet for tablet 61e7345bb41047a2896de16e89540bbb (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:13.114758  1692 tablet_service.cc:1511] Processing CreateTablet for tablet d72a925fb22046e2be514a7913c613a4 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:59:13.112098  1698 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3a7d726b2f43c28355175906793af9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:13.121723  1618 tablet_service.cc:1511] Processing CreateTablet for tablet 8cb801d86c874e1686eceacb9de1d0a6 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:13.123067  1618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cb801d86c874e1686eceacb9de1d0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.126355  1622 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3a7d726b2f43c28355175906793af9 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:13.128273  1697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac931d9355284a168be64d7debd009ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.131608  1621 tablet_service.cc:1511] Processing CreateTablet for tablet ac931d9355284a168be64d7debd009ae (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:13.131731  1542 tablet_service.cc:1511] Processing CreateTablet for tablet 25feb0fae8094dd29fea235dd1af7a14 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:13.132987  1700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61e7345bb41047a2896de16e89540bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.114791  1545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3a7d726b2f43c28355175906793af9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:13.135604  1526 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:13.136897  1755 tablet_bootstrap.cc:492] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:13.137094  1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25feb0fae8094dd29fea235dd1af7a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.138713  1693 tablet_service.cc:1511] Processing CreateTablet for tablet 8cb801d86c874e1686eceacb9de1d0a6 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:13.137434  1755 ts_tablet_manager.cc:1403] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.192s	user 0.026s	sys 0.009s
I20260709 12:59:13.146428  1622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3a7d726b2f43c28355175906793af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.146728  1692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d72a925fb22046e2be514a7913c613a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.147382  1698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3a7d726b2f43c28355175906793af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.149823  1696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77e952b7f0f34618a5ce435fa48e0448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.151582  1693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cb801d86c874e1686eceacb9de1d0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.164909  1623 tablet_service.cc:1511] Processing CreateTablet for tablet d72a925fb22046e2be514a7913c613a4 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:59:13.170053  1617 tablet_service.cc:1511] Processing CreateTablet for tablet 25feb0fae8094dd29fea235dd1af7a14 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:13.170554  1620 tablet_service.cc:1511] Processing CreateTablet for tablet 61e7345bb41047a2896de16e89540bbb (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:13.172456  1759 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.173112  1759 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.176935  1759 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:13.177672  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:13.179255  1636 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:13.180667  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:13.182593  1626 tablet_service.cc:1511] Processing CreateTablet for tablet 77e952b7f0f34618a5ce435fa48e0448 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:13.145056  1755 raft_consensus.cc:359] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.184356  1755 raft_consensus.cc:385] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.182950  1674 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:13.184718  1755 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
W20260709 12:59:13.185148  1675 leader_election.cc:343] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.175740  1621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac931d9355284a168be64d7debd009ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.186216  1759 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:13.186851  1759 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:13.185871  1755 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.187597  1759 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.189631  1753 tablet_bootstrap.cc:492] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:13.192883  1617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25feb0fae8094dd29fea235dd1af7a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.198616  1753 ts_tablet_manager.cc:1403] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.296s	user 0.018s	sys 0.002s
I20260709 12:59:13.198989  1626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77e952b7f0f34618a5ce435fa48e0448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.206408  1753 raft_consensus.cc:359] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.210180  1623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d72a925fb22046e2be514a7913c613a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.214569  1620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61e7345bb41047a2896de16e89540bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.220748  1694 tablet_service.cc:1511] Processing CreateTablet for tablet 25feb0fae8094dd29fea235dd1af7a14 (DEFAULT_TABLE table=test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:13.221517  1781 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:13.222973  1694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25feb0fae8094dd29fea235dd1af7a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:13.211434  1755 ts_tablet_manager.cc:1434] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.069s	user 0.005s	sys 0.003s
I20260709 12:59:13.211586  1763 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.226791  1753 raft_consensus.cc:385] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.232788  1763 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.240072  1755 tablet_bootstrap.cc:492] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:13.257295  1763 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.258522  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:13.259217  1710 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:13.233147  1759 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.260624  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:13.243427  1753 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.263589  1763 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:13.263973  1763 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:13.264284  1763 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.266160  1759 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:13.269618  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 12:59:13.277854  1675 leader_election.cc:343] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.279368  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:13.280094  1634 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.282598  1754 tablet_bootstrap.cc:492] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:13.282917  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:13.283634  1635 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:13.283172  1754 ts_tablet_manager.cc:1403] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.371s	user 0.020s	sys 0.025s
I20260709 12:59:13.284694  1755 tablet_bootstrap.cc:654] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.284859  1753 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.287577  1754 raft_consensus.cc:359] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.288512  1754 raft_consensus.cc:385] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.288817  1754 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.289587  1754 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.292294  1754 ts_tablet_manager.cc:1434] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:59:13.294277  1754 tablet_bootstrap.cc:492] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:13.308607  1754 tablet_bootstrap.cc:654] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.288168  1753 ts_tablet_manager.cc:1434] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.089s	user 0.007s	sys 0.000s
I20260709 12:59:13.317710  1781 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.318254  1781 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.320438  1781 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.320555  1763 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.327183  1753 tablet_bootstrap.cc:492] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:13.331784  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:13.332561  1710 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:13.342137  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:13.345672  1781 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:13.346114  1781 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:13.346429  1753 tablet_bootstrap.cc:654] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.346493  1781 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.341814  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 12:59:13.352672  1525 leader_election.cc:343] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.354458  1634 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
W20260709 12:59:13.356494  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:13.357539  1674 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:13.359275  1759 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:13.359902  1759 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:13.361011  1759 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.365546  1763 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.366243  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:13.366669  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:13.376355  1710 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.371769  1769 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.381275  1769 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.384366  1769 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.385954  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 12:59:13.388062  1601 leader_election.cc:343] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.390228  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:13.392315  1601 leader_election.cc:343] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.392920  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:13.394557  1769 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:13.376356  1634 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.405911  1710 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
W20260709 12:59:13.406131  1444 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 62225.9us
I20260709 12:59:13.408903  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:13.423328  1755 tablet_bootstrap.cc:492] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:13.424085  1755 ts_tablet_manager.cc:1403] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.184s	user 0.026s	sys 0.022s
I20260709 12:59:13.423362  1457 catalog_manager.cc:5697] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:13.426249  1754 tablet_bootstrap.cc:492] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:13.425724  1457 catalog_manager.cc:5697] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:13.426759  1754 ts_tablet_manager.cc:1403] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.133s	user 0.038s	sys 0.027s
I20260709 12:59:13.428719  1755 raft_consensus.cc:359] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.429774  1755 raft_consensus.cc:385] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.430161  1755 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.430773  1753 tablet_bootstrap.cc:492] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:13.432262  1753 ts_tablet_manager.cc:1403] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.116s	user 0.023s	sys 0.001s
I20260709 12:59:13.436223  1753 raft_consensus.cc:359] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.436961  1753 raft_consensus.cc:385] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.431131  1755 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.431483  1763 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:13.438820  1763 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:13.440019  1755 ts_tablet_manager.cc:1434] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.003s
I20260709 12:59:13.431164  1754 raft_consensus.cc:359] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.439741  1763 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.441352  1769 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.441867  1769 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.440446  1754 raft_consensus.cc:385] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.443161  1754 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.446444  1754 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.447415  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:13.448607  1769 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 12:59:13.449183  1601 leader_election.cc:343] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.449116  1759 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:13.450729  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:13.452457  1755 tablet_bootstrap.cc:492] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:13.437856  1781 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.455335  1754 ts_tablet_manager.cc:1434] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.004s
I20260709 12:59:13.456733  1754 tablet_bootstrap.cc:492] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
W20260709 12:59:13.457096  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 48348us
I20260709 12:59:13.458719  1759 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:13.460693  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:13.462086  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 12:59:13.462370  1525 leader_election.cc:343] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.462718  1709 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.467299  1754 tablet_bootstrap.cc:654] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.476287  1456 catalog_manager.cc:5697] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:13.480121  1709 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:13.477198  1781 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.483687  1634 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:13.486317  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:13.487759  1781 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:13.488459  1781 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:13.489115  1754 tablet_bootstrap.cc:492] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:13.489647  1754 ts_tablet_manager.cc:1403] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.014s
I20260709 12:59:13.489679  1781 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.496770  1754 raft_consensus.cc:359] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.497452  1754 raft_consensus.cc:385] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.497781  1754 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.499047  1754 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.499588  1755 tablet_bootstrap.cc:654] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.501031  1754 ts_tablet_manager.cc:1434] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 12:59:13.501945  1754 tablet_bootstrap.cc:492] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
W20260709 12:59:13.506201  1674 consensus_peers.cc:597] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:13.437279  1753 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.514737  1456 catalog_manager.cc:5697] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:13.514825  1753 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.516567  1710 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:13.516939  1754 tablet_bootstrap.cc:654] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.520766  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:13.522141  1769 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:13.522532  1769 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:13.522868  1769 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.528024  1781 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.528735  1781 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.533731  1781 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.534591  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:13.535241  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:13.535316  1710 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
W20260709 12:59:13.537283  1525 leader_election.cc:343] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.538740  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:13.539695  1781 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:13.540270  1781 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:13.540799  1781 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.545364  1769 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.548944  1769 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.550402  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:13.550244  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:13.551189  1710 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:13.557574  1601 leader_election.cc:343] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.556180  1781 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.559885  1754 tablet_bootstrap.cc:492] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:13.560416  1754 ts_tablet_manager.cc:1403] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.036s
I20260709 12:59:13.563478  1754 raft_consensus.cc:359] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.564426  1754 raft_consensus.cc:385] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.564687  1754 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.565399  1754 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.567605  1754 ts_tablet_manager.cc:1434] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:13.570386  1754 tablet_bootstrap.cc:492] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:13.577042  1754 tablet_bootstrap.cc:654] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.582151  1801 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.583313  1753 ts_tablet_manager.cc:1434] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.151s	user 0.006s	sys 0.005s
I20260709 12:59:13.582919  1801 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.584908  1753 tablet_bootstrap.cc:492] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:13.587865  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:13.593935  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:13.594679  1801 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.595903  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:13.598284  1781 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.598719  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 12:59:13.601864  1525 leader_election.cc:343] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.602398  1754 tablet_bootstrap.cc:492] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:13.598915  1781 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:13.602916  1754 ts_tablet_manager.cc:1403] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.016s
I20260709 12:59:13.593281  1710 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:13.597813  1755 tablet_bootstrap.cc:492] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:13.600308  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:13.605923  1754 raft_consensus.cc:359] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.606294  1709 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.606647  1754 raft_consensus.cc:385] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.607069  1754 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.607697  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:13.607762  1754 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.606681  1755 ts_tablet_manager.cc:1403] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.154s	user 0.025s	sys 0.048s
I20260709 12:59:13.609355  1769 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:13.605722  1753 tablet_bootstrap.cc:654] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.612502  1755 raft_consensus.cc:359] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.613330  1755 raft_consensus.cc:385] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.613731  1755 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.614636  1801 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.614730  1755 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.615334  1801 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.617836  1754 ts_tablet_manager.cc:1434] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:59:13.619356  1754 tablet_bootstrap.cc:492] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:13.619405  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:13.628392  1769 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:13.629556  1769 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.637650  1754 tablet_bootstrap.cc:654] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.627040  1709 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:13.634449  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:13.640193  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:13.641837  1710 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 1: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 12:59:13.642545  1781 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:13.643036  1781 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:13.643889  1781 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:13.632537  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:13.633195  1801 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.655548  1753 tablet_bootstrap.cc:492] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:13.656095  1753 ts_tablet_manager.cc:1403] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.072s	user 0.014s	sys 0.032s
I20260709 12:59:13.659366  1755 ts_tablet_manager.cc:1434] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.050s	user 0.021s	sys 0.002s
I20260709 12:59:13.660893  1753 raft_consensus.cc:359] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.662227  1525 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:13.662323  1459 catalog_manager.cc:5697] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:13.666009  1753 raft_consensus.cc:385] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.666503  1753 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.667457  1753 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.670921  1755 tablet_bootstrap.cc:492] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:13.671705  1457 catalog_manager.cc:5697] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:13.677258  1781 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.683612  1754 tablet_bootstrap.cc:492] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:13.684211  1754 ts_tablet_manager.cc:1403] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.065s	user 0.023s	sys 0.013s
I20260709 12:59:13.679466  1753 ts_tablet_manager.cc:1434] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.001s
I20260709 12:59:13.688241  1753 tablet_bootstrap.cc:492] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:13.688055  1754 raft_consensus.cc:359] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.688612  1755 tablet_bootstrap.cc:654] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.688848  1754 raft_consensus.cc:385] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.689128  1754 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.690011  1754 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.692704  1754 ts_tablet_manager.cc:1434] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:59:13.693939  1754 tablet_bootstrap.cc:492] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:13.700924  1781 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:13.702359  1754 tablet_bootstrap.cc:654] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.703064  1753 tablet_bootstrap.cc:654] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.706067  1755 tablet_bootstrap.cc:492] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:13.706643  1755 ts_tablet_manager.cc:1403] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.012s
I20260709 12:59:13.709853  1755 raft_consensus.cc:359] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.710672  1755 raft_consensus.cc:385] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.710995  1755 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.711848  1755 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.714337  1755 ts_tablet_manager.cc:1434] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:13.715385  1755 tablet_bootstrap.cc:492] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:13.722296  1755 tablet_bootstrap.cc:654] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.734711  1755 tablet_bootstrap.cc:492] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:13.735335  1755 ts_tablet_manager.cc:1403] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.003s
I20260709 12:59:13.738312  1755 raft_consensus.cc:359] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.739068  1755 raft_consensus.cc:385] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.739334  1755 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.740049  1755 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.740272  1754 tablet_bootstrap.cc:492] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:13.740610  1753 tablet_bootstrap.cc:492] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:13.740898  1754 ts_tablet_manager.cc:1403] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.047s	user 0.021s	sys 0.018s
I20260709 12:59:13.741014  1753 ts_tablet_manager.cc:1403] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.053s	user 0.028s	sys 0.015s
I20260709 12:59:13.742102  1755 ts_tablet_manager.cc:1434] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:13.743373  1755 tablet_bootstrap.cc:492] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:13.744592  1753 raft_consensus.cc:359] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.744903  1754 raft_consensus.cc:359] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.745285  1753 raft_consensus.cc:385] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.745558  1754 raft_consensus.cc:385] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.745573  1753 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.745910  1754 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.746722  1753 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.746678  1754 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.748999  1754 ts_tablet_manager.cc:1434] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:59:13.750618  1754 tablet_bootstrap.cc:492] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:13.752696  1755 tablet_bootstrap.cc:654] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.757016  1753 ts_tablet_manager.cc:1434] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:59:13.758214  1753 tablet_bootstrap.cc:492] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:13.759241  1754 tablet_bootstrap.cc:654] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.773590  1754 tablet_bootstrap.cc:492] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:13.774292  1754 ts_tablet_manager.cc:1403] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.011s
I20260709 12:59:13.777661  1754 raft_consensus.cc:359] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.778650  1754 raft_consensus.cc:385] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.778837  1755 tablet_bootstrap.cc:492] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:13.779240  1755 ts_tablet_manager.cc:1403] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.013s
I20260709 12:59:13.779100  1754 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.780938  1754 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.782815  1755 raft_consensus.cc:359] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 12:59:13.783418  1753 tablet_bootstrap.cc:654] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.783519  1754 ts_tablet_manager.cc:1434] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:13.783542  1755 raft_consensus.cc:385] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.784225  1755 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.785111  1754 tablet_bootstrap.cc:492] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:13.785130  1755 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 12:59:13.793676  1754 tablet_bootstrap.cc:654] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.794456  1755 ts_tablet_manager.cc:1434] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.006s
I20260709 12:59:13.795895  1755 tablet_bootstrap.cc:492] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:13.797245  1781 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.797852  1781 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.800349  1781 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.801806  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:13.802289  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:13.802987  1635 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
W20260709 12:59:13.803584  1526 leader_election.cc:343] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.805051  1755 tablet_bootstrap.cc:654] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.805364  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:13.806767  1781 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:13.807282  1781 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:13.807724  1781 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.816216  1769 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.816483  1753 tablet_bootstrap.cc:492] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:13.816782  1769 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.817198  1753 ts_tablet_manager.cc:1403] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.028s
I20260709 12:59:13.819025  1769 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.818674  1781 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.821616  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:13.821457  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:13.822326  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:13.822515  1561 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:13.823103  1635 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:13.823312  1601 leader_election.cc:343] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.824216  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:13.825753  1814 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.825464  1753 raft_consensus.cc:359] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.826447  1753 raft_consensus.cc:385] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.826249  1814 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.826886  1753 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.827921  1753 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.828951  1814 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:13.831851  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:13.832813  1634 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:13.833194  1753 ts_tablet_manager.cc:1434] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.003s
I20260709 12:59:13.831862  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:13.834270  1561 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:13.834715  1769 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:13.834815  1753 tablet_bootstrap.cc:492] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:13.835666  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:13.836570  1814 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:13.836931  1814 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:13.837280  1814 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.837988  1755 tablet_bootstrap.cc:492] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:13.838524  1755 ts_tablet_manager.cc:1403] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.043s	user 0.024s	sys 0.011s
I20260709 12:59:13.838864  1769 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:13.839335  1769 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.841217  1811 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:13.842563  1811 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.842727  1755 raft_consensus.cc:359] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.843590  1755 raft_consensus.cc:385] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.843974  1753 tablet_bootstrap.cc:654] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.845153  1811 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:13.843911  1755 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.847103  1755 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.850400  1635 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:13.852063  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:13.852784  1634 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:13.854027  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:13.854563  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:13.857084  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 12:59:13.859472  1526 leader_election.cc:343] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.860428  1811 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:13.860793  1811 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:13.861147  1811 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.862923  1769 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.863524  1755 ts_tablet_manager.cc:1434] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.001s
I20260709 12:59:13.864769  1781 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.854717  1561 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:13.867004  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:13.869516  1755 tablet_bootstrap.cc:492] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:13.868808  1781 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:13.870718  1781 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:13.871608  1781 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.872400  1561 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.878845  1769 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:13.882900  1801 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:13.880882  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:13.886401  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 12:59:13.893150  1601 leader_election.cc:343] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:13.890415  1560 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.914229  1559 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.957234  1814 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.967721  1753 tablet_bootstrap.cc:492] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:13.968421  1753 ts_tablet_manager.cc:1403] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.134s	user 0.014s	sys 0.010s
I20260709 12:59:13.970516  1754 tablet_bootstrap.cc:492] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:13.971114  1754 ts_tablet_manager.cc:1403] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.186s	user 0.026s	sys 0.051s
I20260709 12:59:13.974402  1754 raft_consensus.cc:359] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.975191  1754 raft_consensus.cc:385] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:13.975488  1754 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:13.978276  1457 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:13.977885  1811 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.981410  1811 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:13.982617  1755 tablet_bootstrap.cc:654] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:13.983074  1710 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.976305  1754 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:13.988380  1754 ts_tablet_manager.cc:1434] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:59:13.988915  1811 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:13.989570  1754 tablet_bootstrap.cc:492] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:13.991104  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:13.991789  1561 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.998360  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:13.999390  1636 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:13.999732  1560 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:13.972690  1753 raft_consensus.cc:359] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.002286  1753 raft_consensus.cc:385] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.003851  1753 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.004279  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:14.004973  1557 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:14.004894  1753 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.008010  1561 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:14.014204  1753 ts_tablet_manager.cc:1434] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.003s
I20260709 12:59:14.220381  1753 tablet_bootstrap.cc:492] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:14.221299  1754 tablet_bootstrap.cc:654] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.009233  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:14.042052  1780 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:14.223802  1780 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:14.225368  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 218630us
I20260709 12:59:14.227599  1755 tablet_bootstrap.cc:492] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:14.228140  1755 ts_tablet_manager.cc:1403] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.359s	user 0.035s	sys 0.019s
I20260709 12:59:14.231429  1755 raft_consensus.cc:359] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.232065  1755 raft_consensus.cc:385] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.232286  1755 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.232918  1755 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.009517  1759 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.230s
I20260709 12:59:14.224079  1818 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:14.106066  1769 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:14.066969  1763 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:14.197001  1525 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:14.178339  1801 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:14.222585  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 182020us
I20260709 12:59:14.251279  1814 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 12:59:14.263041  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66223.7us
I20260709 12:59:14.295372  1755 ts_tablet_manager.cc:1434] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.067s	user 0.012s	sys 0.012s
I20260709 12:59:14.262823  1769 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 12:59:14.274871  1557 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:14.264945  1763 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.301431  1755 tablet_bootstrap.cc:492] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:14.302146  1818 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:14.266012  1801 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.334782  1753 tablet_bootstrap.cc:654] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.296027  1781 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 12:59:14.356078  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:14.377367  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:14.378050  1633 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:14.378656  1801 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:14.378796  1763 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:14.395743  1780 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:14.396426  1636 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:14.396628  1818 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.330881  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:14.435853  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:14.424803  1780 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:14.439270  1780 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:14.440503  1780 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:14.416894  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 97268.2us
I20260709 12:59:14.457064  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 12:59:14.458297  1836 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:59:14.458019  1636 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:14.461318  1759 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:14.468439  1754 tablet_bootstrap.cc:492] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:14.461998  1847 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:59:14.461974  1759 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 12:59:14.462708  1674 leader_election.cc:343] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:14.479775  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:14.511368  1769 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:14.474471  1759 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.532666  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:14.523820  1675 leader_election.cc:343] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:14.571872  1675 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:14.553220  1769 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:59:14.574460  1754 ts_tablet_manager.cc:1403] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.585s	user 0.038s	sys 0.038s
I20260709 12:59:14.548070  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:14.581609  1837 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:14.577201  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 80243.6us
W20260709 12:59:14.579227  1526 leader_election.cc:343] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:14.596292  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:14.597088  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:14.598104  1635 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:14.599238  1754 raft_consensus.cc:359] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.599967  1754 raft_consensus.cc:385] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.600217  1754 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.600751  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:14.606503  1754 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.610323  1754 ts_tablet_manager.cc:1434] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 12:59:14.615841  1754 tablet_bootstrap.cc:492] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:14.618089  1633 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:14.623909  1754 tablet_bootstrap.cc:654] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.628455  1850 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:59:14.638438  1634 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:14.632691  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:14.641619  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:14.646725  1801 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:14.647145  1801 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:14.647477  1801 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:14.647632  1847 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:14.652995  1841 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:14.653659  1841 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.656064  1841 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:14.656272  1853 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 12:59:14.665823  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:59:14.658322  1525 leader_election.cc:343] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:14.633442  1850 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
W20260709 12:59:14.650321  1526 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:14.688963  1753 tablet_bootstrap.cc:492] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:14.689702  1753 ts_tablet_manager.cc:1403] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.470s	user 0.018s	sys 0.018s
W20260709 12:59:14.686633  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 46250.5us
W20260709 12:59:14.690443  1526 leader_election.cc:343] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:14.690896  1526 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:14.693823  1755 tablet_bootstrap.cc:654] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.701920  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:14.698132  1853 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:14.714939  1525 leader_election.cc:343] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:14.715763  1853 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:14.719094  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:14.722093  1526 leader_election.cc:343] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:14.722600  1526 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:14.723608  1457 catalog_manager.cc:5697] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:14.725091  1818 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:14.731683  1852 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:14.736819  1818 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.746660  1818 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:14.748440  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:14.749948  1753 raft_consensus.cc:359] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.750497  1753 raft_consensus.cc:385] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.750733  1753 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.748775  1457 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:14.760910  1753 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.761965  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:14.764434  1753 ts_tablet_manager.cc:1434] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.074s	user 0.009s	sys 0.001s
I20260709 12:59:14.765954  1753 tablet_bootstrap.cc:492] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:14.767496  1801 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.781549  1753 tablet_bootstrap.cc:654] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.788892  1754 tablet_bootstrap.cc:492] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:14.790341  1754 ts_tablet_manager.cc:1403] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.175s	user 0.016s	sys 0.029s
I20260709 12:59:14.792407  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:14.795986  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:14.796180  1754 raft_consensus.cc:359] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.797022  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:14.798449  1635 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:14.798956  1818 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:14.799430  1801 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:14.797516  1754 raft_consensus.cc:385] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:14.797694  1526 leader_election.cc:343] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:14.806227  1755 tablet_bootstrap.cc:492] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:14.806950  1755 ts_tablet_manager.cc:1403] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.506s	user 0.035s	sys 0.044s
I20260709 12:59:14.807356  1754 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.811126  1635 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:14.817047  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:14.812091  1754 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.818500  1801 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:14.819192  1801 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:14.810807  1755 raft_consensus.cc:359] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.820119  1755 raft_consensus.cc:385] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.820434  1755 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.820238  1801 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.822395  1754 ts_tablet_manager.cc:1434] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.001s
I20260709 12:59:14.822266  1755 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.825204  1754 tablet_bootstrap.cc:492] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:14.825903  1753 tablet_bootstrap.cc:492] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:14.826607  1753 ts_tablet_manager.cc:1403] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.061s	user 0.021s	sys 0.005s
I20260709 12:59:14.827843  1755 ts_tablet_manager.cc:1434] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:59:14.828975  1755 tablet_bootstrap.cc:492] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:14.830754  1753 raft_consensus.cc:359] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.831506  1753 raft_consensus.cc:385] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.831830  1753 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.832716  1753 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.838668  1755 tablet_bootstrap.cc:654] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.838869  1753 ts_tablet_manager.cc:1434] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 12:59:14.840849  1753 tablet_bootstrap.cc:492] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:14.843147  1754 tablet_bootstrap.cc:654] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.847847  1456 catalog_manager.cc:5697] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:14.849267  1753 tablet_bootstrap.cc:654] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.853689  1837 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:14.854236  1837 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.856731  1837 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:14.858048  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:14.861761  1707 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:14.862531  1754 tablet_bootstrap.cc:492] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:14.863610  1754 ts_tablet_manager.cc:1403] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.007s
I20260709 12:59:14.868220  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:14.869108  1754 raft_consensus.cc:359] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.869831  1754 raft_consensus.cc:385] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.870079  1675 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:14.872395  1753 tablet_bootstrap.cc:492] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:14.872882  1753 ts_tablet_manager.cc:1403] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.001s
I20260709 12:59:14.876530  1753 raft_consensus.cc:359] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.877362  1753 raft_consensus.cc:385] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.877648  1753 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.878444  1753 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.870112  1754 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.881179  1753 ts_tablet_manager.cc:1434] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:59:14.882180  1753 tablet_bootstrap.cc:492] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:14.881584  1754 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.888319  1754 ts_tablet_manager.cc:1434] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:59:14.889518  1754 tablet_bootstrap.cc:492] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:14.890450  1753 tablet_bootstrap.cc:654] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.900194  1837 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:14.902479  1753 tablet_bootstrap.cc:492] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:14.903015  1753 ts_tablet_manager.cc:1403] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.000s
I20260709 12:59:14.905790  1754 tablet_bootstrap.cc:654] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.906649  1753 raft_consensus.cc:359] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.907595  1755 tablet_bootstrap.cc:492] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:14.907773  1753 raft_consensus.cc:385] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.908175  1753 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.908313  1755 ts_tablet_manager.cc:1403] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.080s	user 0.020s	sys 0.049s
I20260709 12:59:14.910163  1753 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.912034  1755 raft_consensus.cc:359] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.912679  1753 ts_tablet_manager.cc:1434] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:59:14.912717  1755 raft_consensus.cc:385] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.913259  1755 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.913810  1753 tablet_bootstrap.cc:492] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:14.914088  1755 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.917220  1755 ts_tablet_manager.cc:1434] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:59:14.918222  1755 tablet_bootstrap.cc:492] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:14.923117  1753 tablet_bootstrap.cc:654] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.925120  1754 tablet_bootstrap.cc:492] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:14.925710  1754 ts_tablet_manager.cc:1403] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.008s
I20260709 12:59:14.929422  1755 tablet_bootstrap.cc:654] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.930899  1707 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:14.931409  1458 catalog_manager.cc:5697] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:14.945797  1818 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:14.950961  1754 raft_consensus.cc:359] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.952646  1754 raft_consensus.cc:385] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.953006  1754 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.953955  1755 tablet_bootstrap.cc:492] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:14.954504  1755 ts_tablet_manager.cc:1403] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.005s
I20260709 12:59:14.954108  1754 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.957257  1754 ts_tablet_manager.cc:1434] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:59:14.957576  1755 raft_consensus.cc:359] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.958357  1755 raft_consensus.cc:385] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.958678  1755 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.959441  1755 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.962554  1755 ts_tablet_manager.cc:1434] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:59:14.963523  1755 tablet_bootstrap.cc:492] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:14.964773  1818 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:14.966149  1818 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.968883  1818 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:14.971628  1755 tablet_bootstrap.cc:654] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:14.974167  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:14.975077  1754 tablet_bootstrap.cc:492] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:14.975018  1561 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:14.976141  1837 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:14.976585  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:14.983927  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:14.984689  1710 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:14.983990  1753 tablet_bootstrap.cc:492] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:14.987584  1753 ts_tablet_manager.cc:1403] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.074s	user 0.008s	sys 0.010s
I20260709 12:59:14.981029  1818 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:14.988205  1818 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:14.989279  1818 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:14.991734  1753 raft_consensus.cc:359] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:14.992770  1753 raft_consensus.cc:385] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:14.993127  1753 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:14.994475  1753 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.001972  1753 ts_tablet_manager.cc:1434] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 12:59:15.002292  1837 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:15.003178  1753 tablet_bootstrap.cc:492] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:15.006804  1843 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:15.014725  1753 tablet_bootstrap.cc:654] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:15.035374  1818 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.036761  1755 tablet_bootstrap.cc:492] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:15.039152  1755 ts_tablet_manager.cc:1403] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.076s	user 0.017s	sys 0.012s
I20260709 12:59:15.043601  1755 raft_consensus.cc:359] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.044842  1852 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.044883  1755 raft_consensus.cc:385] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:15.050407  1755 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:15.045878  1818 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:15.045598  1852 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.059885  1852 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:15.046319  1759 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:59:15.064121  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:15.063735  1755 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.064721  1710 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.057521  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:15.065428  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:15.067162  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:15.072714  1556 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:15.073557  1526 leader_election.cc:343] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.073696  1636 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:15.082425  1525 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:15.083905  1755 ts_tablet_manager.cc:1434] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.002s
I20260709 12:59:15.085196  1801 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:15.086222  1852 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.086496  1801 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:15.086968  1801 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.086899  1852 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.090696  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:15.091092  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:15.090570  1755 tablet_bootstrap.cc:492] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
W20260709 12:59:15.092937  1526 leader_election.cc:343] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:15.093802  1525 leader_election.cc:343] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.097190  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:15.098272  1850 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:15.106252  1838 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:15.094998  1852 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:15.121598  1838 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.126495  1754 tablet_bootstrap.cc:654] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:15.122225  1838 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.133133  1753 tablet_bootstrap.cc:492] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:15.133795  1753 ts_tablet_manager.cc:1403] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.131s	user 0.023s	sys 0.023s
I20260709 12:59:15.137049  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:15.137077  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:15.140291  1635 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 1: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 12:59:15.142371  1759 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.143039  1759 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.144304  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:15.144248  1753 raft_consensus.cc:359] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.145300  1837 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.145471  1753 raft_consensus.cc:385] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:15.146081  1753 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:15.146890  1753 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.145339  1759 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:15.151800  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:15.153236  1635 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:15.154744  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:15.155429  1561 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:15.157155  1753 ts_tablet_manager.cc:1434] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:59:15.134742  1838 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:15.158267  1753 tablet_bootstrap.cc:492] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:15.154991  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:15.162395  1780 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:15.162791  1780 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:15.163158  1780 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.163718  1801 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.177718  1556 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:15.180418  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:15.181576  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:15.182839  1818 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:59:15.183275  1526 leader_election.cc:343] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.183415  1818 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:15.184350  1818 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.185624  1837 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:15.188709  1801 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:15.192559  1755 tablet_bootstrap.cc:654] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:15.181630  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:15.196174  1635 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.198225  1838 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.199034  1838 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.210460  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:15.211231  1561 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:15.212612  1838 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:15.212877  1675 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:15.214253  1837 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:15.214641  1837 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:15.214962  1837 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.216193  1753 tablet_bootstrap.cc:654] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:15.214129  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:15.219419  1780 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:15.220890  1674 leader_election.cc:343] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.224655  1710 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:15.228856  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:15.229846  1634 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.230657  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:15.234824  1556 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.230329  1780 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:15.242717  1837 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.252552  1635 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:15.255882  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:15.257302  1561 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.253211  1556 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:15.264753  1837 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:15.265102  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 12:59:15.274387  1674 leader_election.cc:343] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.277987  1754 tablet_bootstrap.cc:492] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:15.278704  1754 ts_tablet_manager.cc:1403] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.316s	user 0.026s	sys 0.019s
I20260709 12:59:15.285257  1754 raft_consensus.cc:359] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.288268  1754 raft_consensus.cc:385] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:15.288606  1754 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:15.292902  1818 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.302202  1818 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.297962  1525 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:15.307664  1634 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:15.305611  1675 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:15.310494  1837 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:15.311056  1837 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:15.311592  1801 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:15.312163  1801 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:15.312024  1837 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.313119  1801 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.318899  1852 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:15.365185  1754 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.368053  1754 ts_tablet_manager.cc:1434] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.089s	user 0.027s	sys 0.021s
I20260709 12:59:15.369086  1754 tablet_bootstrap.cc:492] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:15.369711  1818 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:15.372805  1753 tablet_bootstrap.cc:492] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:15.373363  1753 ts_tablet_manager.cc:1403] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.215s	user 0.038s	sys 0.009s
I20260709 12:59:15.376019  1755 tablet_bootstrap.cc:492] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:15.376525  1755 ts_tablet_manager.cc:1403] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.286s	user 0.018s	sys 0.023s
I20260709 12:59:15.378580  1753 raft_consensus.cc:359] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.379614  1753 raft_consensus.cc:385] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:15.379962  1753 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:15.380889  1753 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.385107  1753 ts_tablet_manager.cc:1434] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:59:15.386663  1753 tablet_bootstrap.cc:492] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:15.388890  1755 raft_consensus.cc:359] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.382233  1754 tablet_bootstrap.cc:654] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:15.438714  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:15.432927  1561 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:15.456173  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:15.470104  1753 tablet_bootstrap.cc:654] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:15.473670  1601 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:15.473369  1457 catalog_manager.cc:5697] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:15.476060  1818 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:15.471364  1675 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:15.490104  1780 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:15.491133  1780 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:15.492811  1780 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.462312  1755 raft_consensus.cc:385] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:15.498023  1755 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:15.500931  1754 tablet_bootstrap.cc:492] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:15.501905  1754 ts_tablet_manager.cc:1403] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.133s	user 0.017s	sys 0.017s
I20260709 12:59:15.504971  1755 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:15.530136  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 241016us
I20260709 12:59:15.539001  1636 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.539983  1837 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.540691  1837 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.542814  1753 tablet_bootstrap.cc:492] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:15.543725  1753 ts_tablet_manager.cc:1403] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.157s	user 0.013s	sys 0.011s
I20260709 12:59:15.547633  1818 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.564042  1801 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:15.548161  1818 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.629086  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:15.630367  1755 ts_tablet_manager.cc:1434] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.254s	user 0.019s	sys 0.022s
I20260709 12:59:15.630230  1556 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:15.632236  1755 tablet_bootstrap.cc:492] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:15.632123  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:15.634302  1847 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:15.634733  1847 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:15.635156  1847 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.631459  1754 raft_consensus.cc:359] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.639686  1754 raft_consensus.cc:385] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:15.640136  1754 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:15.640884  1818 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:15.641662  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:15.645049  1601 leader_election.cc:343] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.663110  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:15.663434  1753 raft_consensus.cc:359] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.664160  1753 raft_consensus.cc:385] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:15.664443  1753 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:15.665170  1753 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.667470  1753 ts_tablet_manager.cc:1434] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.123s	user 0.015s	sys 0.011s
I20260709 12:59:15.668923  1753 tablet_bootstrap.cc:492] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:15.669575  1456 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:15.674997  1837 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:15.676667  1754 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.683452  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:15.684341  1635 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 1: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 12:59:15.687335  1801 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.688035  1801 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.690027  1674 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:15.693568  1847 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.695462  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 12:59:15.697187  1525 leader_election.cc:343] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.697183  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:15.698115  1558 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.700320  1801 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:15.701030  1837 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.701687  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:15.702587  1707 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:15.704003  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:15.705657  1801 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:15.710544  1847 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:15.706701  1801 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:15.716514  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:15.713850  1801 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:15.724715  1601 leader_election.cc:343] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.726881  1908 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:15.731719  1558 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:15.733832  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:15.740765  1908 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:15.742826  1754 ts_tablet_manager.cc:1434] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.240s	user 0.058s	sys 0.061s
I20260709 12:59:15.744496  1754 tablet_bootstrap.cc:492] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:15.752065  1755 tablet_bootstrap.cc:654] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:15.752319  1847 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:15.754108  1753 tablet_bootstrap.cc:654] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:15.780826  1837 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:15.786598  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:15.789227  1754 tablet_bootstrap.cc:654] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:15.803027  1458 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:15.810053  1850 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.810930  1850 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.815104  1850 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:15.816627  1801 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.817857  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:15.819731  1801 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 12:59:15.820503  1526 leader_election.cc:343] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.820748  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 12:59:15.827314  1525 leader_election.cc:343] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.827674  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:15.828730  1710 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:15.834586  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:15.822350  1908 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 12:59:15.846175  1525 leader_election.cc:343] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.852749  1525 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:15.847134  1780 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.853843  1780 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.856400  1780 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:15.859272  1780 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:15.860849  1850 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:15.862015  1755 tablet_bootstrap.cc:492] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:15.862597  1755 ts_tablet_manager.cc:1403] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.231s	user 0.034s	sys 0.005s
I20260709 12:59:15.862546  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 12:59:15.864663  1675 leader_election.cc:343] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.865695  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:15.849975  1908 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:15.867406  1674 leader_election.cc:343] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:15.870201  1755 raft_consensus.cc:359] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.870841  1755 raft_consensus.cc:385] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:15.871099  1755 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:15.867894  1674 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 12:59:15.882691  1912 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.115s	user 0.007s	sys 0.000s
I20260709 12:59:15.880434  1780 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:15.932551  1674 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:15.938349  1850 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:15.948410  1710 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:15.952929  1755 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.954182  1850 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:15.957454  1753 tablet_bootstrap.cc:492] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:15.960289  1753 ts_tablet_manager.cc:1403] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.292s	user 0.018s	sys 0.031s
I20260709 12:59:15.959089  1780 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:15.959426  1754 tablet_bootstrap.cc:492] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:15.961781  1754 ts_tablet_manager.cc:1403] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.218s	user 0.028s	sys 0.006s
I20260709 12:59:15.963294  1755 ts_tablet_manager.cc:1434] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.100s	user 0.021s	sys 0.013s
I20260709 12:59:15.964517  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:15.966496  1913 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:15.967756  1913 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.970000  1755 tablet_bootstrap.cc:492] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:15.970950  1893 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:15.992556  1754 raft_consensus.cc:359] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:15.993590  1754 raft_consensus.cc:385] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:15.993937  1754 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:15.996672  1754 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.000692  1913 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.013872  1754 ts_tablet_manager.cc:1434] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.000s
I20260709 12:59:16.015758  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:16.018217  1707 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 1: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 12:59:16.037559  1852 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:16.038918  1852 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:16.040238  1852 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.043479  1753 raft_consensus.cc:359] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.044327  1753 raft_consensus.cc:385] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.044682  1753 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.057830  1753 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.061563  1755 tablet_bootstrap.cc:654] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.069065  1753 ts_tablet_manager.cc:1434] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.109s	user 0.004s	sys 0.004s
I20260709 12:59:16.076000  1753 tablet_bootstrap.cc:492] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:16.091141  1926 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.091814  1926 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.094439  1754 tablet_bootstrap.cc:492] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:16.094120  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:16.098141  1755 tablet_bootstrap.cc:492] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.098760  1755 ts_tablet_manager.cc:1403] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.129s	user 0.012s	sys 0.008s
I20260709 12:59:16.101675  1926 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.108183  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:16.109035  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:16.110312  1919 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.113890  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:16.114581  1710 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:16.116870  1755 raft_consensus.cc:359] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.117599  1755 raft_consensus.cc:385] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.117843  1755 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
W20260709 12:59:16.112730  1525 leader_election.cc:343] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:16.118541  1755 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.118599  1753 tablet_bootstrap.cc:654] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.120249  1456 catalog_manager.cc:5697] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:16.124341  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:16.128964  1801 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:16.129451  1801 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:16.129894  1801 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.178577  1755 ts_tablet_manager.cc:1434] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.079s	user 0.004s	sys 0.005s
I20260709 12:59:16.184163  1754 tablet_bootstrap.cc:654] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.194417  1847 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.195334  1847 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.199301  1847 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.200804  1801 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.186614  1919 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:16.214116  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:16.218112  1458 catalog_manager.cc:5697] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:16.225165  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 12:59:16.227672  1525 leader_election.cc:343] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:16.233697  1801 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.235505  1755 tablet_bootstrap.cc:492] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:16.240882  1753 tablet_bootstrap.cc:492] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:16.243242  1753 ts_tablet_manager.cc:1403] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.168s	user 0.018s	sys 0.010s
I20260709 12:59:16.246408  1753 raft_consensus.cc:359] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.247519  1753 raft_consensus.cc:385] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.248122  1753 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.249099  1753 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.249775  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:16.251564  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:16.252171  1706 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.252522  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:16.253526  1919 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:16.265305  1919 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:16.269690  1755 tablet_bootstrap.cc:654] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.271963  1710 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.277115  1706 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:16.251613  1753 ts_tablet_manager.cc:1434] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:59:16.283967  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:16.285209  1801 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:16.285898  1801 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:16.286944  1801 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.289094  1755 tablet_bootstrap.cc:492] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.289636  1755 ts_tablet_manager.cc:1403] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.002s
I20260709 12:59:16.291687  1893 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.292219  1753 tablet_bootstrap.cc:492] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:16.292496  1919 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:16.292248  1893 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.294111  1754 tablet_bootstrap.cc:492] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:16.294770  1754 ts_tablet_manager.cc:1403] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.201s	user 0.042s	sys 0.044s
I20260709 12:59:16.295336  1893 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:16.297672  1754 raft_consensus.cc:359] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.298046  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:16.298408  1754 raft_consensus.cc:385] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.298736  1754 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.299405  1754 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.303241  1755 raft_consensus.cc:359] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.303424  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:16.298977  1456 catalog_manager.cc:5697] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:16.304912  1755 raft_consensus.cc:385] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.305258  1755 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.306609  1755 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.308892  1755 ts_tablet_manager.cc:1434] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 12:59:16.309979  1755 tablet_bootstrap.cc:492] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:16.313944  1675 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:16.321403  1754 ts_tablet_manager.cc:1434] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:59:16.319650  1893 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:16.343006  1754 tablet_bootstrap.cc:492] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:16.343796  1753 tablet_bootstrap.cc:654] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.341368  1755 tablet_bootstrap.cc:654] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.358651  1754 tablet_bootstrap.cc:654] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.361227  1753 tablet_bootstrap.cc:492] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:16.361941  1753 ts_tablet_manager.cc:1403] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.078s	user 0.012s	sys 0.009s
I20260709 12:59:16.370847  1753 raft_consensus.cc:359] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.371801  1753 raft_consensus.cc:385] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.372145  1753 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.373075  1753 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.376044  1753 ts_tablet_manager.cc:1434] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 12:59:16.377467  1753 tablet_bootstrap.cc:492] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:16.378494  1636 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.380609  1754 tablet_bootstrap.cc:492] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:16.381242  1754 ts_tablet_manager.cc:1403] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.006s
I20260709 12:59:16.384428  1753 tablet_bootstrap.cc:654] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.375396  1457 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:16.386409  1755 tablet_bootstrap.cc:492] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.386960  1755 ts_tablet_manager.cc:1403] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.077s	user 0.014s	sys 0.002s
I20260709 12:59:16.388362  1847 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.388868  1847 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.391220  1847 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.393939  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:16.395531  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:16.396420  1706 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 1: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 12:59:16.403846  1601 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:16.406486  1847 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.410831  1755 raft_consensus.cc:359] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.412001  1755 raft_consensus.cc:385] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.413350  1755 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.414494  1755 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.417548  1755 ts_tablet_manager.cc:1434] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 12:59:16.412446  1754 raft_consensus.cc:359] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.418192  1754 raft_consensus.cc:385] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.418635  1754 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.419809  1754 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.423283  1754 ts_tablet_manager.cc:1434] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.001s
I20260709 12:59:16.426337  1754 tablet_bootstrap.cc:492] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:16.424932  1755 tablet_bootstrap.cc:492] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:16.434041  1753 tablet_bootstrap.cc:492] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:16.434635  1753 ts_tablet_manager.cc:1403] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.058s	user 0.016s	sys 0.036s
I20260709 12:59:16.436932  1755 tablet_bootstrap.cc:654] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.437727  1753 raft_consensus.cc:359] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.438505  1753 raft_consensus.cc:385] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.438771  1753 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.445240  1754 tablet_bootstrap.cc:654] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.449750  1753 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.453289  1753 ts_tablet_manager.cc:1434] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:59:16.455615  1753 tablet_bootstrap.cc:492] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:16.464208  1754 tablet_bootstrap.cc:492] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:16.464968  1754 ts_tablet_manager.cc:1403] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.002s
I20260709 12:59:16.468871  1754 raft_consensus.cc:359] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.469821  1893 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.470402  1893 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.472147  1753 tablet_bootstrap.cc:654] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.473912  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:16.474694  1559 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:16.469870  1754 raft_consensus.cc:385] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.476548  1754 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.477970  1675 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:16.477614  1754 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.479107  1838 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:16.479485  1838 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:16.479825  1838 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.480343  1754 ts_tablet_manager.cc:1434] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.007s
I20260709 12:59:16.482100  1754 tablet_bootstrap.cc:492] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:16.482097  1893 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:16.421196  1847 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:16.514277  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:16.504699  1893 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:16.517903  1674 leader_election.cc:343] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:16.519716  1801 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:16.519892  1838 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.525444  1755 tablet_bootstrap.cc:492] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.525909  1754 tablet_bootstrap.cc:654] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.526086  1755 ts_tablet_manager.cc:1403] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.102s	user 0.032s	sys 0.025s
I20260709 12:59:16.527058  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:16.527758  1559 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.517892  1893 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.529124  1755 raft_consensus.cc:359] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.529870  1755 raft_consensus.cc:385] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.530136  1755 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.530860  1753 tablet_bootstrap.cc:492] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:16.531338  1753 ts_tablet_manager.cc:1403] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.076s	user 0.015s	sys 0.017s
I20260709 12:59:16.532917  1893 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:16.533424  1847 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.533651  1755 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.534749  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:16.535168  1753 raft_consensus.cc:359] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.540097  1753 raft_consensus.cc:385] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.540763  1753 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.540906  1755 ts_tablet_manager.cc:1434] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:59:16.541519  1753 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.542090  1755 tablet_bootstrap.cc:492] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
W20260709 12:59:16.542299  1675 leader_election.cc:343] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:16.545954  1753 ts_tablet_manager.cc:1434] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 12:59:16.548308  1753 tablet_bootstrap.cc:492] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:16.552260  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 12:59:16.554040  1674 leader_election.cc:343] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:16.554049  1838 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:16.554519  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:16.537096  1847 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.554920  1918 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:16.557919  1838 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:16.558212  1559 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:16.558851  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:16.559535  1558 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:16.558880  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:16.561187  1636 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.563386  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:16.564487  1847 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.564651  1919 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:16.568749  1755 tablet_bootstrap.cc:654] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.569813  1919 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:16.570263  1919 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.584764  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:16.585600  1706 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:16.593353  1754 tablet_bootstrap.cc:492] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:16.595247  1754 ts_tablet_manager.cc:1403] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.113s	user 0.020s	sys 0.005s
I20260709 12:59:16.589241  1636 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:16.596942  1675 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:16.600627  1838 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:16.601257  1838 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:16.602265  1838 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.605741  1754 raft_consensus.cc:359] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.606487  1754 raft_consensus.cc:385] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.606879  1754 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.607273  1801 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.607622  1754 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.607852  1801 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.609336  1754 ts_tablet_manager.cc:1434] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:59:16.610203  1801 leader_election.cc:290] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.611016  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:16.611977  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:16.613870  1525 leader_election.cc:343] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:16.614660  1526 leader_election.cc:343] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:16.614557  1919 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.615146  1755 tablet_bootstrap.cc:492] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.615217  1526 leader_election.cc:304] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:16.615906  1755 ts_tablet_manager.cc:1403] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.074s	user 0.011s	sys 0.018s
I20260709 12:59:16.616122  1801 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:16.619398  1755 raft_consensus.cc:359] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.620129  1755 raft_consensus.cc:385] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.620432  1755 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.622083  1755 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.624658  1755 ts_tablet_manager.cc:1434] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:59:16.625829  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:16.626434  1558 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.627993  1755 tablet_bootstrap.cc:492] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:16.636080  1558 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:16.636644  1755 tablet_bootstrap.cc:654] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.639382  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:16.641553  1753 tablet_bootstrap.cc:654] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.642026  1948 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:16.642549  1948 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:16.643016  1754 tablet_bootstrap.cc:492] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:16.644011  1948 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.649947  1754 tablet_bootstrap.cc:654] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.643177  1919 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.651722  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:16.652365  1706 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.671586  1706 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:16.645270  1457 catalog_manager.cc:5697] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:16.682019  1801 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.682705  1801 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.684301  1753 tablet_bootstrap.cc:492] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:16.684829  1753 ts_tablet_manager.cc:1403] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.137s	user 0.029s	sys 0.015s
I20260709 12:59:16.684908  1801 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.687824  1755 tablet_bootstrap.cc:492] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.687958  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:16.688441  1755 ts_tablet_manager.cc:1403] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.061s	user 0.007s	sys 0.017s
I20260709 12:59:16.687937  1753 raft_consensus.cc:359] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.688841  1753 raft_consensus.cc:385] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.689116  1753 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.689990  1753 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.691660  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:16.692313  1710 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:16.692513  1753 ts_tablet_manager.cc:1434] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:59:16.692553  1755 raft_consensus.cc:359] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.693342  1755 raft_consensus.cc:385] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.693645  1755 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.693796  1753 tablet_bootstrap.cc:492] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:16.694489  1755 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:16.695865  1525 leader_election.cc:343] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:16.697419  1755 ts_tablet_manager.cc:1434] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:16.698522  1755 tablet_bootstrap.cc:492] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:16.702221  1753 tablet_bootstrap.cc:654] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.703912  1755 tablet_bootstrap.cc:654] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.706911  1456 catalog_manager.cc:5697] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:16.709811  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:16.711760  1926 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:16.712144  1926 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:16.712558  1926 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.716408  1755 tablet_bootstrap.cc:492] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.716933  1755 ts_tablet_manager.cc:1403] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.009s
I20260709 12:59:16.719853  1755 raft_consensus.cc:359] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.720589  1755 raft_consensus.cc:385] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.720867  1755 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.720783  1926 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.721839  1755 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.723958  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:16.724320  1755 ts_tablet_manager.cc:1434] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
W20260709 12:59:16.725734  1525 leader_election.cc:343] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:16.727300  1926 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.728648  1755 tablet_bootstrap.cc:492] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:16.729794  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:16.730437  1708 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.739320  1708 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:16.742758  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:16.744452  1926 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:16.744671  1755 tablet_bootstrap.cc:654] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.745150  1926 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:16.746374  1926 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.755117  1754 tablet_bootstrap.cc:492] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:16.755697  1754 ts_tablet_manager.cc:1403] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.115s	user 0.015s	sys 0.013s
I20260709 12:59:16.759145  1754 raft_consensus.cc:359] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.760329  1754 raft_consensus.cc:385] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.760834  1755 tablet_bootstrap.cc:492] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.760748  1754 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.761355  1755 ts_tablet_manager.cc:1403] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.007s
I20260709 12:59:16.762362  1754 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.763813  1755 raft_consensus.cc:359] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.764393  1755 raft_consensus.cc:385] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.764628  1755 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.765440  1755 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.767706  1755 ts_tablet_manager.cc:1434] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:16.770273  1755 tablet_bootstrap.cc:492] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:16.771539  1754 ts_tablet_manager.cc:1434] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 12:59:16.772769  1754 tablet_bootstrap.cc:492] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:16.781404  1753 tablet_bootstrap.cc:492] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:16.781821  1755 tablet_bootstrap.cc:654] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.781962  1753 ts_tablet_manager.cc:1403] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.088s	user 0.027s	sys 0.045s
I20260709 12:59:16.782274  1754 tablet_bootstrap.cc:654] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.785323  1753 raft_consensus.cc:359] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.786610  1753 raft_consensus.cc:385] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.786916  1753 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.788218  1753 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.791006  1753 ts_tablet_manager.cc:1434] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 12:59:16.792606  1753 tablet_bootstrap.cc:492] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:16.795152  1755 tablet_bootstrap.cc:492] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.794693  1456 catalog_manager.cc:5697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:16.795642  1755 ts_tablet_manager.cc:1403] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.005s
I20260709 12:59:16.798547  1755 raft_consensus.cc:359] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.799213  1753 tablet_bootstrap.cc:654] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.799266  1755 raft_consensus.cc:385] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.799697  1755 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.800424  1755 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.810598  1755 ts_tablet_manager.cc:1434] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:59:16.812122  1755 tablet_bootstrap.cc:492] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:16.821798  1948 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.822360  1948 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.826215  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:16.828979  1948 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.830674  1755 tablet_bootstrap.cc:654] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.833452  1754 tablet_bootstrap.cc:492] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:16.834358  1754 ts_tablet_manager.cc:1403] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.005s
I20260709 12:59:16.834153  1926 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:16.840394  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:16.844448  1708 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 1: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 12:59:16.844530  1755 tablet_bootstrap.cc:492] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.845764  1755 ts_tablet_manager.cc:1403] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.034s	user 0.018s	sys 0.002s
I20260709 12:59:16.847937  1601 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:16.848311  1755 raft_consensus.cc:359] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.848205  1754 raft_consensus.cc:359] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.849033  1755 raft_consensus.cc:385] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.849151  1754 raft_consensus.cc:385] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.849354  1755 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.849563  1754 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.850051  1948 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.850267  1755 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.850633  1754 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.852912  1755 ts_tablet_manager.cc:1434] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:16.853969  1755 tablet_bootstrap.cc:492] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
W20260709 12:59:16.861689  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:16.852918  1754 ts_tablet_manager.cc:1434] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:16.864411  1754 tablet_bootstrap.cc:492] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:16.872342  1755 tablet_bootstrap.cc:654] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.872342  1754 tablet_bootstrap.cc:654] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.887873  1801 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:16.891296  1755 tablet_bootstrap.cc:492] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.891865  1755 ts_tablet_manager.cc:1403] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.006s
I20260709 12:59:16.895517  1755 raft_consensus.cc:359] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.896237  1755 raft_consensus.cc:385] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.899444  1919 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.900144  1919 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.896514  1755 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.902314  1755 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.903409  1919 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.905272  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:16.906036  1755 ts_tablet_manager.cc:1434] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.002s
I20260709 12:59:16.905905  1948 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:16.907240  1755 tablet_bootstrap.cc:492] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:16.909421  1838 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.910198  1838 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:16.907927  1601 leader_election.cc:343] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:16.914837  1753 tablet_bootstrap.cc:492] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:16.914479  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:16.915314  1753 ts_tablet_manager.cc:1403] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.123s	user 0.035s	sys 0.045s
I20260709 12:59:16.915426  1708 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:16.918373  1753 raft_consensus.cc:359] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.919047  1753 raft_consensus.cc:385] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.919308  1753 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.919581  1838 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:16.920780  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:16.920398  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:16.923121  1948 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:16.923473  1558 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:16.923753  1948 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:16.923142  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:16.924294  1948 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.924070  1753 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.924906  1636 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:16.926826  1753 ts_tablet_manager.cc:1434] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 12:59:16.926690  1675 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:16.927564  1838 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:16.927913  1838 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:16.928232  1838 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.939544  1838 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.942699  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:16.943389  1558 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.936800  1948 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.944016  1753 tablet_bootstrap.cc:492] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:16.944448  1755 tablet_bootstrap.cc:654] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.949012  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:16.951409  1919 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:16.951086  1601 leader_election.cc:343] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:16.952754  1919 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.953897  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:16.952128  1948 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.954850  1636 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.955273  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:16.964924  1708 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:16.951121  1838 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:16.963865  1754 tablet_bootstrap.cc:492] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:16.967123  1754 ts_tablet_manager.cc:1403] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.103s	user 0.029s	sys 0.048s
I20260709 12:59:16.968654  1558 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:16.970995  1753 tablet_bootstrap.cc:654] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:16.973284  1675 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:16.974303  1838 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:16.974814  1838 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:16.977195  1838 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.980417  1919 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:16.981197  1754 raft_consensus.cc:359] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.982034  1754 raft_consensus.cc:385] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.982364  1754 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.983208  1754 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.984140  1708 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:16.988121  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:16.988535  1755 tablet_bootstrap.cc:492] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:16.989053  1755 ts_tablet_manager.cc:1403] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.082s	user 0.015s	sys 0.001s
I20260709 12:59:16.990497  1893 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:16.990690  1754 ts_tablet_manager.cc:1434] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 12:59:16.992257  1755 raft_consensus.cc:359] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.992975  1755 raft_consensus.cc:385] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:16.993257  1755 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:16.993952  1754 tablet_bootstrap.cc:492] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:16.991041  1893 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:16.996122  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:16.996569  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:16.996970  1558 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:16.994865  1919 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:16.999025  1919 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:16.999564  1893 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:16.999900  1919 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.000733  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:17.001384  1635 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:16.995273  1755 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.002291  1707 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:17.002417  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:17.002722  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:17.002976  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:17.014428  1558 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:17.005314  1753 tablet_bootstrap.cc:492] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:17.015050  1948 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:17.015437  1948 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:17.015707  1838 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:17.015990  1948 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.016076  1838 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:17.016495  1838 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.018042  1847 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:17.018661  1847 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.022604  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:17.023419  1847 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.009353  1754 tablet_bootstrap.cc:654] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.026157  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:17.027523  1601 leader_election.cc:343] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:17.011698  1636 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:17.015221  1753 ts_tablet_manager.cc:1403] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.005s
W20260709 12:59:17.029726  1601 leader_election.cc:343] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:17.030202  1601 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:17.032239  1753 raft_consensus.cc:359] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.035724  1753 raft_consensus.cc:385] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.036103  1753 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:17.036322  1780 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:17.032219  1948 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.034515  1838 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.037163  1459 catalog_manager.cc:5697] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:17.040876  1908 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:17.043004  1838 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:17.043678  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:17.043747  1753 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.045013  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:17.046670  1948 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.045635  1908 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:17.046186  1847 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:17.044400  1558 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.047736  1908 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.049118  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:17.049800  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:17.050002  1707 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 1: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 12:59:17.050223  1908 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.046046  1753 ts_tablet_manager.cc:1434] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.002s
I20260709 12:59:17.051654  1635 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 1: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 12:59:17.051751  1753 tablet_bootstrap.cc:492] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:17.033828  1755 ts_tablet_manager.cc:1434] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 12:59:17.054857  1755 tablet_bootstrap.cc:492] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:17.057821  1754 tablet_bootstrap.cc:492] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:17.058310  1754 ts_tablet_manager.cc:1403] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.067s	user 0.015s	sys 0.000s
I20260709 12:59:17.061427  1755 tablet_bootstrap.cc:654] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.062218  1754 raft_consensus.cc:359] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.064978  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:17.066339  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:17.062913  1754 raft_consensus.cc:385] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.068665  1754 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
W20260709 12:59:17.069891  1601 leader_election.cc:343] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:17.072319  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:17.074571  1459 catalog_manager.cc:5697] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:17.075407  1847 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:17.078297  1754 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.078785  1753 tablet_bootstrap.cc:654] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.085799  1754 ts_tablet_manager.cc:1434] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.001s
I20260709 12:59:17.084694  1559 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:17.085816  1558 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:17.090196  1754 tablet_bootstrap.cc:492] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:17.090898  1675 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:17.093093  1811 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:17.093614  1811 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:17.094447  1811 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.096328  1926 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:17.096971  1926 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.101274  1755 tablet_bootstrap.cc:492] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:17.102329  1755 ts_tablet_manager.cc:1403] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.048s	user 0.018s	sys 0.003s
I20260709 12:59:17.110198  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:17.122917  1847 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:17.123544  1847 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:17.124003  1847 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.147274  1908 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:17.147989  1908 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.152556  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:17.153505  1707 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:17.155696  1755 raft_consensus.cc:359] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.156636  1755 raft_consensus.cc:385] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.156951  1755 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:17.124874  1926 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.158972  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:17.159778  1635 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:17.160897  1754 tablet_bootstrap.cc:654] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.162271  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:17.173369  1926 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:17.173848  1926 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:17.174252  1926 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.173645  1893 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:17.163209  1755 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.185420  1755 ts_tablet_manager.cc:1434] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.000s
I20260709 12:59:17.191537  1919 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:59:17.202831  1755 tablet_bootstrap.cc:492] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:17.210309  1908 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.212173  1753 tablet_bootstrap.cc:492] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:17.212806  1753 ts_tablet_manager.cc:1403] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.161s	user 0.015s	sys 0.013s
I20260709 12:59:17.213282  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:17.215184  1710 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:17.232467  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:17.233269  1558 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:17.234653  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 12:59:17.223903  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 55619.2us
I20260709 12:59:17.239734  1908 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:17.240219  1908 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:17.240653  1908 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.221122  1458 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:17.246982  1847 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.321259  1847 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.330312  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 12:59:17.334316  1601 leader_election.cc:343] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:17.344249  1801 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:17.309347  1926 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.368306  1754 tablet_bootstrap.cc:492] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:17.368970  1754 ts_tablet_manager.cc:1403] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.280s	user 0.018s	sys 0.015s
I20260709 12:59:17.371091  1908 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.377031  1754 raft_consensus.cc:359] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.379844  1801 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:17.424815  1754 raft_consensus.cc:385] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.425206  1754 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:17.425947  1754 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.428339  1754 ts_tablet_manager.cc:1434] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 12:59:17.430130  1948 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:17.430248  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:17.431051  1632 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.430851  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:17.432453  1558 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.430861  1948 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.436820  1948 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.437577  1754 tablet_bootstrap.cc:492] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:17.448115  1632 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:17.450618  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:17.455106  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:17.457058  1984 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:17.457659  1984 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 12:59:17.458992  1601 leader_election.cc:343] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:17.375072  1753 raft_consensus.cc:359] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.458911  1984 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.459878  1753 raft_consensus.cc:385] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.460446  1753 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:17.461421  1753 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.463318  1753 ts_tablet_manager.cc:1434] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.250s	user 0.012s	sys 0.016s
I20260709 12:59:17.373719  1926 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.477911  1755 tablet_bootstrap.cc:654] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.502307  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:17.505627  1558 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:17.508240  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:17.509109  1556 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.514048  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:17.515182  1710 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:17.523658  1601 leader_election.cc:343] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:17.524199  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 12:59:17.524942  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44653.1us
I20260709 12:59:17.528067  1754 tablet_bootstrap.cc:654] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.530897  1983 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:17.551092  1983 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.536612  1981 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:17.537657  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:17.541682  1556 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:17.554477  1983 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.541676  1926 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:17.562304  1710 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:17.523458  1908 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.560736  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:17.563313  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:17.564109  1708 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.562016  1926 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.547083  1978 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:17.569890  1926 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.570556  1908 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:17.571485  1919 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:17.573294  1908 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:17.575217  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:17.574405  1908 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.576011  1629 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:17.577663  1525 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:17.578701  1926 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:17.579156  1926 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:17.557421  1981 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:17.579964  1926 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.580402  1981 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:17.581616  1981 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.559569  1556 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.585888  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:17.586798  1706 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 1: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 12:59:17.556274  1753 tablet_bootstrap.cc:492] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:17.592375  1926 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.594380  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:17.595131  1706 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:17.595979  1978 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:17.596966  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:17.597687  1629 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.607333  1978 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:17.610584  1926 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.623257  1754 tablet_bootstrap.cc:492] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:17.623939  1754 ts_tablet_manager.cc:1403] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.195s	user 0.023s	sys 0.008s
I20260709 12:59:17.626530  1456 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:17.629691  1754 raft_consensus.cc:359] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.630594  1754 raft_consensus.cc:385] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.630941  1754 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:17.632369  1754 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.636377  1754 ts_tablet_manager.cc:1434] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.002s
I20260709 12:59:17.618268  1629 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:17.612223  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:17.645663  1706 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.644729  1837 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:17.652016  1754 tablet_bootstrap.cc:492] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:17.642153  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:17.662098  1525 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:17.668819  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:17.669505  1708 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:17.669166  1983 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:17.673635  1755 tablet_bootstrap.cc:492] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:17.674255  1755 ts_tablet_manager.cc:1403] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.477s	user 0.016s	sys 0.029s
I20260709 12:59:17.682727  1755 raft_consensus.cc:359] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.683568  1755 raft_consensus.cc:385] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.683871  1755 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:17.677700  1983 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:17.676460  1926 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:17.685431  1983 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.702840  1754 tablet_bootstrap.cc:654] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.711428  1753 tablet_bootstrap.cc:654] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.703631  1755 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.722505  1981 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:17.739950  1706 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:17.751473  1456 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:17.755887  1456 catalog_manager.cc:5697] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:17.748978  1458 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:17.765390  1755 ts_tablet_manager.cc:1434] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.091s	user 0.012s	sys 0.001s
I20260709 12:59:17.770519  1755 tablet_bootstrap.cc:492] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:17.753790  1981 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:17.777854  1981 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.782303  1981 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.784878  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 12:59:17.786376  1999 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 12:59:17.788759  1601 leader_election.cc:343] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:17.794480  1755 tablet_bootstrap.cc:654] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.807679  1754 tablet_bootstrap.cc:492] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:17.809252  1754 ts_tablet_manager.cc:1403] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.158s	user 0.014s	sys 0.038s
I20260709 12:59:17.816092  1926 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:17.816138  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:17.816803  1926 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.826236  1753 tablet_bootstrap.cc:492] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:17.826719  1926 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 12:59:17.826669  1601 leader_election.cc:343] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:17.826836  1753 ts_tablet_manager.cc:1403] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.271s	user 0.006s	sys 0.037s
I20260709 12:59:17.827461  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:17.829023  1981 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:17.832398  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:17.832991  1753 raft_consensus.cc:359] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.833129  1754 raft_consensus.cc:359] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.833732  1753 raft_consensus.cc:385] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.833129  1632 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:17.834039  1753 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:17.834771  1753 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.835222  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:17.836264  1753 ts_tablet_manager.cc:1434] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.000s
I20260709 12:59:17.836620  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:17.836300  1926 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:17.837352  1707 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:17.837544  1753 tablet_bootstrap.cc:492] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:17.837745  1926 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:17.838781  1926 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.833895  1754 raft_consensus.cc:385] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.840055  1754 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:17.841219  1754 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.864500  1755 tablet_bootstrap.cc:492] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:17.865073  1755 ts_tablet_manager.cc:1403] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.097s	user 0.022s	sys 0.003s
I20260709 12:59:17.868846  1755 raft_consensus.cc:359] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.869596  1755 raft_consensus.cc:385] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.869885  1755 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:17.870541  1755 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.871806  1754 ts_tablet_manager.cc:1434] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.000s
I20260709 12:59:17.874913  1754 tablet_bootstrap.cc:492] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:17.876776  1926 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.883466  1755 ts_tablet_manager.cc:1434] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.018s	user 0.000s	sys 0.007s
I20260709 12:59:17.895323  1754 tablet_bootstrap.cc:654] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.896703  1755 tablet_bootstrap.cc:492] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:17.912614  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:17.913311  1753 tablet_bootstrap.cc:654] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.913335  1632 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.917196  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:17.917840  1710 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:17.918004  1926 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:17.923240  1632 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:17.924438  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:17.925319  1918 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:17.925845  1918 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:17.926368  1983 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:17.927932  1755 tablet_bootstrap.cc:654] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.926807  1918 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.934898  1753 tablet_bootstrap.cc:492] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:17.925814  1710 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:17.935406  1753 ts_tablet_manager.cc:1403] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.098s	user 0.012s	sys 0.002s
I20260709 12:59:17.938314  1754 tablet_bootstrap.cc:492] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:17.938838  1754 ts_tablet_manager.cc:1403] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.005s
I20260709 12:59:17.941601  1753 raft_consensus.cc:359] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.942763  1753 raft_consensus.cc:385] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.943106  1753 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:17.943923  1753 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.948130  1754 raft_consensus.cc:359] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.948841  1754 raft_consensus.cc:385] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:17.949095  1754 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:17.951642  1753 ts_tablet_manager.cc:1434] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:59:17.952492  1754 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:17.953893  1753 tablet_bootstrap.cc:492] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
W20260709 12:59:17.961150  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:17.961388  1754 ts_tablet_manager.cc:1434] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:59:17.962481  1754 tablet_bootstrap.cc:492] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:17.968608  1926 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:17.973057  1753 tablet_bootstrap.cc:654] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.981285  1754 tablet_bootstrap.cc:654] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:17.996691  1754 tablet_bootstrap.cc:492] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:17.997233  1754 ts_tablet_manager.cc:1403] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.002s
I20260709 12:59:17.998713  1755 tablet_bootstrap.cc:492] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:17.999387  1755 ts_tablet_manager.cc:1403] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.115s	user 0.008s	sys 0.012s
I20260709 12:59:18.000350  1754 raft_consensus.cc:359] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.001055  1754 raft_consensus.cc:385] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.001592  1754 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.002429  1754 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.004688  1754 ts_tablet_manager.cc:1434] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:59:18.005779  1754 tablet_bootstrap.cc:492] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
W20260709 12:59:18.006031  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:18.014599  1918 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.015282  1918 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.017554  1780 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.018059  1780 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.018929  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:18.019644  1631 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:18.020309  1780 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:18.022110  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:18.022316  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:18.023190  1926 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:59:18.023630  1526 leader_election.cc:343] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.023710  1926 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:18.024216  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:18.024302  1926 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.024998  1631 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:18.026355  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:18.027042  1559 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:18.027164  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:18.017563  1918 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:18.028602  1780 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:18.029075  1984 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.029567  1780 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:18.029639  1984 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.017068  1755 raft_consensus.cc:359] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.031203  1755 raft_consensus.cc:385] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.032079  1984 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:18.032866  1755 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.034898  1458 catalog_manager.cc:5697] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:18.037796  1754 tablet_bootstrap.cc:654] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.043330  1926 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.030071  1780 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.034911  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:18.045310  1847 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:18.036378  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:18.046290  1926 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 12:59:18.047626  1526 leader_election.cc:343] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:18.050282  1525 leader_election.cc:343] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.051551  1525 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:18.052564  1985 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:18.053650  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:18.054275  1632 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.057951  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 12:59:18.059664  1526 leader_election.cc:343] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.071357  1981 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:18.079933  1755 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.086889  1780 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.110103  1847 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:18.089429  1755 ts_tablet_manager.cc:1434] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.090s	user 0.006s	sys 0.001s
I20260709 12:59:18.125684  1755 tablet_bootstrap.cc:492] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:18.082243  1632 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:18.130573  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:18.131108  1753 tablet_bootstrap.cc:492] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:18.131621  1753 ts_tablet_manager.cc:1403] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.178s	user 0.039s	sys 0.057s
I20260709 12:59:18.131845  1984 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:18.132592  1984 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:18.132987  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:18.133713  1559 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.134428  1984 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.135109  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:18.135769  1632 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.135859  1753 raft_consensus.cc:359] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.136679  1753 raft_consensus.cc:385] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.136979  1753 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.145785  1753 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.137704  1780 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:18.151384  1559 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:18.153656  1675 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:18.154599  1780 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:18.155879  1632 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:18.158583  1755 tablet_bootstrap.cc:654] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.158367  1780 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:18.160454  1753 ts_tablet_manager.cc:1434] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.002s
I20260709 12:59:18.159313  1847 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.161543  1847 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.166023  1753 tablet_bootstrap.cc:492] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:18.166472  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 12:59:18.168211  1601 leader_election.cc:343] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.169046  1780 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.174297  1847 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:18.175926  1754 tablet_bootstrap.cc:492] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:18.176486  1754 ts_tablet_manager.cc:1403] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.171s	user 0.026s	sys 0.049s
I20260709 12:59:18.175865  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:18.177923  1601 leader_election.cc:343] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.178406  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:18.179188  1847 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:18.180909  1753 tablet_bootstrap.cc:654] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.182103  1754 raft_consensus.cc:359] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.185665  1754 raft_consensus.cc:385] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.186004  1754 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.186941  1754 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.189229  1754 ts_tablet_manager.cc:1434] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:59:18.190316  1755 tablet_bootstrap.cc:492] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:18.190335  1754 tablet_bootstrap.cc:492] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:18.197088  1755 ts_tablet_manager.cc:1403] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.072s	user 0.016s	sys 0.003s
I20260709 12:59:18.200879  1755 raft_consensus.cc:359] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.204852  1755 raft_consensus.cc:385] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.205217  1755 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.206079  1755 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.208510  1755 ts_tablet_manager.cc:1434] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:59:18.208106  1456 catalog_manager.cc:5697] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:18.214557  1755 tablet_bootstrap.cc:492] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:18.216976  1753 tablet_bootstrap.cc:492] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:18.217566  1753 ts_tablet_manager.cc:1403] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.007s
I20260709 12:59:18.221429  1753 raft_consensus.cc:359] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.222151  1753 raft_consensus.cc:385] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.222424  1753 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.223274  1755 tablet_bootstrap.cc:654] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.204572  1754 tablet_bootstrap.cc:654] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.223111  1753 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.229060  1753 ts_tablet_manager.cc:1434] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 12:59:18.230358  1753 tablet_bootstrap.cc:492] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:18.231103  1926 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:18.232558  1457 catalog_manager.cc:5697] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:18.238890  1755 tablet_bootstrap.cc:492] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:18.239454  1755 ts_tablet_manager.cc:1403] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.010s
I20260709 12:59:18.240267  1754 tablet_bootstrap.cc:492] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:18.240741  1754 ts_tablet_manager.cc:1403] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.008s
I20260709 12:59:18.243213  1755 raft_consensus.cc:359] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.243968  1755 raft_consensus.cc:385] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.244215  1755 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.244746  1754 raft_consensus.cc:359] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.245425  1754 raft_consensus.cc:385] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.245126  1755 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.245831  1754 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.246460  1754 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.247726  1755 ts_tablet_manager.cc:1434] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:59:18.248359  1754 ts_tablet_manager.cc:1434] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:59:18.248759  1755 tablet_bootstrap.cc:492] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:18.249308  1754 tablet_bootstrap.cc:492] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:18.268795  1755 tablet_bootstrap.cc:654] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.268621  1985 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:18.272207  1753 tablet_bootstrap.cc:654] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.282868  1754 tablet_bootstrap.cc:654] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.284746  1985 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.285265  1985 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.291927  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:18.294305  1526 leader_election.cc:343] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.294978  1753 tablet_bootstrap.cc:492] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:18.295199  1985 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:18.295516  1753 ts_tablet_manager.cc:1403] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.065s	user 0.011s	sys 0.002s
I20260709 12:59:18.298548  1753 raft_consensus.cc:359] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.299271  1753 raft_consensus.cc:385] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.299553  1753 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.300221  1753 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.296331  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 12:59:18.304080  1525 leader_election.cc:343] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.304327  1753 ts_tablet_manager.cc:1434] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:59:18.304579  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:18.305275  1753 tablet_bootstrap.cc:492] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:18.305387  1985 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:18.314107  1754 tablet_bootstrap.cc:492] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:18.314114  1753 tablet_bootstrap.cc:654] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.315034  1754 ts_tablet_manager.cc:1403] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.066s	user 0.008s	sys 0.006s
I20260709 12:59:18.317458  1754 raft_consensus.cc:359] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.318118  1754 raft_consensus.cc:385] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.318363  1754 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.318965  1754 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.320951  1754 ts_tablet_manager.cc:1434] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:59:18.322907  1754 tablet_bootstrap.cc:492] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:18.324155  1837 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.324388  1755 tablet_bootstrap.cc:492] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:18.324932  1755 ts_tablet_manager.cc:1403] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.076s	user 0.016s	sys 0.008s
I20260709 12:59:18.324712  1837 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.327678  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:18.328107  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:18.327653  1755 raft_consensus.cc:359] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.329653  1755 raft_consensus.cc:385] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.329926  1755 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.329839  1674 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:18.330776  1755 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.331108  1780 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:18.332923  1755 ts_tablet_manager.cc:1434] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:59:18.334169  1837 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:18.335759  1753 tablet_bootstrap.cc:492] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:18.336300  1753 ts_tablet_manager.cc:1403] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.011s
I20260709 12:59:18.338260  1755 tablet_bootstrap.cc:492] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:18.340549  1753 raft_consensus.cc:359] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.341266  1753 raft_consensus.cc:385] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.341563  1753 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.342303  1753 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.344498  1753 ts_tablet_manager.cc:1434] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:59:18.345029  1985 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.345319  1755 tablet_bootstrap.cc:654] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.345566  1985 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.349721  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:18.349854  1985 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:18.350349  1632 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:18.350613  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:18.351405  1707 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:18.352792  1526 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:18.354079  1985 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:18.354463  1985 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:18.354820  1985 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.355225  1754 tablet_bootstrap.cc:654] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.361286  1753 tablet_bootstrap.cc:492] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:18.367473  1755 tablet_bootstrap.cc:492] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:18.368057  1755 ts_tablet_manager.cc:1403] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.009s
I20260709 12:59:18.367697  1985 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.369107  1753 tablet_bootstrap.cc:654] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.370082  1985 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:18.371273  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:18.371572  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:18.372012  1632 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.372133  1707 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.371121  1755 raft_consensus.cc:359] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.381858  1754 tablet_bootstrap.cc:492] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:18.381740  1632 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:18.382459  1754 ts_tablet_manager.cc:1403] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.002s
I20260709 12:59:18.381740  1755 raft_consensus.cc:385] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.382979  1755 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.384418  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:18.385399  1985 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:18.386135  1755 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.386904  1754 raft_consensus.cc:359] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.387682  1754 raft_consensus.cc:385] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.387943  1754 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.387689  1707 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:18.388600  1755 ts_tablet_manager.cc:1434] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 12:59:18.390089  1755 tablet_bootstrap.cc:492] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:18.388600  1754 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.390228  1753 tablet_bootstrap.cc:492] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:18.386780  1985 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:18.393198  1753 ts_tablet_manager.cc:1403] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.001s
I20260709 12:59:18.393875  1985 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.399822  1754 ts_tablet_manager.cc:1434] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.001s
I20260709 12:59:18.401013  1754 tablet_bootstrap.cc:492] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:18.400652  1753 raft_consensus.cc:359] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.401506  1753 raft_consensus.cc:385] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.401705  1755 tablet_bootstrap.cc:654] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.401790  1753 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.402658  1753 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.404983  1753 ts_tablet_manager.cc:1434] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 12:59:18.406168  1753 tablet_bootstrap.cc:492] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
W20260709 12:59:18.411692  1587 tablet.cc:2406] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:59:18.422204  1753 tablet_bootstrap.cc:654] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.430169  1837 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.431001  1837 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.433329  1837 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:18.438356  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 12:59:18.441869  1674 leader_election.cc:343] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.439715  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:18.443516  1559 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:18.444976  1675 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:18.446192  1847 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.446643  1847 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.448671  1847 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:18.451193  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:18.453048  1601 leader_election.cc:343] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.453939  1837 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:18.454425  1837 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:18.454847  1837 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.458927  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 12:59:18.460970  1601 leader_election.cc:343] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.461623  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:18.463627  1908 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:18.472023  1983 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:18.491492  1754 tablet_bootstrap.cc:654] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.498553  1753 tablet_bootstrap.cc:492] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:18.499042  1753 ts_tablet_manager.cc:1403] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.093s	user 0.014s	sys 0.016s
I20260709 12:59:18.499576  1837 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.505520  1456 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:18.509407  1707 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.509577  1753 raft_consensus.cc:359] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.510308  1753 raft_consensus.cc:385] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.510557  1753 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.511178  1753 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.513288  1753 ts_tablet_manager.cc:1434] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 12:59:18.517570  1753 tablet_bootstrap.cc:492] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:18.518513  1755 tablet_bootstrap.cc:492] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:18.519004  1755 ts_tablet_manager.cc:1403] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.129s	user 0.022s	sys 0.040s
I20260709 12:59:18.521911  1755 raft_consensus.cc:359] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.522639  1755 raft_consensus.cc:385] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.522897  1755 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.523918  1753 tablet_bootstrap.cc:654] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.523545  1755 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.514249  1837 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:18.515507  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:18.538872  1755 ts_tablet_manager.cc:1434] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.001s
I20260709 12:59:18.540019  1755 tablet_bootstrap.cc:492] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:18.540683  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:18.541740  1559 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:18.541981  1674 leader_election.cc:343] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.557391  1755 tablet_bootstrap.cc:654] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.580073  1926 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:18.593044  1754 tablet_bootstrap.cc:492] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:18.593647  1754 ts_tablet_manager.cc:1403] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.193s	user 0.038s	sys 0.007s
I20260709 12:59:18.594643  1753 tablet_bootstrap.cc:492] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:18.595131  1753 ts_tablet_manager.cc:1403] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.078s	user 0.014s	sys 0.005s
I20260709 12:59:18.596434  1837 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.596442  1754 raft_consensus.cc:359] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.597311  1754 raft_consensus.cc:385] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.597672  1754 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.597182  1837 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.598395  1754 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.600661  1754 ts_tablet_manager.cc:1434] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:59:18.601085  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:18.600879  1559 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:18.601864  1754 tablet_bootstrap.cc:492] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:18.604952  1755 tablet_bootstrap.cc:492] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:18.605419  1755 ts_tablet_manager.cc:1403] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.066s	user 0.019s	sys 0.015s
I20260709 12:59:18.608337  1755 raft_consensus.cc:359] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.609032  1755 raft_consensus.cc:385] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.609272  1755 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.610008  1754 tablet_bootstrap.cc:654] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.609970  1755 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.612457  1755 ts_tablet_manager.cc:1434] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:18.609750  1753 raft_consensus.cc:359] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.614146  1753 raft_consensus.cc:385] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.614436  1753 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.615096  1753 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:18.616981  1675 leader_election.cc:343] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.617286  1753 ts_tablet_manager.cc:1434] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 12:59:18.617836  1675 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:18.618309  1753 tablet_bootstrap.cc:492] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:18.619172  1978 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:18.619860  1978 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:18.620968  1978 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.623430  1755 tablet_bootstrap.cc:492] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
W20260709 12:59:18.627880  1526 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:18.628715  1754 tablet_bootstrap.cc:492] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:18.629972  1755 tablet_bootstrap.cc:654] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.629221  1754 ts_tablet_manager.cc:1403] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.004s
I20260709 12:59:18.640090  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:18.640971  1631 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:18.642426  1780 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.643139  1780 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.645702  1918 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.646189  1918 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.648289  1918 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:18.651563  1754 raft_consensus.cc:359] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.653110  1754 raft_consensus.cc:385] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.653378  1754 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.654026  1754 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.655709  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:18.656857  1754 ts_tablet_manager.cc:1434] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.003s
I20260709 12:59:18.658540  1979 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:18.659097  1979 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:18.659507  1979 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.662622  1985 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:18.663160  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:18.665036  1753 tablet_bootstrap.cc:654] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.645401  1780 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:18.667317  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:18.668746  1632 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:18.669992  1674 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:18.670807  1780 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:18.672281  1780 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:18.672765  1780 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.673280  1754 tablet_bootstrap.cc:492] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
W20260709 12:59:18.672896  1526 leader_election.cc:343] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:18.642321  1837 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:18.680773  1847 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.681608  1847 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.684366  1754 tablet_bootstrap.cc:654] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.688225  1755 tablet_bootstrap.cc:492] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:18.688731  1755 ts_tablet_manager.cc:1403] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.075s	user 0.021s	sys 0.024s
I20260709 12:59:18.695112  1755 raft_consensus.cc:359] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.695953  1755 raft_consensus.cc:385] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.696251  1755 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.697001  1755 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.697353  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:18.698457  1908 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.699172  1908 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.695489  1847 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:18.758241  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:18.771930  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:18.922734  1559 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:18.903788  1629 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:18.949445  1908 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:18.930534  1556 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:18.956014  1908 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.967934  1780 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.946729  1979 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.980808  2048 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.981561  1979 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:18.985224  2048 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.985162  1847 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.974135  1908 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:18.979326  2049 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:19.012476  2048 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:19.006441  1847 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.035403  2053 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:19.029731  2049 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.073081  1753 tablet_bootstrap.cc:492] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:19.073833  1753 ts_tablet_manager.cc:1403] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.456s	user 0.025s	sys 0.057s
I20260709 12:59:19.063151  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:19.081099  1629 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.064939  2053 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.130443  1755 ts_tablet_manager.cc:1434] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.441s	user 0.016s	sys 0.026s
I20260709 12:59:19.080770  2049 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:19.162133  1755 tablet_bootstrap.cc:492] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:19.156764  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:19.174129  1780 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:19.173870  1753 raft_consensus.cc:359] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.179009  1753 raft_consensus.cc:385] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:19.179374  1753 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:19.180094  2053 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:19.160425  1908 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:19.185298  1754 tablet_bootstrap.cc:492] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:19.187417  1754 ts_tablet_manager.cc:1403] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.514s	user 0.038s	sys 0.074s
I20260709 12:59:19.187382  1753 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.185139  1629 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:19.203536  1753 ts_tablet_manager.cc:1434] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.129s	user 0.009s	sys 0.000s
I20260709 12:59:19.202456  2049 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:19.198302  1847 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:19.215833  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:19.218583  1755 tablet_bootstrap.cc:654] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.221248  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:19.222088  1629 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:19.223541  1754 raft_consensus.cc:359] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.224565  1754 raft_consensus.cc:385] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:19.224087  1979 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:19.224869  1754 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:19.231354  1754 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.229624  1753 tablet_bootstrap.cc:492] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:19.233297  1979 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:19.234361  1979 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.206252  2049 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.240144  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:19.240931  1556 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:19.245606  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:19.247195  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:19.251092  2049 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:19.251936  1707 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:19.261163  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 12:59:19.269332  1601 leader_election.cc:343] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:19.261969  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:19.265055  1847 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:19.274578  1847 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:19.275034  1847 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.273969  1754 ts_tablet_manager.cc:1434] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.086s	user 0.009s	sys 0.000s
I20260709 12:59:19.283839  1525 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:19.295269  1459 catalog_manager.cc:5697] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:19.290655  1556 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 1: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 12:59:19.307292  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:19.307762  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:19.286809  1918 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:59:19.315382  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:19.290234  1985 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:59:19.315963  1754 tablet_bootstrap.cc:492] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:19.323592  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:19.324469  1558 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:19.326144  2061 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:19.318157  1918 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:19.334905  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:19.358186  2063 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:19.339830  1918 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.361953  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:19.358776  2063 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:19.371115  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:19.392109  2063 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.383015  1556 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.402953  1753 tablet_bootstrap.cc:654] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.404053  1755 tablet_bootstrap.cc:492] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:19.404584  1755 ts_tablet_manager.cc:1403] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.263s	user 0.013s	sys 0.010s
I20260709 12:59:19.373672  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:19.375523  1847 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.415741  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:19.417302  1847 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:19.414433  1628 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.403326  1631 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:19.425168  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:19.426038  1705 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:19.426522  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:19.417359  1560 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.433887  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:19.440331  1755 raft_consensus.cc:359] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.441202  1755 raft_consensus.cc:385] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:19.441550  1755 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:19.442329  1755 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.444782  1755 ts_tablet_manager.cc:1434] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 12:59:19.447337  2070 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:19.447742  1674 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:19.451694  2055 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260709 12:59:19.450860  1601 leader_election.cc:343] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:19.455569  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:19.456691  1754 tablet_bootstrap.cc:654] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.427318  1555 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:19.465770  2061 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:19.467146  1755 tablet_bootstrap.cc:492] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:19.471495  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:19.478909  1847 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:19.467906  2070 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:19.479808  2070 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:19.479339  1847 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:19.480216  2070 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.470670  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:19.480427  1847 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.456399  1552 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
W20260709 12:59:19.483299  1601 leader_election.cc:343] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:19.480216  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:19.486764  2069 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:19.490545  2061 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:19.491380  1675 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:19.486315  1601 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:19.494012  1979 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:19.494432  1979 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:19.494792  1979 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.498781  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:19.500389  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:19.515618  2058 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.491729  2061 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.513662  1908 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:19.541510  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:19.542971  1710 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:19.534116  1560 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:19.570716  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:19.588840  1753 tablet_bootstrap.cc:492] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:19.589653  1753 ts_tablet_manager.cc:1403] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.360s	user 0.040s	sys 0.040s
I20260709 12:59:19.588793  2049 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:19.617502  2049 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:19.619347  2049 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.622161  1755 tablet_bootstrap.cc:654] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.629649  1754 tablet_bootstrap.cc:492] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:19.630201  1754 ts_tablet_manager.cc:1403] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.321s	user 0.020s	sys 0.038s
I20260709 12:59:19.612920  2063 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.635519  1847 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.639672  1753 raft_consensus.cc:359] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.640401  1753 raft_consensus.cc:385] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:19.643291  1601 leader_election.cc:343] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:19.640671  1753 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:19.638526  1847 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:19.651059  1754 raft_consensus.cc:359] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.651865  1754 raft_consensus.cc:385] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:19.652143  1754 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:19.653249  1753 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.659436  1754 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.662619  1753 ts_tablet_manager.cc:1434] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.073s	user 0.010s	sys 0.001s
I20260709 12:59:19.676966  1754 ts_tablet_manager.cc:1434] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.000s
I20260709 12:59:19.680967  1753 tablet_bootstrap.cc:492] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:19.686332  1754 tablet_bootstrap.cc:492] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:19.668171  2055 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 12:59:19.680902  1633 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 1: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 12:59:19.695147  2063 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:19.674165  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:19.675875  1918 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.693817  2055 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.746743  1753 tablet_bootstrap.cc:654] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.746958  2061 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.761826  1557 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 1: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 12:59:19.728102  1556 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:19.763850  1628 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:19.752367  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:19.698441  2070 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.766146  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:19.775758  1754 tablet_bootstrap.cc:654] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.779106  1707 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 1: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 12:59:19.780097  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45591.9us
I20260709 12:59:19.787125  1560 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 1: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 12:59:19.806686  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 76014.3us
I20260709 12:59:19.825663  2062 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:19.826433  2062 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.957594  1753 tablet_bootstrap.cc:492] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:19.958570  1753 ts_tablet_manager.cc:1403] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.281s	user 0.038s	sys 0.043s
I20260709 12:59:19.829183  2062 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:19.967052  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:19.875954  1985 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:19.967957  1780 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:19.969792  1780 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.972028  1780 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:19.971777  1753 raft_consensus.cc:359] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.972587  1753 raft_consensus.cc:385] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:19.972872  1753 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
W20260709 12:59:19.858697  2083 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.091s	user 0.001s	sys 0.003s
I20260709 12:59:19.973812  1753 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.926282  1979 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.969692  1985 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:19.977157  1753 ts_tablet_manager.cc:1434] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.002s
I20260709 12:59:19.977553  1705 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 1: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 12:59:19.978286  1753 tablet_bootstrap.cc:492] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:19.978544  1985 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:19.829766  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:19.980077  1633 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 1: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 12:59:19.871635  2082 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.098s	user 0.002s	sys 0.000s
I20260709 12:59:19.992972  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:19.836475  2058 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:19.996304  2048 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:59:19.995609  1526 leader_election.cc:343] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:19.834193  2070 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:19.856590  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:19.830307  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:19.959970  2055 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 12:59:19.949054  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 77645us
I20260709 12:59:19.977180  1979 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:20.017701  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:20.003914  1557 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.022920  1628 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 1: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 12:59:20.021215  1633 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:20.025599  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:20.030550  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:20.032341  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:20.041780  1979 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:20.042819  1753 tablet_bootstrap.cc:654] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.049804  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:20.076812  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:20.049024  1456 catalog_manager.cc:5697] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:20.079654  1456 catalog_manager.cc:5697] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
W20260709 12:59:20.089071  1601 leader_election.cc:343] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.048758  2061 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:20.103739  1755 tablet_bootstrap.cc:492] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:20.104893  1755 ts_tablet_manager.cc:1403] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.638s	user 0.012s	sys 0.016s
I20260709 12:59:20.081336  1555 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:20.105840  1709 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 1: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 12:59:20.114501  1918 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:20.103407  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:20.120752  1702 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.113312  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:20.125674  1633 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 1: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 12:59:20.127831  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:20.128764  1556 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 1: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 12:59:20.132146  1755 raft_consensus.cc:359] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.132994  1755 raft_consensus.cc:385] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.133285  1755 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.133680  1525 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:20.134001  1755 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.134594  1918 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:20.135125  1918 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:20.136334  1918 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.136734  1754 tablet_bootstrap.cc:492] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:20.138621  1754 ts_tablet_manager.cc:1403] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.453s	user 0.018s	sys 0.020s
I20260709 12:59:20.146448  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:20.149442  1526 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:20.143968  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:20.148643  1753 tablet_bootstrap.cc:492] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:20.150337  1634 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:20.150913  1753 ts_tablet_manager.cc:1403] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.173s	user 0.028s	sys 0.019s
I20260709 12:59:20.150818  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:20.151844  1629 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 1: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 12:59:20.155200  2068 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.165556  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:20.168879  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:20.171526  2051 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.174439  1702 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:20.163136  1753 raft_consensus.cc:359] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.175354  1675 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:20.175792  1753 raft_consensus.cc:385] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.176393  1753 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.177417  1755 ts_tablet_manager.cc:1434] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 12:59:20.153648  1754 raft_consensus.cc:359] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.178951  1754 raft_consensus.cc:385] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.179349  1754 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.180292  1754 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.182101  1753 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.188575  1753 ts_tablet_manager.cc:1434] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.002s
I20260709 12:59:20.189666  1753 tablet_bootstrap.cc:492] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:20.201359  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:20.201714  1755 tablet_bootstrap.cc:492] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:20.205365  1754 ts_tablet_manager.cc:1434] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 12:59:20.208983  1983 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:20.216332  1983 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:20.216363  1754 tablet_bootstrap.cc:492] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:20.218279  2050 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:20.218729  2050 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:20.219086  2050 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.219442  2070 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.229725  1753 tablet_bootstrap.cc:654] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.228479  2070 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.218309  1983 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.311775  2070 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:20.313951  1674 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:20.318169  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:20.313735  1918 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:20.326295  1601 leader_election.cc:343] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.326972  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:20.328071  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:20.329584  2054 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.329686  2058 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.330924  1754 tablet_bootstrap.cc:654] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.325783  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:20.320823  1557 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:20.331984  2055 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.359030  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:20.359824  1977 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
W20260709 12:59:20.365578  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:20.372551  1457 catalog_manager.cc:5697] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:20.371043  2063 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:59:20.376544  1675 leader_election.cc:343] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.391935  2063 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:20.396283  1918 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:20.396165  2063 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.402236  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:20.403581  1755 tablet_bootstrap.cc:654] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:20.404083  1526 leader_election.cc:343] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.410208  1753 tablet_bootstrap.cc:492] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:20.410676  1753 ts_tablet_manager.cc:1403] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.221s	user 0.034s	sys 0.035s
I20260709 12:59:20.393314  2066 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:20.404654  2070 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:59:20.412576  2050 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.419508  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:20.420208  1628 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.432999  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:20.435034  1559 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.437558  2050 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:20.454077  1753 raft_consensus.cc:359] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.455076  1753 raft_consensus.cc:385] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.455389  1753 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.456171  1753 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.469005  1754 tablet_bootstrap.cc:492] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:20.469401  1628 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:20.469656  1754 ts_tablet_manager.cc:1403] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.254s	user 0.027s	sys 0.003s
W20260709 12:59:20.464900  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 122756us
I20260709 12:59:20.472226  1525 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:20.478715  2068 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:20.479250  2068 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:20.484508  1753 ts_tablet_manager.cc:1434] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.002s
I20260709 12:59:20.485834  1753 tablet_bootstrap.cc:492] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:20.496982  1457 catalog_manager.cc:5697] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:20.479720  2110 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:59:20.480113  2068 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.505425  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:20.506129  1631 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.510250  1755 tablet_bootstrap.cc:492] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:20.510766  1755 ts_tablet_manager.cc:1403] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.318s	user 0.019s	sys 0.012s
I20260709 12:59:20.514885  1754 raft_consensus.cc:359] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.517668  1754 raft_consensus.cc:385] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.517952  1754 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.518584  1754 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.491513  1559 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:20.526081  1754 ts_tablet_manager.cc:1434] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.056s	user 0.005s	sys 0.002s
I20260709 12:59:20.532444  1754 tablet_bootstrap.cc:492] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:20.531808  1675 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:20.529112  1979 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:59:20.538398  2055 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:20.539083  2055 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:20.540078  2055 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.542719  1753 tablet_bootstrap.cc:654] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.556968  1631 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:20.550753  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:20.549346  1755 raft_consensus.cc:359] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.558369  1633 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:20.558526  1755 raft_consensus.cc:385] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.558907  1755 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.560320  1755 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.564239  1755 ts_tablet_manager.cc:1434] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.000s
I20260709 12:59:20.589133  1755 tablet_bootstrap.cc:492] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:20.595384  1754 tablet_bootstrap.cc:654] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.595013  1983 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.621420  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:20.624801  2050 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:20.619217  1983 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.641659  1983 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:20.633045  2050 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:20.647627  1755 tablet_bootstrap.cc:654] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.646620  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:20.659834  1753 tablet_bootstrap.cc:492] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:20.662420  1753 ts_tablet_manager.cc:1403] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.177s	user 0.018s	sys 0.006s
I20260709 12:59:20.651837  2050 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.668156  1707 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:20.679533  1753 raft_consensus.cc:359] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.680583  1753 raft_consensus.cc:385] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.672014  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:20.680915  1753 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.690485  1753 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.683457  2106 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:20.698433  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47194.9us
I20260709 12:59:20.699137  1753 ts_tablet_manager.cc:1434] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.002s
I20260709 12:59:20.700302  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 12:59:20.706524  1525 leader_election.cc:343] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.714316  1754 tablet_bootstrap.cc:492] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:20.715023  1754 ts_tablet_manager.cc:1403] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.184s	user 0.012s	sys 0.006s
I20260709 12:59:20.713683  1459 catalog_manager.cc:5697] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:20.719578  1753 tablet_bootstrap.cc:492] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:20.722290  1754 raft_consensus.cc:359] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.723032  1754 raft_consensus.cc:385] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.723308  1754 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.710738  2106 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.732543  2106 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:20.712961  2061 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:20.734966  2061 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:20.735368  2061 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.736311  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:20.738147  1601 leader_election.cc:343] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.754459  1754 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.785461  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:20.789268  1918 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:20.792290  2061 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.794608  2061 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 12:59:20.797993  1601 leader_election.cc:343] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.798368  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:20.804544  1754 ts_tablet_manager.cc:1434] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.089s	user 0.011s	sys 0.000s
I20260709 12:59:20.805394  2063 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.805598  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:20.806365  1754 tablet_bootstrap.cc:492] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
W20260709 12:59:20.808940  1525 leader_election.cc:343] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.811079  1753 tablet_bootstrap.cc:654] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.815451  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:20.820569  2050 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.821357  1705 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.824313  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:20.825011  1557 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.826373  1755 tablet_bootstrap.cc:492] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:20.826879  1755 ts_tablet_manager.cc:1403] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.242s	user 0.013s	sys 0.018s
I20260709 12:59:20.829861  1755 raft_consensus.cc:359] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.830595  1755 raft_consensus.cc:385] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.830914  1755 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.831692  1755 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.804195  2061 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:59:20.834337  1755 ts_tablet_manager.cc:1434] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:59:20.840608  1459 catalog_manager.cc:5697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:20.841991  2050 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:20.847096  1557 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:20.848613  2063 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:20.854355  1675 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:20.856689  1979 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:20.858125  1979 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:20.859746  1755 tablet_bootstrap.cc:492] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:20.859164  1979 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:20.914537  2120 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.915870  2055 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.968204  1705 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:20.960064  2120 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.014737  2120 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:20.966358  2055 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.028493  2055 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:21.086971  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:21.089599  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:21.090873  1983 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.091291  1753 tablet_bootstrap.cc:492] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:21.091378  1983 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:21.091850  1753 ts_tablet_manager.cc:1403] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.376s	user 0.021s	sys 0.014s
I20260709 12:59:21.083220  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:21.092351  1983 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.077328  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:21.102370  1753 raft_consensus.cc:359] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.103268  1753 raft_consensus.cc:385] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.103516  1753 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.106328  1754 tablet_bootstrap.cc:654] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.104182  1753 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:21.122368  1526 leader_election.cc:343] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.130775  1753 ts_tablet_manager.cc:1434] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.002s
W20260709 12:59:21.118155  1675 leader_election.cc:343] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.139744  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:21.141641  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:21.111864  1628 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:21.169409  1674 leader_election.cc:343] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.170197  1674 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:21.173827  1704 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:21.176488  1525 leader_election.cc:343] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.177102  1525 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:21.188391  1984 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.191076  2054 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.230710  1459 catalog_manager.cc:5697] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:21.232725  1628 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:21.233929  1753 tablet_bootstrap.cc:492] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:21.243827  1755 tablet_bootstrap.cc:654] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.248116  2130 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:21.238212  2121 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.087s	user 0.008s	sys 0.000s
I20260709 12:59:21.237319  2106 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.275231  2120 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:21.293694  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 57251.3us
W20260709 12:59:21.299731  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52899.6us
I20260709 12:59:21.297574  2106 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.299278  2055 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
W20260709 12:59:21.304788  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 50411.1us
I20260709 12:59:21.354501  1754 tablet_bootstrap.cc:492] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:21.355109  1754 ts_tablet_manager.cc:1403] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.549s	user 0.022s	sys 0.025s
I20260709 12:59:21.429917  2132 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:21.431633  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 114442us
I20260709 12:59:21.450354  2106 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:21.437649  2110 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:21.457513  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:21.455034  2110 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.483144  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:21.503108  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:21.495456  1754 raft_consensus.cc:359] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.508561  1754 raft_consensus.cc:385] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.509023  1754 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.484530  1984 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
W20260709 12:59:21.512655  1601 leader_election.cc:343] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.522035  1754 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.529400  1755 tablet_bootstrap.cc:492] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:21.530201  1755 ts_tablet_manager.cc:1403] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.672s	user 0.072s	sys 0.100s
I20260709 12:59:21.562252  1754 ts_tablet_manager.cc:1434] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.207s	user 0.010s	sys 0.000s
W20260709 12:59:21.563328  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 98469.5us
I20260709 12:59:21.611495  1755 raft_consensus.cc:359] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.612938  1755 raft_consensus.cc:385] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.613399  1755 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.615223  1755 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.622368  1753 tablet_bootstrap.cc:654] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:21.627640  1601 leader_election.cc:343] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.628832  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:21.630155  2151 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.636281  1755 ts_tablet_manager.cc:1434] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.106s	user 0.008s	sys 0.005s
I20260709 12:59:21.638062  1755 tablet_bootstrap.cc:492] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:21.651168  2110 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:21.657142  1755 tablet_bootstrap.cc:654] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.681667  2148 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:21.687340  1753 tablet_bootstrap.cc:492] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:21.688131  1753 ts_tablet_manager.cc:1403] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.454s	user 0.029s	sys 0.025s
I20260709 12:59:21.691757  1753 raft_consensus.cc:359] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.692691  1753 raft_consensus.cc:385] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.693892  1753 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.703830  1753 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.692430  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:21.701632  1755 tablet_bootstrap.cc:492] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:21.706890  1755 ts_tablet_manager.cc:1403] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.069s	user 0.009s	sys 0.011s
I20260709 12:59:21.706880  1753 ts_tablet_manager.cc:1434] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.001s
I20260709 12:59:21.708242  1753 tablet_bootstrap.cc:492] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:21.720162  1754 tablet_bootstrap.cc:492] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:21.720701  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:21.729660  1753 tablet_bootstrap.cc:654] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.733721  1457 catalog_manager.cc:5697] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:21.739358  1780 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.739925  1780 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.738637  1983 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260709 12:59:21.742797  2150 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.004s
I20260709 12:59:21.747669  2152 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:21.741382  1755 raft_consensus.cc:359] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.751094  1755 raft_consensus.cc:385] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.751399  1755 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.752097  1755 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.756245  1780 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:21.757838  1754 tablet_bootstrap.cc:654] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.758181  1755 ts_tablet_manager.cc:1434] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.001s
I20260709 12:59:21.766086  1674 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:21.772765  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:21.771695  1780 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:21.777087  1629 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.785193  1755 tablet_bootstrap.cc:492] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
W20260709 12:59:21.784106  1675 leader_election.cc:343] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.834946  1918 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.837378  1918 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.839931  1918 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:21.841049  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:21.841816  1705 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:21.843940  1526 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:21.851804  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:21.854352  1632 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:21.841049  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 12:59:21.856427  1525 leader_election.cc:343] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.858480  1918 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.861150  1918 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.863700  1674 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:21.865267  1983 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:21.865556  1918 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.866465  1780 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.866827  1780 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.867151  1780 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.885061  1753 tablet_bootstrap.cc:492] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:21.886334  1753 ts_tablet_manager.cc:1403] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.178s	user 0.019s	sys 0.010s
I20260709 12:59:21.892378  1780 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.894970  1780 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:21.905350  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 12:59:21.907886  1675 leader_election.cc:343] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.922209  1754 tablet_bootstrap.cc:492] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:21.923004  1754 ts_tablet_manager.cc:1403] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.203s	user 0.007s	sys 0.019s
I20260709 12:59:21.923898  1755 tablet_bootstrap.cc:654] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.927582  1754 raft_consensus.cc:359] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.929884  1754 raft_consensus.cc:385] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.930253  1754 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.930895  1754 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.935115  1754 ts_tablet_manager.cc:1434] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:59:21.943006  2151 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 12:59:21.943722  2151 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.947217  1753 raft_consensus.cc:359] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.948019  1753 raft_consensus.cc:385] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.948357  1753 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.950253  1754 tablet_bootstrap.cc:492] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:21.959252  1753 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.965312  1918 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:21.967840  1918 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:21.970918  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:21.971680  1704 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.970932  1753 ts_tablet_manager.cc:1434] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.084s	user 0.005s	sys 0.007s
I20260709 12:59:21.969933  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:21.974005  1627 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.974205  1753 tablet_bootstrap.cc:492] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:21.974335  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 12:59:21.976516  1525 leader_election.cc:343] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.988150  1704 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:21.990119  1526 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:21.998365  1627 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:22.012205  2153 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.016101  2151 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.013136  2153 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:22.022246  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:22.025496  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:22.029098  2153 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.030521  1674 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:22.031454  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:22.040766  2132 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.033299  2106 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:22.044947  2132 raft_consensus.cc:697] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:22.050357  2132 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.057294  1754 tablet_bootstrap.cc:654] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.080327  1780 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.080873  1780 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.082876  1755 tablet_bootstrap.cc:492] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:22.083464  1755 ts_tablet_manager.cc:1403] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.299s	user 0.008s	sys 0.021s
I20260709 12:59:22.089218  1780 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:22.093801  1753 tablet_bootstrap.cc:654] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.094467  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:22.101356  1629 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:22.104478  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:22.105696  2110 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.106932  2110 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.107293  2110 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.111640  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 12:59:22.115234  1675 leader_election.cc:343] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.125600  1456 catalog_manager.cc:5697] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.144595  1457 catalog_manager.cc:5697] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:22.150228  1755 raft_consensus.cc:359] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.152150  1755 raft_consensus.cc:385] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.152493  1755 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.151654  2106 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.164660  1755 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.163669  2106 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.167776  2106 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.170765  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:22.172322  1754 tablet_bootstrap.cc:492] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:22.172923  1754 ts_tablet_manager.cc:1403] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.223s	user 0.028s	sys 0.052s
I20260709 12:59:22.174789  1755 ts_tablet_manager.cc:1434] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.091s	user 0.006s	sys 0.002s
I20260709 12:59:22.171654  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:22.176016  2110 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:22.177146  1601 leader_election.cc:343] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:22.178105  1601 leader_election.cc:343] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.178506  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:22.179158  2106 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.181077  1755 tablet_bootstrap.cc:492] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:22.181275  1754 raft_consensus.cc:359] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.182054  1754 raft_consensus.cc:385] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.182371  1754 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.183194  1754 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.185527  1754 ts_tablet_manager.cc:1434] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:59:22.185281  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:22.186739  1754 tablet_bootstrap.cc:492] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
W20260709 12:59:22.187143  1675 leader_election.cc:343] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.257011  2110 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:22.258342  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:22.259084  1627 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.264922  1755 tablet_bootstrap.cc:654] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.282079  1755 tablet_bootstrap.cc:492] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:22.282719  1755 ts_tablet_manager.cc:1403] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.102s	user 0.016s	sys 0.014s
I20260709 12:59:22.284211  2106 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.284785  2106 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.286999  2106 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.286890  1755 raft_consensus.cc:359] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.287775  1755 raft_consensus.cc:385] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.288086  1755 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.288910  1755 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.289876  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:22.290617  1705 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:22.291472  1755 ts_tablet_manager.cc:1434] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:59:22.268215  1753 tablet_bootstrap.cc:492] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:22.292265  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:22.292585  1755 tablet_bootstrap.cc:492] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:22.293087  2151 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:22.293442  2151 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.293774  2151 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:22.292279  1753 ts_tablet_manager.cc:1403] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.318s	user 0.017s	sys 0.018s
I20260709 12:59:22.299635  1755 tablet_bootstrap.cc:654] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.299777  1753 raft_consensus.cc:359] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.300504  1753 raft_consensus.cc:385] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.300762  1753 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.290714  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:22.302366  1552 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:22.305094  1753 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.308153  2151 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.267581  1627 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:22.315017  1755 tablet_bootstrap.cc:492] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:22.317366  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:22.318084  1556 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:22.322424  2151 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.327877  1755 ts_tablet_manager.cc:1403] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.005s
I20260709 12:59:22.331511  1755 raft_consensus.cc:359] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.323199  1753 ts_tablet_manager.cc:1434] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:59:22.323558  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:22.335664  1755 raft_consensus.cc:385] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.336040  1705 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:22.318071  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:22.337034  2066 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.337145  1753 tablet_bootstrap.cc:492] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:22.337546  2066 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:22.338387  2066 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.334985  1556 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:22.323028  1754 tablet_bootstrap.cc:654] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.345865  1753 tablet_bootstrap.cc:654] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.345875  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:22.336084  1755 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.348734  2151 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:22.349385  1755 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.349356  2151 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:22.352420  2151 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.361359  1753 tablet_bootstrap.cc:492] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:22.362557  1753 ts_tablet_manager.cc:1403] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.004s
I20260709 12:59:22.366194  1753 raft_consensus.cc:359] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.367040  1753 raft_consensus.cc:385] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.367409  1753 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.368209  1753 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.370585  1753 ts_tablet_manager.cc:1434] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:59:22.361804  1755 ts_tablet_manager.cc:1434] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:59:22.372200  1755 tablet_bootstrap.cc:492] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:22.377952  1753 tablet_bootstrap.cc:492] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:22.377336  1705 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:22.392930  1456 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 2, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.388657  2153 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.397976  1754 tablet_bootstrap.cc:492] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:22.398518  1754 ts_tablet_manager.cc:1403] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.212s	user 0.009s	sys 0.009s
I20260709 12:59:22.396050  2153 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.402258  2120 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.402796  2120 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.408185  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:22.409121  1631 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 1: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 12:59:22.411105  1754 raft_consensus.cc:359] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.411818  1754 raft_consensus.cc:385] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.412082  1754 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.412907  2066 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.413405  2066 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.411839  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:22.415112  2120 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 12:59:22.416332  1525 leader_election.cc:343] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.418778  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:22.420818  1754 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.406107  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:22.422513  1674 leader_election.cc:343] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.422832  2151 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.421962  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:22.423507  1754 ts_tablet_manager.cc:1434] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:59:22.423333  2151 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.424531  1754 tablet_bootstrap.cc:492] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
W20260709 12:59:22.424611  1675 leader_election.cc:343] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.425100  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:22.427134  2151 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.406056  1457 catalog_manager.cc:5697] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:22.422510  2066 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:22.427325  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:22.432600  1601 leader_election.cc:343] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.433123  1753 tablet_bootstrap.cc:654] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.435271  1526 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:22.436612  2120 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.439401  1755 tablet_bootstrap.cc:654] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.432026  2132 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.451931  2106 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.428947  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:22.452651  2106 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.453207  1552 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:22.454941  2106 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.456902  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:22.457780  1706 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 1: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 12:59:22.459203  2120 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.465112  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:22.474937  2130 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.475349  2130 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.475741  2130 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.475435  1754 tablet_bootstrap.cc:654] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.470325  2153 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.477365  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:22.485790  1526 leader_election.cc:343] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.486382  1526 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:22.488513  2134 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.477689  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:22.493331  2153 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.476964  1918 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.496738  1918 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.522944  1627 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.523584  1918 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.530416  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:22.531637  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:22.532454  1629 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 1: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 12:59:22.516921  1754 tablet_bootstrap.cc:492] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:22.534822  1754 ts_tablet_manager.cc:1403] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.111s	user 0.020s	sys 0.016s
I20260709 12:59:22.538770  1753 tablet_bootstrap.cc:492] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:22.539327  1753 ts_tablet_manager.cc:1403] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.162s	user 0.014s	sys 0.000s
W20260709 12:59:22.541308  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 66748.4us
I20260709 12:59:22.542438  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:22.545132  1753 raft_consensus.cc:359] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.545980  1753 raft_consensus.cc:385] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.546165  2153 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.531108  2130 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.551010  2130 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.551858  1753 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.552585  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:22.553205  1753 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.555668  1753 ts_tablet_manager.cc:1434] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
W20260709 12:59:22.552606  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:22.557565  2151 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.558499  2132 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.558691  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:22.559258  1556 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.559793  2110 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.560478  2110 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.570312  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:22.571074  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:22.571703  1706 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.570595  1753 tablet_bootstrap.cc:492] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:22.578852  2110 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:22.587553  1556 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:22.589330  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:22.586308  2153 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.591135  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:22.553650  2182 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 12:59:22.592236  1627 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 1: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 12:59:22.593780  1674 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:22.594141  1754 raft_consensus.cc:359] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.594574  1706 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:22.594848  1754 raft_consensus.cc:385] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.595170  1754 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.595319  2110 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.595892  1754 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.604351  1753 tablet_bootstrap.cc:654] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.598038  2130 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.605706  2130 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:22.605955  1754 ts_tablet_manager.cc:1434] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.000s
I20260709 12:59:22.607147  1754 tablet_bootstrap.cc:492] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:22.606766  2130 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.619467  1755 tablet_bootstrap.cc:492] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:22.620321  1755 ts_tablet_manager.cc:1403] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.249s	user 0.014s	sys 0.022s
I20260709 12:59:22.626597  2106 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.628698  2106 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.639848  1755 raft_consensus.cc:359] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.640671  1755 raft_consensus.cc:385] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.640938  1755 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.643342  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:22.645324  1755 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.647064  2106 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.647969  1755 ts_tablet_manager.cc:1434] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.027s	user 0.003s	sys 0.003s
I20260709 12:59:22.648479  2110 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:59:22.651152  1601 leader_election.cc:343] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.669785  1753 tablet_bootstrap.cc:492] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:22.670454  1753 ts_tablet_manager.cc:1403] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.100s	user 0.028s	sys 0.002s
I20260709 12:59:22.674399  1755 tablet_bootstrap.cc:492] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:22.673527  1459 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.663770  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:22.679999  1601 leader_election.cc:343] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.681529  1601 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:22.683277  2106 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.709163  1755 tablet_bootstrap.cc:654] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.721705  1754 tablet_bootstrap.cc:654] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.724433  1753 raft_consensus.cc:359] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.728082  1753 raft_consensus.cc:385] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.728459  1753 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.736352  2110 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.736937  2110 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.737074  1753 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.739738  1753 ts_tablet_manager.cc:1434] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.069s	user 0.004s	sys 0.003s
I20260709 12:59:22.740796  1753 tablet_bootstrap.cc:492] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:22.743255  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:22.744536  2110 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 12:59:22.745445  1675 leader_election.cc:343] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.746259  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 12:59:22.750779  1674 leader_election.cc:343] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.751235  1674 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:22.752060  2132 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.766913  1753 tablet_bootstrap.cc:654] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.767127  1754 tablet_bootstrap.cc:492] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:22.768030  1754 ts_tablet_manager.cc:1403] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.161s	user 0.008s	sys 0.016s
I20260709 12:59:22.772291  1755 tablet_bootstrap.cc:492] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:22.773022  1552 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:59:22.773079  1755 ts_tablet_manager.cc:1403] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.108s	user 0.012s	sys 0.007s
I20260709 12:59:22.774595  2130 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.778270  1753 tablet_bootstrap.cc:492] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:22.778162  1754 raft_consensus.cc:359] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.778789  1753 ts_tablet_manager.cc:1403] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.003s
I20260709 12:59:22.778949  1754 raft_consensus.cc:385] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.779245  1754 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.780395  1755 raft_consensus.cc:359] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.784888  1755 raft_consensus.cc:385] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.785249  1755 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.786075  1755 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.788658  2110 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.788789  1754 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.783345  1753 raft_consensus.cc:359] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.789628  1753 raft_consensus.cc:385] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.789933  1753 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.790635  1753 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.794373  1754 ts_tablet_manager.cc:1434] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:59:22.795800  1754 tablet_bootstrap.cc:492] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:22.794368  2110 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:22.796506  2130 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.797021  2130 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.800297  2130 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.800566  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:22.800594  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:22.802150  1601 leader_election.cc:343] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.804430  1755 ts_tablet_manager.cc:1434] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.000s
I20260709 12:59:22.805713  1755 tablet_bootstrap.cc:492] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
W20260709 12:59:22.806382  1601 leader_election.cc:343] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.806846  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:22.807019  2192 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.807518  2130 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.807590  2192 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.811973  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:22.812676  1704 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:22.812945  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:22.813768  1753 ts_tablet_manager.cc:1434] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.001s
I20260709 12:59:22.814571  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:22.819015  2191 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.819275  1753 tablet_bootstrap.cc:492] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:22.819427  2191 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.819820  2191 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.815387  1634 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:22.815920  2192 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.832461  1754 tablet_bootstrap.cc:654] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.839872  1755 tablet_bootstrap.cc:654] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.840237  2191 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.843556  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:22.844192  1634 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.847786  1753 tablet_bootstrap.cc:654] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.848385  1754 tablet_bootstrap.cc:492] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:22.848958  1754 ts_tablet_manager.cc:1403] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.000s
I20260709 12:59:22.852725  1754 raft_consensus.cc:359] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.853696  1754 raft_consensus.cc:385] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.854101  1754 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.875325  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:22.855160  1754 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.876197  1704 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.886162  1754 ts_tablet_manager.cc:1434] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.003s
I20260709 12:59:22.888351  1704 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:22.889124  1754 tablet_bootstrap.cc:492] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:22.894811  1755 tablet_bootstrap.cc:492] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:22.895283  1755 ts_tablet_manager.cc:1403] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.090s	user 0.006s	sys 0.025s
I20260709 12:59:22.898088  1755 raft_consensus.cc:359] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.898838  1755 raft_consensus.cc:385] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.899169  1755 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.901098  1755 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.903703  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:22.904826  1755 ts_tablet_manager.cc:1434] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:59:22.905009  1918 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.905596  1918 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:22.905932  1755 tablet_bootstrap.cc:492] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:22.906442  1918 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.908475  2134 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.894026  1634 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:22.908949  2134 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.910830  1754 tablet_bootstrap.cc:654] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.911237  2134 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.902297  1706 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:59:22.916251  1753 tablet_bootstrap.cc:492] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:22.916779  1753 ts_tablet_manager.cc:1403] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.098s	user 0.024s	sys 0.006s
I20260709 12:59:22.919459  1753 raft_consensus.cc:359] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.919981  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:22.920301  1753 raft_consensus.cc:385] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.920648  1753 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.920678  1629 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:22.921371  2151 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.921696  1753 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.922185  1525 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:22.924132  1753 ts_tablet_manager.cc:1434] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:59:22.923097  2134 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.925313  2134 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.925805  2134 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.889232  2191 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:22.937080  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:22.938009  1706 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:22.939610  1753 tablet_bootstrap.cc:492] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:22.944801  2134 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.958204  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:22.960031  2133 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.960697  2133 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.964491  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:22.965261  1629 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:22.967114  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:22.968045  2110 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.968453  2110 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.968817  2110 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.965895  2133 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:22.977805  1754 tablet_bootstrap.cc:492] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:22.959893  1634 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.978430  1754 ts_tablet_manager.cc:1403] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.090s	user 0.005s	sys 0.009s
I20260709 12:59:22.980677  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:22.986052  1755 tablet_bootstrap.cc:654] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.977103  2110 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.990604  1754 raft_consensus.cc:359] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.994000  1754 raft_consensus.cc:385] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.994457  1754 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.995694  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:22.995390  1754 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:22.997418  1552 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:22.998306  1754 ts_tablet_manager.cc:1434] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.006s
I20260709 12:59:22.999503  1754 tablet_bootstrap.cc:492] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:23.002837  1753 tablet_bootstrap.cc:654] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:23.080173  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 78358.4us
I20260709 12:59:23.080832  1634 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:23.083356  1754 tablet_bootstrap.cc:654] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.084517  1755 tablet_bootstrap.cc:492] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:23.098176  1755 ts_tablet_manager.cc:1403] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.192s	user 0.021s	sys 0.005s
I20260709 12:59:23.099184  1525 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:23.100874  2182 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.101389  2182 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:23.102279  2182 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.103842  1755 raft_consensus.cc:359] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.104568  1755 raft_consensus.cc:385] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.104848  1755 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.086364  2134 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:23.087199  2110 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:22.984969  1459 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:23.090986  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:23.093801  1753 tablet_bootstrap.cc:492] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:23.106191  1629 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.106575  1753 ts_tablet_manager.cc:1403] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.167s	user 0.007s	sys 0.012s
I20260709 12:59:23.110275  1753 raft_consensus.cc:359] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.110967  1753 raft_consensus.cc:385] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.111270  1753 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.111948  1753 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.116801  1753 ts_tablet_manager.cc:1434] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:59:23.125005  1753 tablet_bootstrap.cc:492] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:23.126554  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:23.127275  1704 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.136932  1629 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:23.105492  1755 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.138701  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:23.141558  2110 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.142195  2110 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:23.143281  2110 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.144647  1704 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:23.152894  2130 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.153596  2130 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.154469  1456 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:23.163189  2130 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:23.157292  1753 tablet_bootstrap.cc:654] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.157114  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 12:59:23.167416  1601 leader_election.cc:343] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.167872  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:23.168628  1702 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:23.170027  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:23.172281  2106 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.172655  2106 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.172977  2106 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.196036  2177 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:23.156955  1755 ts_tablet_manager.cc:1434] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.000s
I20260709 12:59:23.198863  1755 tablet_bootstrap.cc:492] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:23.205158  2106 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.215615  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:23.215909  2130 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:23.217907  1601 leader_election.cc:343] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.227510  1754 tablet_bootstrap.cc:492] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:23.228163  1754 ts_tablet_manager.cc:1403] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.229s	user 0.021s	sys 0.037s
I20260709 12:59:23.231345  2106 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:23.230419  1459 catalog_manager.cc:5697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:23.241894  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:23.242678  1704 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.242285  1754 raft_consensus.cc:359] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.243165  1754 raft_consensus.cc:385] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.243458  1754 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.245054  1754 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.249102  1754 ts_tablet_manager.cc:1434] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:59:23.251425  1755 tablet_bootstrap.cc:654] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.256084  1754 tablet_bootstrap.cc:492] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:23.264431  1754 tablet_bootstrap.cc:654] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.268213  1753 tablet_bootstrap.cc:492] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:23.276175  1753 ts_tablet_manager.cc:1403] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.151s	user 0.011s	sys 0.018s
I20260709 12:59:23.275683  1704 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:23.282939  1755 tablet_bootstrap.cc:492] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:23.286070  1755 ts_tablet_manager.cc:1403] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.087s	user 0.018s	sys 0.004s
I20260709 12:59:23.287616  1753 raft_consensus.cc:359] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.288595  1753 raft_consensus.cc:385] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.288923  1753 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.289000  1755 raft_consensus.cc:359] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.289913  1753 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.292426  1753 ts_tablet_manager.cc:1434] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.005s
I20260709 12:59:23.293635  1755 raft_consensus.cc:385] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.293455  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:23.294121  1755 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.294903  2130 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.294866  1755 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.296741  2130 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:23.297447  1755 ts_tablet_manager.cc:1434] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:59:23.298622  1755 tablet_bootstrap.cc:492] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:23.298054  2130 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.293447  1753 tablet_bootstrap.cc:492] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:23.307346  1755 tablet_bootstrap.cc:654] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.313042  1754 tablet_bootstrap.cc:492] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:23.313619  1754 ts_tablet_manager.cc:1403] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.016s
I20260709 12:59:23.316443  1754 raft_consensus.cc:359] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.317133  1754 raft_consensus.cc:385] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.317391  1754 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.318122  1754 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.320307  1754 ts_tablet_manager.cc:1434] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.003s
I20260709 12:59:23.321539  1754 tablet_bootstrap.cc:492] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:23.321722  1753 tablet_bootstrap.cc:654] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.330477  2130 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.330379  1457 catalog_manager.cc:5697] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:23.331065  2130 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.331655  1755 tablet_bootstrap.cc:492] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:23.332122  1755 ts_tablet_manager.cc:1403] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.011s
I20260709 12:59:23.333287  2130 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:23.344663  1753 tablet_bootstrap.cc:492] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:23.344807  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:23.345352  1753 ts_tablet_manager.cc:1403] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.008s
I20260709 12:59:23.345579  1552 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:23.345852  1754 tablet_bootstrap.cc:654] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.348456  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:23.358256  1702 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:23.360476  1755 raft_consensus.cc:359] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.362389  1755 raft_consensus.cc:385] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.362632  1755 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.355028  1753 raft_consensus.cc:359] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.364084  1753 raft_consensus.cc:385] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.364359  1753 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.346633  2209 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.365000  1753 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.365633  2209 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.367640  1753 ts_tablet_manager.cc:1434] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 12:59:23.354666  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:23.368894  1753 tablet_bootstrap.cc:492] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:23.369951  1754 tablet_bootstrap.cc:492] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:23.369757  1755 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.370376  1754 ts_tablet_manager.cc:1403] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.000s
I20260709 12:59:23.370407  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:23.371124  2151 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.371521  2151 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.371851  2151 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.373355  1754 raft_consensus.cc:359] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.374179  1754 raft_consensus.cc:385] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.374428  1754 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.375888  1754 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.377745  1753 tablet_bootstrap.cc:654] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.378122  1754 ts_tablet_manager.cc:1434] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:59:23.382082  1754 tablet_bootstrap.cc:492] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:23.371133  1629 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 0.
I20260709 12:59:23.379427  1755 ts_tablet_manager.cc:1434] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.003s
I20260709 12:59:23.385673  1755 tablet_bootstrap.cc:492] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:23.386507  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:23.387375  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:23.388428  1983 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:59:23.388751  1526 leader_election.cc:343] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.388827  1983 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.389454  1983 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.368125  2209 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:23.394167  1754 tablet_bootstrap.cc:654] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.396291  2066 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.396885  2066 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.400337  2066 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:23.401542  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:23.402274  1629 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:23.403623  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:23.403900  1755 tablet_bootstrap.cc:654] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.405402  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:23.405783  2066 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.407084  2151 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:23.407904  1675 leader_election.cc:343] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.415272  2209 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:23.420872  1755 tablet_bootstrap.cc:492] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:23.421631  1755 ts_tablet_manager.cc:1403] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.004s
I20260709 12:59:23.424537  1755 raft_consensus.cc:359] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.425364  1755 raft_consensus.cc:385] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.425729  1755 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.427392  1755 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.429280  1755 ts_tablet_manager.cc:1434] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:59:23.412599  2066 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.431610  2066 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.432333  1755 tablet_bootstrap.cc:492] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:23.441574  1754 tablet_bootstrap.cc:492] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:23.442379  1754 ts_tablet_manager.cc:1403] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.061s	user 0.015s	sys 0.003s
I20260709 12:59:23.446750  1753 tablet_bootstrap.cc:492] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:23.447378  1753 ts_tablet_manager.cc:1403] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.079s	user 0.005s	sys 0.008s
I20260709 12:59:23.450908  1755 tablet_bootstrap.cc:654] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.453898  2066 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.455307  1754 raft_consensus.cc:359] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.457111  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:23.463045  1754 raft_consensus.cc:385] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.463178  1552 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.463513  1754 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.464658  1754 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.467245  1754 ts_tablet_manager.cc:1434] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 12:59:23.414297  1983 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.458638  2151 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:23.461532  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:23.471745  1983 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:23.472335  1702 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.473343  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:23.474273  1629 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.479269  1702 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:23.481945  1552 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:23.482920  2066 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:23.461462  1753 raft_consensus.cc:359] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.484138  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:23.485060  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:23.485643  1631 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.487530  1983 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.488950  1753 raft_consensus.cc:385] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.489328  1753 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.484725  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:23.498780  1753 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:23.500243  1526 leader_election.cc:343] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.497958  1754 tablet_bootstrap.cc:492] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:23.501928  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:23.504261  1552 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.508590  1631 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:23.503154  1753 ts_tablet_manager.cc:1434] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.055s	user 0.010s	sys 0.000s
I20260709 12:59:23.510574  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:23.510941  1753 tablet_bootstrap.cc:492] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:23.503561  2151 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.512126  2133 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.512665  2133 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:23.512832  2151 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:23.513932  2133 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.519395  2151 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.529361  2066 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.542492  2066 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.559073  2066 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:23.561345  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:23.559216  2132 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.562865  1560 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:23.563040  2132 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.566092  1675 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:23.566430  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:23.567088  1627 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:23.567139  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:23.568683  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:23.568903  1556 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 0.
I20260709 12:59:23.565348  2132 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:23.569351  1674 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:23.569742  2132 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.570133  2132 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.570225  2110 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.570473  2132 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.570758  2110 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:59:23.571871  1674 leader_election.cc:343] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.571187  2110 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.589497  2130 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.599953  2130 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.669070  1755 tablet_bootstrap.cc:492] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:23.670038  1755 ts_tablet_manager.cc:1403] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.239s	user 0.051s	sys 0.044s
I20260709 12:59:23.671015  2130 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:23.673256  1753 tablet_bootstrap.cc:654] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.674270  1755 raft_consensus.cc:359] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.675420  1755 raft_consensus.cc:385] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.676051  1755 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.679737  1552 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:23.689625  1755 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.692037  1754 tablet_bootstrap.cc:654] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.692090  1755 ts_tablet_manager.cc:1434] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:59:23.690366  1629 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 12:59:23.728874  1755 tablet_bootstrap.cc:492] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Bootstrap starting.
I20260709 12:59:23.772500  1753 tablet_bootstrap.cc:492] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:23.773228  1753 ts_tablet_manager.cc:1403] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.263s	user 0.016s	sys 0.007s
I20260709 12:59:23.760583  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 12:59:23.790273  2228 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:59:23.785081  1552 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:23.815505  2132 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.817680  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:23.819120  2177 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.818511  1459 catalog_manager.cc:5697] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:23.819518  2177 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.819866  2177 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.784384  2110 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.775748  2238 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 12:59:23.842672  1753 raft_consensus.cc:359] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.843637  1753 raft_consensus.cc:385] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.843963  1753 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.844700  1753 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.854652  1753 ts_tablet_manager.cc:1434] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.079s	user 0.005s	sys 0.003s
I20260709 12:59:23.860502  1556 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.861445  1456 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:23.863209  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 143698us
I20260709 12:59:23.864213  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:23.862174  2132 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:23.867569  1753 tablet_bootstrap.cc:492] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: Bootstrap starting.
I20260709 12:59:23.868213  2153 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.884413  2235 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:23.887885  2153 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:23.888286  1755 tablet_bootstrap.cc:654] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.870851  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:23.894263  1704 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:23.900151  2110 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:23.899271  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:23.902194  1754 tablet_bootstrap.cc:492] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:23.902951  1754 ts_tablet_manager.cc:1403] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.405s	user 0.012s	sys 0.013s
I20260709 12:59:23.912034  1553 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.898190  2153 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.940569  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:23.941289  1556 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.948122  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:23.948841  1633 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:23.958093  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:23.975504  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:23.976649  1635 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:23.984041  1601 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:23.983971  1754 raft_consensus.cc:359] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.984839  1754 raft_consensus.cc:385] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.985139  1754 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.987011  2132 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.990028  2132 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.990819  2177 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:23.993811  1754 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.030242  2177 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:24.037803  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:24.038836  1554 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.057013  2132 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:24.077608  1553 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:24.079342  1754 ts_tablet_manager.cc:1434] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.176s	user 0.014s	sys 0.013s
I20260709 12:59:24.079571  1755 tablet_bootstrap.cc:492] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: No bootstrap required, opened a new log
I20260709 12:59:24.080088  1755 ts_tablet_manager.cc:1403] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Time spent bootstrapping tablet: real 0.377s	user 0.021s	sys 0.031s
I20260709 12:59:24.080411  1753 tablet_bootstrap.cc:654] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.080690  1754 tablet_bootstrap.cc:492] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1: Bootstrap starting.
I20260709 12:59:24.080951  1556 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:24.085598  1755 raft_consensus.cc:359] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.086637  1755 raft_consensus.cc:385] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.086956  1755 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.087504  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:24.089031  1755 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.091266  1755 ts_tablet_manager.cc:1434] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:59:24.103083  2066 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:24.103694  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:24.104507  1636 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 1: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 12:59:24.123209  1675 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:24.132851  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:24.147145  1635 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:24.136315  2246 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.138926  2110 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.165230  1675 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:24.176895  2237 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.179318  2246 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:24.184262  2110 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:24.250824  1918 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.251582  1753 tablet_bootstrap.cc:492] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: No bootstrap required, opened a new log
I20260709 12:59:24.251991  1753 ts_tablet_manager.cc:1403] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent bootstrapping tablet: real 0.386s	user 0.022s	sys 0.027s
I20260709 12:59:24.233937  2231 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.247922  2232 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:24.237300  2151 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:59:24.265424  1633 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 12:59:24.269971  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:24.258189  2231 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.284309  2110 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.263310  2246 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:24.310622  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 80526.2us
I20260709 12:59:24.303736  1754 tablet_bootstrap.cc:654] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.311779  1704 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.345952  2238 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.346622  2238 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.349138  2238 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:24.298874  2243 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
W20260709 12:59:24.362164  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 111046us
W20260709 12:59:24.368844  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55923.6us
I20260709 12:59:24.385654  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:24.428368  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:24.438721  2231 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:24.466573  1753 raft_consensus.cc:359] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.467716  1753 raft_consensus.cc:385] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.468066  1753 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.468987  1753 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.459360  1554 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:24.551007  1754 tablet_bootstrap.cc:492] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1: No bootstrap required, opened a new log
I20260709 12:59:24.551568  1754 ts_tablet_manager.cc:1403] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1: Time spent bootstrapping tablet: real 0.471s	user 0.019s	sys 0.021s
I20260709 12:59:24.555130  1753 ts_tablet_manager.cc:1434] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent starting tablet: real 0.303s	user 0.011s	sys 0.000s
W20260709 12:59:24.566975  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 149145us
I20260709 12:59:24.572584  2208 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.572351  1456 catalog_manager.cc:5697] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:24.575171  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:24.578812  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:24.585973  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 107142us
I20260709 12:59:24.552582  2237 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:24.579792  2261 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.592418  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:24.594269  1557 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.579790  1705 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
I20260709 12:59:24.595674  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:24.605669  2261 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:24.607116  2261 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.587136  1704 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:24.609189  2231 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
W20260709 12:59:24.667701  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 56117.9us
W20260709 12:59:24.669380  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56825.4us
I20260709 12:59:24.678905  1754 raft_consensus.cc:359] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.679708  1754 raft_consensus.cc:385] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.680002  1754 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.680567  1918 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.680697  1754 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.685314  1754 ts_tablet_manager.cc:1434] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1: Time spent starting tablet: real 0.133s	user 0.004s	sys 0.004s
I20260709 12:59:24.688591  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:24.706652  2262 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.726379  2262 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:59:24.733527  2270 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:59:24.733947  2262 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.734596  1554 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 0.
W20260709 12:59:24.746299  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 68160.9us
W20260709 12:59:24.787568  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59974.5us
I20260709 12:59:24.786675  1983 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:24.793531  2280 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:24.815797  2262 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:24.832137  2243 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 12:59:24.884873  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:24.886473  1553 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.902153  2066 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:24.935278  2284 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.098s	user 0.000s	sys 0.002s
I20260709 12:59:24.937600  2262 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:24.926010  2066 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:59:24.939157  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:25.003629  2239 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:59:25.021274  1553 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:25.014359  1702 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.030164  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 12:59:25.068038  2289 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.098s	user 0.001s	sys 0.001s
I20260709 12:59:25.081986  2231 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:59:25.122992  1457 catalog_manager.cc:5697] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:25.124627  1457 catalog_manager.cc:5697] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
W20260709 12:59:25.128795  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 45346.8us
I20260709 12:59:25.130347  2282 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:25.135447  2282 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 12:59:25.160869  2292 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.148s	user 0.003s	sys 0.000s
I20260709 12:59:25.180387  2282 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:25.193037  1702 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 12:59:25.210536  1459 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:25.260093  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 74098.6us
W20260709 12:59:25.262835  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 76796.4us
W20260709 12:59:25.272094  1507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:25.268594  2262 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:59:25.373678  1459 catalog_manager.cc:5697] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 0 to 1, leader changed from <none> to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:25.695326  2263 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:25.698382  2281 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:26.059772   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:26.096338  2310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:26.096930  2307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:26.101622  2308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:26.102219   431 server_base.cc:1061] running on GCE node
I20260709 12:59:26.105939   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:26.106245   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:26.106470   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601966106451 us; error 0 us; skew 500 ppm
I20260709 12:59:26.107441   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:26.111871   431 webserver.cc:533] Webserver started at http://127.0.107.196:34337/ using document root <none> and password file <none>
I20260709 12:59:26.112547   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:26.112843   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:26.113840   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:26.115952   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-3-root/instance:
uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
format_stamp: "Formatted at 2026-07-09 12:59:26 on dist-test-slave-l8g5"
I20260709 12:59:26.124382   431 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.006s	sys 0.003s
I20260709 12:59:26.129258  2315 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:26.130645   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:59:26.131086   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-3-root
uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
format_stamp: "Formatted at 2026-07-09 12:59:26 on dist-test-slave-l8g5"
I20260709 12:59:26.131609   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:26.150975   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:26.153120   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:26.155588   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:26.158850   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:26.159106   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:26.159471   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:26.159927   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:26.345922   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:36257
I20260709 12:59:26.350853  2377 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:36257 every 8 connection(s)
I20260709 12:59:26.364542   431 auto_leader_rebalancer-test.cc:384] add a tserver: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:26.392972  2378 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43857
I20260709 12:59:26.393592  2378 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:26.394596  2378 heartbeater.cc:507] Master 127.0.107.254:43857 requested a full tablet report, sending...
I20260709 12:59:26.397205  1459 ts_manager.cc:194] Registered new tserver with Master: 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 12:59:26.399013  1459 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42922
I20260709 12:59:27.402961  2378 heartbeater.cc:499] Master 127.0.107.254:43857 was elected leader, sending a full tablet report...
I20260709 12:59:27.475610  1553 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:27.484577  1628 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:27.487426  1706 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:27.488806  2388 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:27.490720  1983 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:27.502461  1983 raft_consensus.cc:2955] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:27.516265  1630 raft_consensus.cc:2955] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 12:59:27.523200  1527 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 946eea23a2334d678895281941dd9926. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:27.533697  1706 raft_consensus.cc:2955] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:27.541848  1459 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:27.568928  1632 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:27.584136  1706 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:27.584781  1553 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:27.586040  2282 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:27.590950  2282 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:27.612396  2282 raft_consensus.cc:2955] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:27.615181  1703 raft_consensus.cc:2955] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 12:59:27.623672  1602 consensus_peers.cc:597] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0835fee76231431da297ec1cf367640d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:27.634397  1553 raft_consensus.cc:2955] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:27.645601  1561 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:27.657461  1457 catalog_manager.cc:5697] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:27.667629  1706 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:27.669060  1632 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:27.670465  1527 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 169e9f6ed9684060b36e93c778925461. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:27.671270  2393 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:27.677591  2388 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:27.691427  2238 raft_consensus.cc:2955] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:27.694787  1703 raft_consensus.cc:2955] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:27.693595  1632 raft_consensus.cc:2955] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:27.719053  1459 catalog_manager.cc:5697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:27.735025  1703 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:27.744948  1561 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:27.746986  2386 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:27.748723  1632 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:27.751194  2386 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:27.763182  2229 raft_consensus.cc:2955] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 12:59:27.770303  1676 consensus_peers.cc:597] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b5838267321348eb83df088988da6449. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:27.775604  1553 raft_consensus.cc:2955] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:27.800462  1457 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:27.830798  1630 raft_consensus.cc:2955] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:28.017987  2414 ts_tablet_manager.cc:933] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:28.022297  2414 tablet_copy_client.cc:323] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 12:59:28.035511  1571 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet 946eea23a2334d678895281941dd9926 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 12:59:28.036199  1571 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet 946eea23a2334d678895281941dd9926 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-946eea23a2334d678895281941dd9926
I20260709 12:59:28.045608  1571 tablet_copy_source_session.cc:215] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:28.050169  2414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 946eea23a2334d678895281941dd9926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.067173  2414 tablet_copy_client.cc:806] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:28.067960  2414 tablet_copy_client.cc:670] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:28.072634  2414 tablet_copy_client.cc:538] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:28.081785  2414 tablet_bootstrap.cc:492] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:28.114846  2414 tablet_bootstrap.cc:492] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:28.116052  2419 ts_tablet_manager.cc:933] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:28.117985  2414 tablet_bootstrap.cc:492] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:28.119346  2414 ts_tablet_manager.cc:1403] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.010s
I20260709 12:59:28.123638  2414 raft_consensus.cc:359] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:28.128551  2419 tablet_copy_client.cc:323] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 12:59:28.124782  2414 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:28.134528  2414 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:28.136595  1571 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet 169e9f6ed9684060b36e93c778925461 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 12:59:28.137279  1571 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet 169e9f6ed9684060b36e93c778925461 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-169e9f6ed9684060b36e93c778925461
I20260709 12:59:28.152025  1571 tablet_copy_source_session.cc:215] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:28.155824  2419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 169e9f6ed9684060b36e93c778925461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.155985  2414 ts_tablet_manager.cc:1434] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.017s
I20260709 12:59:28.161378  1571 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-946eea23a2334d678895281941dd9926 received from {username='slave'} at 127.0.0.1:42562
I20260709 12:59:28.162024  1571 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-946eea23a2334d678895281941dd9926 on tablet 946eea23a2334d678895281941dd9926 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:28.184253  2419 tablet_copy_client.cc:806] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:28.184930  2419 tablet_copy_client.cc:670] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:28.189302  2419 tablet_copy_client.cc:538] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:28.199621  2419 tablet_bootstrap.cc:492] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:28.216128  2414 ts_tablet_manager.cc:933] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:28.238694  2424 ts_tablet_manager.cc:933] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:28.242969  2414 tablet_copy_client.cc:323] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45431
I20260709 12:59:28.249879  2424 tablet_copy_client.cc:323] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
I20260709 12:59:28.253921  2419 tablet_bootstrap.cc:492] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:28.262706  2419 tablet_bootstrap.cc:492] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:28.263742  2419 ts_tablet_manager.cc:1403] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.064s	user 0.037s	sys 0.008s
I20260709 12:59:28.270493  2419 raft_consensus.cc:359] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:28.277802  2419 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:28.281420  2419 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:28.297385  2419 ts_tablet_manager.cc:1434] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.004s
I20260709 12:59:28.302318  1720 tablet_copy_service.cc:140] P a2ba7fc60f9641cdac25f1e016ef91d8: Received BeginTabletCopySession request for tablet b5838267321348eb83df088988da6449 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:52646)
I20260709 12:59:28.303112  1720 tablet_copy_service.cc:161] P a2ba7fc60f9641cdac25f1e016ef91d8: Beginning new tablet copy session on tablet b5838267321348eb83df088988da6449 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:52646: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-b5838267321348eb83df088988da6449
I20260709 12:59:28.309335  1571 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-169e9f6ed9684060b36e93c778925461 received from {username='slave'} at 127.0.0.1:42562
I20260709 12:59:28.309906  1571 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-169e9f6ed9684060b36e93c778925461 on tablet 169e9f6ed9684060b36e93c778925461 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:28.317162  1720 tablet_copy_source_session.cc:215] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:28.317137  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet 0835fee76231431da297ec1cf367640d from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 12:59:28.317977  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet 0835fee76231431da297ec1cf367640d from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-0835fee76231431da297ec1cf367640d
I20260709 12:59:28.323994  1646 tablet_copy_source_session.cc:215] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:28.327417  2424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0835fee76231431da297ec1cf367640d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.328128  2414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5838267321348eb83df088988da6449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.353214  2414 tablet_copy_client.cc:806] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:28.354241  2414 tablet_copy_client.cc:670] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:28.365221  2414 tablet_copy_client.cc:538] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:28.378516  2424 tablet_copy_client.cc:806] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:28.380149  2424 tablet_copy_client.cc:670] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:28.383132  2414 tablet_bootstrap.cc:492] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:28.398034  2424 tablet_copy_client.cc:538] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:28.410629  2424 tablet_bootstrap.cc:492] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:28.427312  2414 tablet_bootstrap.cc:492] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:28.428484  2414 tablet_bootstrap.cc:492] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:28.429289  2414 ts_tablet_manager.cc:1403] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.019s
I20260709 12:59:28.433696  2414 raft_consensus.cc:359] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:28.434692  2414 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:28.435447  2414 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:28.443910  2414 ts_tablet_manager.cc:1434] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:59:28.447999  1720 tablet_copy_service.cc:342] P a2ba7fc60f9641cdac25f1e016ef91d8: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-b5838267321348eb83df088988da6449 received from {username='slave'} at 127.0.0.1:52646
I20260709 12:59:28.448549  1720 tablet_copy_service.cc:434] P a2ba7fc60f9641cdac25f1e016ef91d8: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-b5838267321348eb83df088988da6449 on tablet b5838267321348eb83df088988da6449 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:28.453408  2424 tablet_bootstrap.cc:492] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:28.454546  2424 tablet_bootstrap.cc:492] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:28.455380  2424 ts_tablet_manager.cc:1403] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.045s	user 0.026s	sys 0.004s
I20260709 12:59:28.458287  2424 raft_consensus.cc:359] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:28.459020  2424 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:28.459651  2424 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:28.463049  2424 ts_tablet_manager.cc:1434] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:59:28.465114  1646 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-0835fee76231431da297ec1cf367640d received from {username='slave'} at 127.0.0.1:48246
I20260709 12:59:28.465714  1646 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-0835fee76231431da297ec1cf367640d on tablet 0835fee76231431da297ec1cf367640d with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:28.520623  2353 raft_consensus.cc:1217] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:28.610368  2353 raft_consensus.cc:1217] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:28.641521  2353 raft_consensus.cc:1217] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:28.788172  2353 raft_consensus.cc:1217] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:29.008297  2393 raft_consensus.cc:1064] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:29.011629  2393 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:29.019795  2353 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.020184  1703 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.021415  1630 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.021847  1983 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:29.023008  2432 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:29.027522  2432 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:29.033279  2282 raft_consensus.cc:1064] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:29.039254  2282 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:29.078413  1706 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.081278  2231 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:29.081801  1559 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.084013  2352 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.083554  2393 raft_consensus.cc:2955] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.087023  2231 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:29.088105  1706 raft_consensus.cc:2955] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.089082  2231 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:29.091480  2432 raft_consensus.cc:1064] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:29.094367  2432 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:29.202812  2433 raft_consensus.cc:2955] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.203855  2353 raft_consensus.cc:2955] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.225346  1456 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.235100  2429 raft_consensus.cc:1064] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:29.253961  1710 raft_consensus.cc:2955] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.260094  1559 raft_consensus.cc:2955] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.254447  2429 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:29.267696  1630 raft_consensus.cc:2955] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.301957  1635 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.302026  1703 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.304935  2441 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:29.308562  2393 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:29.308794  2352 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.319764  2353 raft_consensus.cc:2955] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.324122  1559 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:29.332046  2393 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:29.347543  1635 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.349179  2448 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:29.357131  2351 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:29.361379  2446 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:29.372890  2447 raft_consensus.cc:2955] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.374980  2351 raft_consensus.cc:2955] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.384007  1561 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.386113  1456 catalog_manager.cc:5697] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.405468  2413 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:59:29.408064  2350 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:29.418917  2382 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:59:29.420200  1636 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:29.455585  2445 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:59:29.496387  1630 raft_consensus.cc:2955] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.500208  1458 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.527621  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 946eea23a2334d678895281941dd9926 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:59:29.615988  1445 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 92341.2us
I20260709 12:59:29.618109  1636 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:29.620911  2446 raft_consensus.cc:2955] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 12:59:29.645205  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 110998us
W20260709 12:59:29.623395  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 93494.4us
I20260709 12:59:29.683844  2351 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:29.685946  2231 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:29.704735  2433 raft_consensus.cc:2955] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.712062  1632 raft_consensus.cc:2955] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.721211  2351 raft_consensus.cc:2955] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.739352  2461 raft_consensus.cc:2955] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.767342  1694 tablet_service.cc:1558] Processing DeleteTablet for tablet 946eea23a2334d678895281941dd9926 with delete_type TABLET_DATA_TOMBSTONED (TS a2ba7fc60f9641cdac25f1e016ef91d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:29.803578  2352 raft_consensus.cc:2955] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.816500  1703 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:29.818984  1459 catalog_manager.cc:5697] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.839277  1632 raft_consensus.cc:2955] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.864483  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0835fee76231431da297ec1cf367640d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:59:29.867157  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 119614us
I20260709 12:59:29.874047  1704 raft_consensus.cc:2955] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.882340  2351 raft_consensus.cc:2955] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:29.906383  2474 tablet_replica.cc:333] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 12:59:29.921046  2474 raft_consensus.cc:2243] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:29.934465  1457 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:59:29.947414  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 46434.2us
I20260709 12:59:29.949604  2231 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:59:29.922564  1542 tablet_service.cc:1558] Processing DeleteTablet for tablet 0835fee76231431da297ec1cf367640d with delete_type TABLET_DATA_TOMBSTONED (TS d62ce354d09a4be59c29094425d09bc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47096
I20260709 12:59:29.965065  2474 raft_consensus.cc:2272] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:30.012144  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 68015.2us
I20260709 12:59:30.015039  2474 ts_tablet_manager.cc:1916] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:30.075166  1458 catalog_manager.cc:5697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:59:30.103777  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 83713.6us
W20260709 12:59:30.117658  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 153366us
I20260709 12:59:30.112741  1701 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:30.119881  2478 tablet_replica.cc:333] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 12:59:30.121033  1709 raft_consensus.cc:2955] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:30.148298  2478 raft_consensus.cc:2243] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:30.149132  2478 raft_consensus.cc:2272] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:30.159466  2478 ts_tablet_manager.cc:1916] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:30.173836  1560 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:30.180853  2351 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:30.181322  1636 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:30.182936  2461 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:30.193152  2472 raft_consensus.cc:2955] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:30.200245  2461 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:59:30.211241  1632 raft_consensus.cc:2955] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:30.224066  2352 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:30.223268  2351 raft_consensus.cc:2955] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:30.226091  2446 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:30.239907  1710 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:30.245468  2470 raft_consensus.cc:2955] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:30.250769  2450 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:30.255059  1456 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:30.261729  1701 raft_consensus.cc:2955] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:30.271853  2474 ts_tablet_manager.cc:1929] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:30.273723  2474 log.cc:1199] T 946eea23a2334d678895281941dd9926 P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/wals/946eea23a2334d678895281941dd9926
I20260709 12:59:30.274698  2351 raft_consensus.cc:2955] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:30.300323  1445 catalog_manager.cc:5028] TS a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): tablet 946eea23a2334d678895281941dd9926 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:30.313807  1542 tablet_service.cc:1558] Processing DeleteTablet for tablet b5838267321348eb83df088988da6449 with delete_type TABLET_DATA_TOMBSTONED (TS d62ce354d09a4be59c29094425d09bc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47096
I20260709 12:59:30.314567  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b5838267321348eb83df088988da6449 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:30.332773  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 169e9f6ed9684060b36e93c778925461 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:30.346554  1458 catalog_manager.cc:5697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:30.358160  2478 ts_tablet_manager.cc:1929] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:30.359807  2478 log.cc:1199] T 0835fee76231431da297ec1cf367640d P d62ce354d09a4be59c29094425d09bc1: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/wals/0835fee76231431da297ec1cf367640d
I20260709 12:59:30.363013  1445 catalog_manager.cc:5028] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): tablet 0835fee76231431da297ec1cf367640d (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:30.363600  2478 tablet_replica.cc:333] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 12:59:30.365773  2478 raft_consensus.cc:2243] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:30.366477  2478 pending_rounds.cc:70] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1: Trying to abort 1 pending ops.
I20260709 12:59:30.366837  2478 pending_rounds.cc:77] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305633665992527872 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "b5838267321348eb83df088988da6449" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } } }
I20260709 12:59:30.368541  2478 raft_consensus.cc:2917] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 12:59:30.369052  2478 raft_consensus.cc:2889] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:59:30.369519  2478 raft_consensus.cc:2272] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:30.370600  1621 tablet_service.cc:1558] Processing DeleteTablet for tablet 169e9f6ed9684060b36e93c778925461 with delete_type TABLET_DATA_TOMBSTONED (TS ba0548b7d2a94e8598475a46c279fe0e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54730
I20260709 12:59:30.379380  2478 ts_tablet_manager.cc:1916] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:30.382616  2482 tablet_replica.cc:333] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 12:59:30.383982  2482 raft_consensus.cc:2243] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:30.384811  2482 raft_consensus.cc:2272] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:30.388953  2482 ts_tablet_manager.cc:1916] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:30.399392  2478 ts_tablet_manager.cc:1929] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:30.399916  2478 log.cc:1199] T b5838267321348eb83df088988da6449 P d62ce354d09a4be59c29094425d09bc1: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/wals/b5838267321348eb83df088988da6449
I20260709 12:59:30.402000  1445 catalog_manager.cc:5028] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): tablet b5838267321348eb83df088988da6449 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:30.410895  2482 ts_tablet_manager.cc:1929] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:30.411511  2482 log.cc:1199] T 169e9f6ed9684060b36e93c778925461 P ba0548b7d2a94e8598475a46c279fe0e: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/wals/169e9f6ed9684060b36e93c778925461
I20260709 12:59:30.413327  1444 catalog_manager.cc:5028] TS ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): tablet 169e9f6ed9684060b36e93c778925461 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:31.442900  1557 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:31.449590  1710 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:31.450029  1527 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d72a925fb22046e2be514a7913c613a4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:31.450979  2450 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.453054  1629 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:31.455080  2450 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.459414  2485 raft_consensus.cc:2955] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.462426  1710 raft_consensus.cc:2955] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.476565  1557 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:31.490262  1636 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:31.490720  1527 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b022b45a9b7462bb4b1a2a30ae36aa9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:31.491995  2485 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.492662  1709 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:31.494310  2470 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.487247  1458 catalog_manager.cc:5697] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.500658  1629 raft_consensus.cc:2955] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.523383  2470 raft_consensus.cc:2955] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.527021  1635 raft_consensus.cc:2955] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.528805  1710 raft_consensus.cc:2955] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.554579  1710 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 12:59:31.571369  1676 consensus_peers.cc:597] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac931d9355284a168be64d7debd009ae. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:31.569178  1636 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:31.573520  1456 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.574882  1557 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:31.576402  2429 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.578017  2497 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.604743  2413 raft_consensus.cc:2955] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.606575  1560 raft_consensus.cc:2955] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.624655  1710 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:31.625906  1636 raft_consensus.cc:2955] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.632119  1456 catalog_manager.cc:5697] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:31.635538  1629 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:31.637624  1676 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0adeaf85877a4103bbd070927413d267. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:31.640683  1557 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:31.643080  2413 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:31.644189  2493 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.664435  2493 raft_consensus.cc:2955] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.670102  1557 raft_consensus.cc:2955] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.723930  1629 raft_consensus.cc:2955] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.736841  1457 catalog_manager.cc:5697] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:31.901108  2504 ts_tablet_manager.cc:933] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:31.903139  2504 tablet_copy_client.cc:323] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 12:59:31.904942  1571 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet 8b022b45a9b7462bb4b1a2a30ae36aa9 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 12:59:31.905431  1571 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet 8b022b45a9b7462bb4b1a2a30ae36aa9 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-8b022b45a9b7462bb4b1a2a30ae36aa9
I20260709 12:59:31.911976  1571 tablet_copy_source_session.cc:215] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:31.918205  2504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b022b45a9b7462bb4b1a2a30ae36aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:31.934465  2504 tablet_copy_client.cc:806] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:31.935268  2504 tablet_copy_client.cc:670] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:31.939860  2504 tablet_copy_client.cc:538] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:31.947145  2504 tablet_bootstrap.cc:492] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:31.968245  2504 tablet_bootstrap.cc:492] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:31.969336  2504 tablet_bootstrap.cc:492] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:31.970268  2504 ts_tablet_manager.cc:1403] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.004s
I20260709 12:59:31.973618  2504 raft_consensus.cc:359] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:31.974673  2504 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:31.975463  2504 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:31.982014  2504 ts_tablet_manager.cc:1434] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 12:59:31.985817  1571 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-8b022b45a9b7462bb4b1a2a30ae36aa9 received from {username='slave'} at 127.0.0.1:42562
I20260709 12:59:31.986377  1571 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-8b022b45a9b7462bb4b1a2a30ae36aa9 on tablet 8b022b45a9b7462bb4b1a2a30ae36aa9 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:32.059573  2504 ts_tablet_manager.cc:933] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:32.061573  2504 tablet_copy_client.cc:323] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 12:59:32.063053  1571 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet d72a925fb22046e2be514a7913c613a4 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 12:59:32.063614  1571 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet d72a925fb22046e2be514a7913c613a4 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-d72a925fb22046e2be514a7913c613a4
I20260709 12:59:32.071136  1571 tablet_copy_source_session.cc:215] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:32.074239  2504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d72a925fb22046e2be514a7913c613a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:32.090842  2504 tablet_copy_client.cc:806] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:32.091600  2504 tablet_copy_client.cc:670] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:32.095301  2504 tablet_copy_client.cc:538] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:32.104123  2504 tablet_bootstrap.cc:492] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:32.121148  2510 ts_tablet_manager.cc:933] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:32.123761  2510 tablet_copy_client.cc:323] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45431
I20260709 12:59:32.125855  2504 tablet_bootstrap.cc:492] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:32.125780  1720 tablet_copy_service.cc:140] P a2ba7fc60f9641cdac25f1e016ef91d8: Received BeginTabletCopySession request for tablet ac931d9355284a168be64d7debd009ae from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:52646)
I20260709 12:59:32.126518  1720 tablet_copy_service.cc:161] P a2ba7fc60f9641cdac25f1e016ef91d8: Beginning new tablet copy session on tablet ac931d9355284a168be64d7debd009ae from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:52646: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-ac931d9355284a168be64d7debd009ae
I20260709 12:59:32.126837  2504 tablet_bootstrap.cc:492] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:32.128767  2504 ts_tablet_manager.cc:1403] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.025s	user 0.025s	sys 0.001s
I20260709 12:59:32.132159  2504 raft_consensus.cc:359] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:32.132997  2504 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:32.133692  2504 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:32.135969  1720 tablet_copy_source_session.cc:215] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:32.138490  2504 ts_tablet_manager.cc:1434] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:59:32.140805  1571 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-d72a925fb22046e2be514a7913c613a4 received from {username='slave'} at 127.0.0.1:42562
I20260709 12:59:32.141242  1571 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-d72a925fb22046e2be514a7913c613a4 on tablet d72a925fb22046e2be514a7913c613a4 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:32.142057  2510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac931d9355284a168be64d7debd009ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:32.160399  2510 tablet_copy_client.cc:806] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:32.161062  2510 tablet_copy_client.cc:670] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:32.161597  2504 ts_tablet_manager.cc:933] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:32.168160  2510 tablet_copy_client.cc:538] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:32.175164  2504 tablet_copy_client.cc:323] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45431
I20260709 12:59:32.177544  1720 tablet_copy_service.cc:140] P a2ba7fc60f9641cdac25f1e016ef91d8: Received BeginTabletCopySession request for tablet 0adeaf85877a4103bbd070927413d267 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:52646)
I20260709 12:59:32.178146  1720 tablet_copy_service.cc:161] P a2ba7fc60f9641cdac25f1e016ef91d8: Beginning new tablet copy session on tablet 0adeaf85877a4103bbd070927413d267 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:52646: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-0adeaf85877a4103bbd070927413d267
I20260709 12:59:32.184108  2510 tablet_bootstrap.cc:492] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:32.189602  1720 tablet_copy_source_session.cc:215] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:32.197296  2504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0adeaf85877a4103bbd070927413d267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:32.228772  2504 tablet_copy_client.cc:806] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:32.229411  2504 tablet_copy_client.cc:670] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:32.233399  2504 tablet_copy_client.cc:538] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:32.244992  2510 tablet_bootstrap.cc:492] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:32.249313  2510 tablet_bootstrap.cc:492] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:32.250150  2510 ts_tablet_manager.cc:1403] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.066s	user 0.024s	sys 0.013s
I20260709 12:59:32.251967  2504 tablet_bootstrap.cc:492] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:32.253712  2510 raft_consensus.cc:359] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:32.254786  2510 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:32.255548  2510 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:32.259513  2510 ts_tablet_manager.cc:1434] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:59:32.266590  1720 tablet_copy_service.cc:342] P a2ba7fc60f9641cdac25f1e016ef91d8: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-ac931d9355284a168be64d7debd009ae received from {username='slave'} at 127.0.0.1:52646
I20260709 12:59:32.267146  1720 tablet_copy_service.cc:434] P a2ba7fc60f9641cdac25f1e016ef91d8: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-ac931d9355284a168be64d7debd009ae on tablet ac931d9355284a168be64d7debd009ae with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:32.279747  2351 raft_consensus.cc:1217] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:32.287799  2504 tablet_bootstrap.cc:492] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:32.288849  2504 tablet_bootstrap.cc:492] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:32.289603  2504 ts_tablet_manager.cc:1403] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.003s
I20260709 12:59:32.294595  2504 raft_consensus.cc:359] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:32.295490  2504 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:32.296103  2504 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:32.298629  2504 ts_tablet_manager.cc:1434] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:32.301047  1720 tablet_copy_service.cc:342] P a2ba7fc60f9641cdac25f1e016ef91d8: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-0adeaf85877a4103bbd070927413d267 received from {username='slave'} at 127.0.0.1:52646
I20260709 12:59:32.301585  1720 tablet_copy_service.cc:434] P a2ba7fc60f9641cdac25f1e016ef91d8: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-0adeaf85877a4103bbd070927413d267 on tablet 0adeaf85877a4103bbd070927413d267 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:32.617928  2351 raft_consensus.cc:1217] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:32.671018  2351 raft_consensus.cc:1217] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:32.690546  2351 raft_consensus.cc:1217] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:32.800961  2450 raft_consensus.cc:1064] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:32.803903  2450 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:32.812494  2351 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:32.814181  2450 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:32.814388  1710 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:32.814810  1636 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:32.816211  2450 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:32.817574  2485 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:32.832576  2485 raft_consensus.cc:2955] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:32.835055  1636 raft_consensus.cc:2955] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:32.836599  1701 raft_consensus.cc:2955] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:32.847406  2351 raft_consensus.cc:2955] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:32.857085  1458 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:32.888729  1557 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:32.895701  2351 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:32.897562  2485 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:32.900712  1701 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:32.906478  2485 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:32.911388  2522 raft_consensus.cc:2955] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:32.916718  1701 raft_consensus.cc:2955] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:32.918500  2351 raft_consensus.cc:2955] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:32.927565  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8b022b45a9b7462bb4b1a2a30ae36aa9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:32.938864  1457 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:32.958003  1621 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b022b45a9b7462bb4b1a2a30ae36aa9 with delete_type TABLET_DATA_TOMBSTONED (TS ba0548b7d2a94e8598475a46c279fe0e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54730
I20260709 12:59:32.960461  2527 tablet_replica.cc:333] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 12:59:32.961783  2527 raft_consensus.cc:2243] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:32.962657  2527 raft_consensus.cc:2272] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:32.966661  2527 ts_tablet_manager.cc:1916] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:32.982419  2527 ts_tablet_manager.cc:1929] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:32.982872  2527 log.cc:1199] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P ba0548b7d2a94e8598475a46c279fe0e: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/wals/8b022b45a9b7462bb4b1a2a30ae36aa9
I20260709 12:59:32.984970  1444 catalog_manager.cc:5028] TS ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): tablet 8b022b45a9b7462bb4b1a2a30ae36aa9 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:33.116942  2450 raft_consensus.cc:1064] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:33.120066  2450 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:33.129658  2493 raft_consensus.cc:1064] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:33.131938  2493 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:33.138998  2351 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.140588  1710 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.144491  2450 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.145495  2522 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.156042  2352 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.156042  1629 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.159627  2522 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:33.162096  2530 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:33.163550  1558 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.171619  2413 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:59:33.178673  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 50691.4us
I20260709 12:59:33.188120  2413 raft_consensus.cc:1064] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:33.194435  1636 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.199713  2413 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:33.221040  2533 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:33.251991  2522 raft_consensus.cc:2955] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.254623  1630 raft_consensus.cc:2955] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.268189  1709 raft_consensus.cc:2955] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.276985  2541 raft_consensus.cc:2955] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.287113  1558 raft_consensus.cc:2955] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.329919  1456 catalog_manager.cc:5697] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 12:59:33.368969  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 104937us
I20260709 12:59:33.374074  2352 raft_consensus.cc:2955] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.377807  1636 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.377580  2350 raft_consensus.cc:2955] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.374120  2351 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.382872  1635 raft_consensus.cc:2955] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 12:59:33.384188  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 87917.7us
I20260709 12:59:33.385625  2544 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:33.390095  1553 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.392033  2540 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:33.395123  1456 catalog_manager.cc:5697] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:33.407225  2539 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:59:33.446813  1701 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:33.453884  2530 raft_consensus.cc:2955] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.466297  1630 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.465628  2350 raft_consensus.cc:2955] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.468641  2542 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.479300  2540 raft_consensus.cc:2955] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.482183  1630 raft_consensus.cc:2955] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.491498  2353 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.494625  2544 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.501098  2353 raft_consensus.cc:2955] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.562166  1557 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:33.570649  1636 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.572154  2549 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.573462  2352 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.575536  1459 catalog_manager.cc:5697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.579931  2545 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:33.587765  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ac931d9355284a168be64d7debd009ae with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:33.596074  1542 tablet_service.cc:1558] Processing DeleteTablet for tablet ac931d9355284a168be64d7debd009ae with delete_type TABLET_DATA_TOMBSTONED (TS d62ce354d09a4be59c29094425d09bc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47096
I20260709 12:59:33.607182  1459 catalog_manager.cc:5697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.621277  1557 raft_consensus.cc:2955] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.628880  2503 raft_consensus.cc:2955] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.640962  2351 raft_consensus.cc:2955] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.632419  1635 raft_consensus.cc:2955] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.632509  1636 raft_consensus.cc:2955] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.631762  2550 tablet_replica.cc:333] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 12:59:33.644590  2550 raft_consensus.cc:2243] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:33.645449  2550 raft_consensus.cc:2272] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:33.678675  2550 ts_tablet_manager.cc:1916] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:33.694871  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d72a925fb22046e2be514a7913c613a4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:33.686908  1706 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:33.705917  2550 ts_tablet_manager.cc:1929] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:33.706576  2550 log.cc:1199] T ac931d9355284a168be64d7debd009ae P d62ce354d09a4be59c29094425d09bc1: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/wals/ac931d9355284a168be64d7debd009ae
I20260709 12:59:33.710922  1445 catalog_manager.cc:5028] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): tablet ac931d9355284a168be64d7debd009ae (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:33.711350  1560 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.714640  2542 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:33.715538  1459 catalog_manager.cc:5697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.721462  2351 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.723307  2539 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.728680  2530 raft_consensus.cc:2955] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.735349  1694 tablet_service.cc:1558] Processing DeleteTablet for tablet d72a925fb22046e2be514a7913c613a4 with delete_type TABLET_DATA_TOMBSTONED (TS a2ba7fc60f9641cdac25f1e016ef91d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:33.735949  1560 raft_consensus.cc:2955] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.770805  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0adeaf85877a4103bbd070927413d267 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:33.774436  2350 raft_consensus.cc:2955] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:33.784973  2551 tablet_replica.cc:333] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 12:59:33.786331  2551 raft_consensus.cc:2243] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:33.788949  2551 raft_consensus.cc:2272] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:33.793015  2551 ts_tablet_manager.cc:1916] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:33.792364  1457 catalog_manager.cc:5697] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:33.814874  2551 ts_tablet_manager.cc:1929] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:33.815433  1621 tablet_service.cc:1558] Processing DeleteTablet for tablet 0adeaf85877a4103bbd070927413d267 with delete_type TABLET_DATA_TOMBSTONED (TS ba0548b7d2a94e8598475a46c279fe0e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54730
I20260709 12:59:33.815542  2551 log.cc:1199] T d72a925fb22046e2be514a7913c613a4 P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/wals/d72a925fb22046e2be514a7913c613a4
I20260709 12:59:33.819106  1445 catalog_manager.cc:5028] TS a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): tablet d72a925fb22046e2be514a7913c613a4 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:33.822315  2552 tablet_replica.cc:333] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 12:59:33.831749  2552 raft_consensus.cc:2243] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:33.832422  2552 raft_consensus.cc:2272] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:33.834985  2552 ts_tablet_manager.cc:1916] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:33.852851  2552 ts_tablet_manager.cc:1929] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:33.853255  2552 log.cc:1199] T 0adeaf85877a4103bbd070927413d267 P ba0548b7d2a94e8598475a46c279fe0e: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/wals/0adeaf85877a4103bbd070927413d267
I20260709 12:59:33.854735  1444 catalog_manager.cc:5028] TS ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): tablet 0adeaf85877a4103bbd070927413d267 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:34.888034  1630 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 12:59:34.898097  1602 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2ba61e8020a4800b9e7fdae5533800b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:34.898375  1557 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:34.899751  1706 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:34.902140  2554 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:34.903213  2464 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:34.911392  2516 raft_consensus.cc:2955] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:34.915335  1560 raft_consensus.cc:2955] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:34.917335  1706 raft_consensus.cc:2955] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:34.930810  1557 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:34.932677  1457 catalog_manager.cc:5697] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:34.947032  1527 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 939d7b92438a4b20bd5f2e81d2291c03. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:34.947144  1630 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:34.947139  1706 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:34.948768  2485 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:34.949990  2553 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:34.961030  2485 raft_consensus.cc:2955] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:34.963408  1636 raft_consensus.cc:2955] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:34.966282  1706 raft_consensus.cc:2955] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:34.995787  1709 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:34.999986  1456 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 12:59:35.013891  1676 consensus_peers.cc:597] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb0b8480ae5646bea5cd44d499b6bcaa. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:35.019344  1557 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.022522  2530 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.031154  1635 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.040191  2530 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:35.044771  2530 raft_consensus.cc:2955] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:35.047495  1557 raft_consensus.cc:2955] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:35.056403  1630 raft_consensus.cc:2955] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:35.063530  1632 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:35.068228  1459 catalog_manager.cc:5697] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:35.073284  1706 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.074752  1557 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.075038  2554 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.078028  2554 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:35.081260  1602 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b73223ab67f4b5aab700c65a70e42cf. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:35.095762  2554 raft_consensus.cc:2955] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:35.099231  1706 raft_consensus.cc:2955] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:35.099581  1557 raft_consensus.cc:2955] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:35.128263  1459 catalog_manager.cc:5697] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:35.341341  2575 ts_tablet_manager.cc:933] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:35.344700  2575 tablet_copy_client.cc:323] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
I20260709 12:59:35.347333  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet c2ba61e8020a4800b9e7fdae5533800b from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 12:59:35.347960  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet c2ba61e8020a4800b9e7fdae5533800b from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-c2ba61e8020a4800b9e7fdae5533800b
I20260709 12:59:35.355969  1646 tablet_copy_source_session.cc:215] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:35.358844  2575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2ba61e8020a4800b9e7fdae5533800b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:35.375522  2575 tablet_copy_client.cc:806] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:35.376581  2575 tablet_copy_client.cc:670] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:35.389829  2575 tablet_copy_client.cc:538] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:35.408089  2575 tablet_bootstrap.cc:492] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:35.437199  2575 tablet_bootstrap.cc:492] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:35.438222  2575 tablet_bootstrap.cc:492] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:35.438970  2575 ts_tablet_manager.cc:1403] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.031s	user 0.024s	sys 0.003s
I20260709 12:59:35.443066  2575 raft_consensus.cc:359] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:35.443950  2575 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:35.444697  2575 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:35.447992  2575 ts_tablet_manager.cc:1434] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:35.450224  1646 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-c2ba61e8020a4800b9e7fdae5533800b received from {username='slave'} at 127.0.0.1:48246
I20260709 12:59:35.450724  1646 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-c2ba61e8020a4800b9e7fdae5533800b on tablet c2ba61e8020a4800b9e7fdae5533800b with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:35.465193  2575 ts_tablet_manager.cc:933] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:35.467268  2575 tablet_copy_client.cc:323] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
I20260709 12:59:35.469061  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet 6b73223ab67f4b5aab700c65a70e42cf from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 12:59:35.469671  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet 6b73223ab67f4b5aab700c65a70e42cf from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-6b73223ab67f4b5aab700c65a70e42cf
I20260709 12:59:35.477162  1646 tablet_copy_source_session.cc:215] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:35.482131  2575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b73223ab67f4b5aab700c65a70e42cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:35.500339  2575 tablet_copy_client.cc:806] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:35.501091  2575 tablet_copy_client.cc:670] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:35.505942  2575 tablet_copy_client.cc:538] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:35.516530  2575 tablet_bootstrap.cc:492] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:35.538280  2575 tablet_bootstrap.cc:492] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:35.539397  2575 tablet_bootstrap.cc:492] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:35.540123  2575 ts_tablet_manager.cc:1403] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.009s
I20260709 12:59:35.543499  2575 raft_consensus.cc:359] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:35.544628  2575 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:35.545437  2575 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:35.548782  2575 ts_tablet_manager.cc:1434] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 12:59:35.551330  1646 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-6b73223ab67f4b5aab700c65a70e42cf received from {username='slave'} at 127.0.0.1:48246
I20260709 12:59:35.551963  1646 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-6b73223ab67f4b5aab700c65a70e42cf on tablet 6b73223ab67f4b5aab700c65a70e42cf with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:35.576228  2575 ts_tablet_manager.cc:933] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:35.582808  2575 tablet_copy_client.cc:323] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 12:59:35.584959  1571 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet 939d7b92438a4b20bd5f2e81d2291c03 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 12:59:35.585572  1571 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet 939d7b92438a4b20bd5f2e81d2291c03 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-939d7b92438a4b20bd5f2e81d2291c03
I20260709 12:59:35.718142  2581 ts_tablet_manager.cc:933] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:35.748588  2581 tablet_copy_client.cc:323] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45431
I20260709 12:59:35.755368  1720 tablet_copy_service.cc:140] P a2ba7fc60f9641cdac25f1e016ef91d8: Received BeginTabletCopySession request for tablet bb0b8480ae5646bea5cd44d499b6bcaa from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:52646)
I20260709 12:59:35.756374  1720 tablet_copy_service.cc:161] P a2ba7fc60f9641cdac25f1e016ef91d8: Beginning new tablet copy session on tablet bb0b8480ae5646bea5cd44d499b6bcaa from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:52646: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-bb0b8480ae5646bea5cd44d499b6bcaa
I20260709 12:59:35.768739  1720 tablet_copy_source_session.cc:215] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:35.771703  1571 tablet_copy_source_session.cc:215] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:35.777230  2575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 939d7b92438a4b20bd5f2e81d2291c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:35.779372  2581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0b8480ae5646bea5cd44d499b6bcaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:35.830751  2575 tablet_copy_client.cc:806] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:35.831344  2351 raft_consensus.cc:1217] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:35.846688  2575 tablet_copy_client.cc:670] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:35.859671  2575 tablet_copy_client.cc:538] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:35.883894  2575 tablet_bootstrap.cc:492] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:35.900673  2581 tablet_copy_client.cc:806] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:35.901902  2581 tablet_copy_client.cc:670] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:35.921254  2581 tablet_copy_client.cc:538] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:35.939201  2581 tablet_bootstrap.cc:492] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:35.953576  2575 tablet_bootstrap.cc:492] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:35.954715  2575 tablet_bootstrap.cc:492] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:35.955472  2575 ts_tablet_manager.cc:1403] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.072s	user 0.025s	sys 0.005s
I20260709 12:59:35.959367  2575 raft_consensus.cc:359] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:35.961057  2575 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:35.961899  2575 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:35.964720  2575 ts_tablet_manager.cc:1434] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:35.967098  2581 tablet_bootstrap.cc:492] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:35.967723  1571 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-939d7b92438a4b20bd5f2e81d2291c03 received from {username='slave'} at 127.0.0.1:42562
I20260709 12:59:35.968304  1571 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-939d7b92438a4b20bd5f2e81d2291c03 on tablet 939d7b92438a4b20bd5f2e81d2291c03 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:35.968315  2581 tablet_bootstrap.cc:492] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:35.969406  2581 ts_tablet_manager.cc:1403] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.030s	user 0.023s	sys 0.003s
I20260709 12:59:35.979755  2353 raft_consensus.cc:1217] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:35.980280  2581 raft_consensus.cc:359] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:35.981304  2581 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:35.982159  2581 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:35.996529  2581 ts_tablet_manager.cc:1434] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.027s	user 0.003s	sys 0.005s
I20260709 12:59:35.999137  1720 tablet_copy_service.cc:342] P a2ba7fc60f9641cdac25f1e016ef91d8: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-bb0b8480ae5646bea5cd44d499b6bcaa received from {username='slave'} at 127.0.0.1:52646
I20260709 12:59:35.999742  1720 tablet_copy_service.cc:434] P a2ba7fc60f9641cdac25f1e016ef91d8: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-bb0b8480ae5646bea5cd44d499b6bcaa on tablet bb0b8480ae5646bea5cd44d499b6bcaa with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:36.007716  2351 raft_consensus.cc:1217] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:36.005906  2353 raft_consensus.cc:1217] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:36.374475  2586 raft_consensus.cc:1064] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:36.377343  2586 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:36.388329  1630 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.388948  2353 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.390750  2566 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:36.392550  2591 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:36.393839  1553 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.395239  2566 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:36.421511  2591 raft_consensus.cc:2955] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.425511  2353 raft_consensus.cc:2955] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.425087  1553 raft_consensus.cc:2955] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.433796  2464 raft_consensus.cc:1064] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:36.446934  2464 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:36.443665  1630 raft_consensus.cc:2955] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.456859  1457 catalog_manager.cc:5697] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:36.488651  1701 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.499056  2554 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:36.499647  1553 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.501858  2353 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.512230  2516 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:36.515920  2516 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:36.552960  2464 raft_consensus.cc:2955] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.564958  1701 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:36.571666  2353 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:36.572175  1636 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:36.573242  2591 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:36.577061  2566 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:36.580940  2574 raft_consensus.cc:1064] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:36.584009  2574 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:36.588099  2351 raft_consensus.cc:2955] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.598477  1636 raft_consensus.cc:2955] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.621294  2554 raft_consensus.cc:1064] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:36.611413  2591 raft_consensus.cc:2955] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.628733  1459 catalog_manager.cc:5697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 12:59:36.654808  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 78298.8us
I20260709 12:59:36.659744  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bb0b8480ae5646bea5cd44d499b6bcaa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:36.663416  2353 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.662439  2351 raft_consensus.cc:2955] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.665002  2587 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:36.683635  2554 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:36.685312  1553 raft_consensus.cc:2955] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.689235  1701 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.689335  1456 catalog_manager.cc:5697] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:36.691195  1706 raft_consensus.cc:2955] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.695990  1542 tablet_service.cc:1558] Processing DeleteTablet for tablet bb0b8480ae5646bea5cd44d499b6bcaa with delete_type TABLET_DATA_TOMBSTONED (TS d62ce354d09a4be59c29094425d09bc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47096
I20260709 12:59:36.737423  2611 tablet_replica.cc:333] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 12:59:36.744433  2611 raft_consensus.cc:2243] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:36.747506  2611 raft_consensus.cc:2272] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:36.753867  2611 ts_tablet_manager.cc:1916] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:36.769553  2584 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:59:36.783880  2611 ts_tablet_manager.cc:1929] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:36.787457  2611 log.cc:1199] T bb0b8480ae5646bea5cd44d499b6bcaa P d62ce354d09a4be59c29094425d09bc1: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/wals/bb0b8480ae5646bea5cd44d499b6bcaa
W20260709 12:59:36.791805  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 47171.8us
I20260709 12:59:36.801973  1630 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:36.810642  1553 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:36.811096  2351 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:59:36.810544  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 98188.5us
I20260709 12:59:36.814218  1629 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.814391  2606 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:36.814775  1445 catalog_manager.cc:5028] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): tablet bb0b8480ae5646bea5cd44d499b6bcaa (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:36.816061  2608 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:36.825753  2608 raft_consensus.cc:2955] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.836833  2614 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:36.836465  2351 raft_consensus.cc:2955] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.845652  1560 raft_consensus.cc:2955] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.847666  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c2ba61e8020a4800b9e7fdae5533800b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:36.843452  2587 raft_consensus.cc:2955] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.876899  1703 raft_consensus.cc:2955] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.877362  1456 catalog_manager.cc:5697] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.883584  1630 raft_consensus.cc:2955] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.904932  1694 tablet_service.cc:1558] Processing DeleteTablet for tablet c2ba61e8020a4800b9e7fdae5533800b with delete_type TABLET_DATA_TOMBSTONED (TS a2ba7fc60f9641cdac25f1e016ef91d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:36.916667  2617 tablet_replica.cc:333] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 12:59:36.918241  2617 raft_consensus.cc:2243] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:36.919430  2617 raft_consensus.cc:2272] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:36.918883  2351 raft_consensus.cc:2955] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.924392  2617 ts_tablet_manager.cc:1916] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:36.947973  2350 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.950594  2516 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:36.951109  1703 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.950690  1456 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:36.953816  2516 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:36.957558  1553 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.959738  2516 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:36.993880  2617 ts_tablet_manager.cc:1929] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:36.994414  2617 log.cc:1199] T c2ba61e8020a4800b9e7fdae5533800b P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/wals/c2ba61e8020a4800b9e7fdae5533800b
I20260709 12:59:36.994896  2554 raft_consensus.cc:2955] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:36.997413  1445 catalog_manager.cc:5028] TS a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): tablet c2ba61e8020a4800b9e7fdae5533800b (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:37.002564  1703 raft_consensus.cc:2955] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:37.006366  1553 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:37.006728  2351 raft_consensus.cc:2955] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:37.022845  1706 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.022845  2350 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.025241  2587 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:37.026481  2612 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:37.034801  2584 raft_consensus.cc:2955] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:37.037322  1553 raft_consensus.cc:2955] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:37.043002  1706 raft_consensus.cc:2955] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:37.046296  2350 raft_consensus.cc:2955] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:37.052630  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 939d7b92438a4b20bd5f2e81d2291c03 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:37.068173  1456 catalog_manager.cc:5697] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:37.094048  1459 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:37.098704  1630 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:37.108879  2350 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.114048  1621 tablet_service.cc:1558] Processing DeleteTablet for tablet 939d7b92438a4b20bd5f2e81d2291c03 with delete_type TABLET_DATA_TOMBSTONED (TS ba0548b7d2a94e8598475a46c279fe0e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54730
I20260709 12:59:37.114544  2608 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:37.128458  1553 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.132227  2554 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.134197  2621 tablet_replica.cc:333] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 12:59:37.136345  2621 raft_consensus.cc:2243] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:37.137068  2621 raft_consensus.cc:2272] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:37.143983  2599 raft_consensus.cc:2955] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:37.148938  1560 raft_consensus.cc:2955] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:37.147567  2350 raft_consensus.cc:2955] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:37.159595  2621 ts_tablet_manager.cc:1916] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:37.179252  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6b73223ab67f4b5aab700c65a70e42cf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:37.182651  2621 ts_tablet_manager.cc:1929] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:37.183290  2621 log.cc:1199] T 939d7b92438a4b20bd5f2e81d2291c03 P ba0548b7d2a94e8598475a46c279fe0e: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/wals/939d7b92438a4b20bd5f2e81d2291c03
I20260709 12:59:37.189144  1444 catalog_manager.cc:5028] TS ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): tablet 939d7b92438a4b20bd5f2e81d2291c03 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:37.188104  1458 catalog_manager.cc:5697] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:37.209322  1694 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b73223ab67f4b5aab700c65a70e42cf with delete_type TABLET_DATA_TOMBSTONED (TS a2ba7fc60f9641cdac25f1e016ef91d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:37.210565  2617 tablet_replica.cc:333] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 12:59:37.211863  2617 raft_consensus.cc:2243] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:37.212859  2617 raft_consensus.cc:2272] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:37.219132  2617 ts_tablet_manager.cc:1916] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:37.236162  2617 ts_tablet_manager.cc:1929] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:37.236598  2617 log.cc:1199] T 6b73223ab67f4b5aab700c65a70e42cf P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/wals/6b73223ab67f4b5aab700c65a70e42cf
I20260709 12:59:37.238180  1445 catalog_manager.cc:5028] TS a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): tablet 6b73223ab67f4b5aab700c65a70e42cf (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:38.309370  1630 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 12:59:38.317541  1602 consensus_peers.cc:597] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 25feb0fae8094dd29fea235dd1af7a14. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:38.319713  1706 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.320276  1560 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.321676  2622 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.323158  2599 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.333518  2622 raft_consensus.cc:2955] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.336055  1560 raft_consensus.cc:2955] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.340685  1703 raft_consensus.cc:2955] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.353077  1560 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:38.361712  1458 catalog_manager.cc:5697] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 12:59:38.362633  1527 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37bfe7ffa6604774b0b035da58f96eb9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:38.365164  1706 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.366881  2612 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.369730  1630 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.375234  2612 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.377565  2584 raft_consensus.cc:2955] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.379477  1706 raft_consensus.cc:2955] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.398803  1560 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:38.402012  1630 raft_consensus.cc:2955] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.417186  1457 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.427757  1706 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:38.428655  1527 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a353430d15945ee9524da3d2b667c90. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:38.430296  2584 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.440984  1630 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.443986  2584 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:38.457016  2624 raft_consensus.cc:2955] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.458318  1706 raft_consensus.cc:2955] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.479090  1630 raft_consensus.cc:2955] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.482867  1636 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:38.490007  1457 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.510247  1706 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:38.511986  1602 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 61e7345bb41047a2896de16e89540bbb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:38.512923  2607 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.522732  1560 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.524215  2599 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.531847  2607 raft_consensus.cc:2955] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.537329  1706 raft_consensus.cc:2955] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.545431  1560 raft_consensus.cc:2955] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:38.554546  1459 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:38.794109  2642 ts_tablet_manager.cc:933] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:38.796394  2642 tablet_copy_client.cc:323] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
I20260709 12:59:38.798527  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet 25feb0fae8094dd29fea235dd1af7a14 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 12:59:38.799033  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet 25feb0fae8094dd29fea235dd1af7a14 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-25feb0fae8094dd29fea235dd1af7a14
I20260709 12:59:38.805850  1646 tablet_copy_source_session.cc:215] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:38.810235  2642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25feb0fae8094dd29fea235dd1af7a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:38.829445  2642 tablet_copy_client.cc:806] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:38.830183  2642 tablet_copy_client.cc:670] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:38.832742  2644 ts_tablet_manager.cc:933] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:38.882409  2642 tablet_copy_client.cc:538] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:38.903620  2644 tablet_copy_client.cc:323] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 12:59:38.905529  1571 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet 37bfe7ffa6604774b0b035da58f96eb9 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 12:59:38.906095  1571 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet 37bfe7ffa6604774b0b035da58f96eb9 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-37bfe7ffa6604774b0b035da58f96eb9
I20260709 12:59:38.906616  2642 tablet_bootstrap.cc:492] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:38.938324  1571 tablet_copy_source_session.cc:215] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:38.969271  2644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37bfe7ffa6604774b0b035da58f96eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:39.021059  2642 tablet_bootstrap.cc:492] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.022653  2642 tablet_bootstrap.cc:492] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:39.023442  2642 ts_tablet_manager.cc:1403] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.123s	user 0.022s	sys 0.023s
I20260709 12:59:39.026785  2642 raft_consensus.cc:359] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:39.045521  2642 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:39.046344  2642 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:39.062687  2650 ts_tablet_manager.cc:933] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:39.066102  2648 ts_tablet_manager.cc:933] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:39.077570  2650 tablet_copy_client.cc:323] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
I20260709 12:59:39.080610  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet 61e7345bb41047a2896de16e89540bbb from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 12:59:39.081215  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet 61e7345bb41047a2896de16e89540bbb from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-61e7345bb41047a2896de16e89540bbb
I20260709 12:59:39.089295  2642 ts_tablet_manager.cc:1434] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.066s	user 0.012s	sys 0.005s
I20260709 12:59:39.091457  1645 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-25feb0fae8094dd29fea235dd1af7a14 received from {username='slave'} at 127.0.0.1:48246
I20260709 12:59:39.091956  1645 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-25feb0fae8094dd29fea235dd1af7a14 on tablet 25feb0fae8094dd29fea235dd1af7a14 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:39.098933  1646 tablet_copy_source_session.cc:215] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:39.102979  2650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61e7345bb41047a2896de16e89540bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:39.103129  2648 tablet_copy_client.cc:323] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 12:59:39.108783  1571 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet 9a353430d15945ee9524da3d2b667c90 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 12:59:39.109464  1571 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet 9a353430d15945ee9524da3d2b667c90 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-9a353430d15945ee9524da3d2b667c90
I20260709 12:59:39.112236  2644 tablet_copy_client.cc:806] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:39.114398  2644 tablet_copy_client.cc:670] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:39.124389  2650 tablet_copy_client.cc:806] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:39.125052  2650 tablet_copy_client.cc:670] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:39.125180  2644 tablet_copy_client.cc:538] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:39.130894  2650 tablet_copy_client.cc:538] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:39.133176  1571 tablet_copy_source_session.cc:215] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:39.137171  2648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a353430d15945ee9524da3d2b667c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:39.143110  2650 tablet_bootstrap.cc:492] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:39.152599  2644 tablet_bootstrap.cc:492] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:39.170979  2648 tablet_copy_client.cc:806] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:39.171875  2648 tablet_copy_client.cc:670] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:39.179190  2648 tablet_copy_client.cc:538] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:39.192167  2648 tablet_bootstrap.cc:492] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:39.197043  2650 tablet_bootstrap.cc:492] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.201938  2650 tablet_bootstrap.cc:492] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:39.206553  2650 ts_tablet_manager.cc:1403] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.064s	user 0.016s	sys 0.023s
I20260709 12:59:39.211148  2650 raft_consensus.cc:359] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:39.212209  2650 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:39.213639  2644 tablet_bootstrap.cc:492] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.213012  2650 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:39.216770  2650 ts_tablet_manager.cc:1434] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:59:39.218868  1646 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-61e7345bb41047a2896de16e89540bbb received from {username='slave'} at 127.0.0.1:48246
I20260709 12:59:39.219188  2644 tablet_bootstrap.cc:492] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:39.219367  1646 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-61e7345bb41047a2896de16e89540bbb on tablet 61e7345bb41047a2896de16e89540bbb with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:39.220054  2644 ts_tablet_manager.cc:1403] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.068s	user 0.029s	sys 0.004s
I20260709 12:59:39.227666  2644 raft_consensus.cc:359] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:39.229453  2644 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:39.235035  2644 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:39.238166  2644 ts_tablet_manager.cc:1434] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 12:59:39.240227  1571 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-37bfe7ffa6604774b0b035da58f96eb9 received from {username='slave'} at 127.0.0.1:42562
I20260709 12:59:39.240692  1571 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-37bfe7ffa6604774b0b035da58f96eb9 on tablet 37bfe7ffa6604774b0b035da58f96eb9 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:39.246091  2648 tablet_bootstrap.cc:492] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.247352  2648 tablet_bootstrap.cc:492] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:39.248157  2648 ts_tablet_manager.cc:1403] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.056s	user 0.024s	sys 0.008s
I20260709 12:59:39.251425  2648 raft_consensus.cc:359] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:39.252276  2648 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:39.252908  2648 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:39.255369  2648 ts_tablet_manager.cc:1434] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:59:39.258455  1571 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-9a353430d15945ee9524da3d2b667c90 received from {username='slave'} at 127.0.0.1:42562
I20260709 12:59:39.258896  1571 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-9a353430d15945ee9524da3d2b667c90 on tablet 9a353430d15945ee9524da3d2b667c90 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:39.311427  2350 raft_consensus.cc:1217] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:39.413604  2350 raft_consensus.cc:1217] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:39.423751  2350 raft_consensus.cc:1217] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:39.454699  2350 raft_consensus.cc:1217] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:39.913774  2626 raft_consensus.cc:1064] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:39.916801  2626 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:39.965799  2646 raft_consensus.cc:1064] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:39.971882  1710 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:39.977100  2350 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:39.977228  2659 raft_consensus.cc:1064] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:39.979935  1557 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:39.980556  2659 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:39.983342  2599 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:39.984457  2646 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:39.986726  2626 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:40.001606  2599 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:40.023372  2662 raft_consensus.cc:1064] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 12:59:40.034808  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51548.5us
I20260709 12:59:40.036711  2662 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:40.037779  1706 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.057402  2612 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:59:40.067409  1630 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.064150  2660 raft_consensus.cc:2955] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.102696  1710 raft_consensus.cc:2955] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.109826  2646 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:40.105022  1458 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:40.120641  2350 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.166566  1560 raft_consensus.cc:2955] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.178151  2624 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:40.189716  2353 raft_consensus.cc:2955] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.195188  1703 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.197841  2659 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:59:40.198731  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 63576.4us
I20260709 12:59:40.201905  1703 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.205799  2351 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.207809  2668 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:40.201937  1553 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.217231  2672 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:40.227231  1630 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.229355  2350 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.232252  2661 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.243762  2612 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:40.251104  2666 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:40.252153  2663 raft_consensus.cc:2955] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.288663  2350 raft_consensus.cc:2955] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.310699  1629 raft_consensus.cc:2955] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.311793  2626 raft_consensus.cc:2955] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.327133  1560 raft_consensus.cc:2955] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.379076  2350 raft_consensus.cc:2955] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.348094  1457 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:59:40.389379  2679 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:59:40.394235  1706 raft_consensus.cc:2955] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.372992  1630 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:40.402917  2353 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.404378  1560 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.404680  2659 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.404487  2646 raft_consensus.cc:2955] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.406744  2659 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.416298  1636 raft_consensus.cc:2955] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.418352  1703 raft_consensus.cc:2955] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.425555  2658 raft_consensus.cc:2955] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.427454  2353 raft_consensus.cc:2955] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.433272  2351 raft_consensus.cc:2955] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.436422  1706 raft_consensus.cc:2955] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.441731  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 61e7345bb41047a2896de16e89540bbb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:40.444206  1553 raft_consensus.cc:2955] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.498234  1560 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:40.503878  1459 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.508998  1459 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.510633  1459 catalog_manager.cc:5697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.521072  2350 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.521059  1630 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.525954  2646 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.529032  2668 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.540786  2663 raft_consensus.cc:2955] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.543812  2351 raft_consensus.cc:2955] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.550462  1630 raft_consensus.cc:2955] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.563956  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 37bfe7ffa6604774b0b035da58f96eb9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:40.567445  1560 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:40.574786  1632 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:40.577410  1459 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.578513  1694 tablet_service.cc:1558] Processing DeleteTablet for tablet 61e7345bb41047a2896de16e89540bbb with delete_type TABLET_DATA_TOMBSTONED (TS a2ba7fc60f9641cdac25f1e016ef91d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:40.584868  1706 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.585108  2689 tablet_replica.cc:333] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 12:59:40.586298  2689 raft_consensus.cc:2243] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.587157  2689 raft_consensus.cc:2272] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.587330  2662 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:40.587694  2350 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.591022  2689 ts_tablet_manager.cc:1916] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:40.590962  2646 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:40.602025  1706 raft_consensus.cc:2955] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.600099  2662 raft_consensus.cc:2955] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.607955  1703 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.609139  1694 tablet_service.cc:1558] Processing DeleteTablet for tablet 37bfe7ffa6604774b0b035da58f96eb9 with delete_type TABLET_DATA_TOMBSTONED (TS a2ba7fc60f9641cdac25f1e016ef91d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:40.610455  2661 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.618664  2658 raft_consensus.cc:2955] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.620517  2351 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.603519  2350 raft_consensus.cc:2955] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.624408  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9a353430d15945ee9524da3d2b667c90 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:40.626521  2660 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.626256  1703 raft_consensus.cc:2955] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.631801  2353 raft_consensus.cc:2955] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.636994  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 25feb0fae8094dd29fea235dd1af7a14 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:40.645169  2689 ts_tablet_manager.cc:1929] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:40.645761  2689 log.cc:1199] T 61e7345bb41047a2896de16e89540bbb P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/wals/61e7345bb41047a2896de16e89540bbb
I20260709 12:59:40.647586  2689 tablet_replica.cc:333] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 12:59:40.648124  1445 catalog_manager.cc:5028] TS a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): tablet 61e7345bb41047a2896de16e89540bbb (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:40.651702  1459 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.652806  2689 raft_consensus.cc:2243] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.653723  2689 raft_consensus.cc:2272] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.653677  1459 catalog_manager.cc:5697] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:40.659956  2689 ts_tablet_manager.cc:1916] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:40.678936  2689 ts_tablet_manager.cc:1929] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:40.679361  2689 log.cc:1199] T 37bfe7ffa6604774b0b035da58f96eb9 P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/wals/37bfe7ffa6604774b0b035da58f96eb9
I20260709 12:59:40.681231  1445 catalog_manager.cc:5028] TS a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): tablet 37bfe7ffa6604774b0b035da58f96eb9 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:40.696619  1621 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a353430d15945ee9524da3d2b667c90 with delete_type TABLET_DATA_TOMBSTONED (TS ba0548b7d2a94e8598475a46c279fe0e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54730
I20260709 12:59:40.703860  1542 tablet_service.cc:1558] Processing DeleteTablet for tablet 25feb0fae8094dd29fea235dd1af7a14 with delete_type TABLET_DATA_TOMBSTONED (TS d62ce354d09a4be59c29094425d09bc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47096
I20260709 12:59:40.706918  2691 tablet_replica.cc:333] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 12:59:40.707073  2690 tablet_replica.cc:333] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 12:59:40.708271  2690 raft_consensus.cc:2243] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.709142  2690 raft_consensus.cc:2272] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.711094  2691 raft_consensus.cc:2243] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.712560  2690 ts_tablet_manager.cc:1916] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:40.713248  2691 raft_consensus.cc:2272] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.716270  2691 ts_tablet_manager.cc:1916] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:40.735559  2690 ts_tablet_manager.cc:1929] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:40.736158  2690 log.cc:1199] T 9a353430d15945ee9524da3d2b667c90 P ba0548b7d2a94e8598475a46c279fe0e: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/wals/9a353430d15945ee9524da3d2b667c90
I20260709 12:59:40.736300  2691 ts_tablet_manager.cc:1929] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:40.736824  2691 log.cc:1199] T 25feb0fae8094dd29fea235dd1af7a14 P d62ce354d09a4be59c29094425d09bc1: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/wals/25feb0fae8094dd29fea235dd1af7a14
I20260709 12:59:40.738458  1444 catalog_manager.cc:5028] TS ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): tablet 9a353430d15945ee9524da3d2b667c90 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:40.738698  1445 catalog_manager.cc:5028] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): tablet 25feb0fae8094dd29fea235dd1af7a14 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:41.776091  1706 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:41.782584  1630 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:41.784142  2676 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:59:41.785334  1676 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c96b268f1f734ec793355ab247e28a75. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.785470  1560 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:41.787343  2676 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:41.793787  2591 raft_consensus.cc:2955] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:41.796262  1630 raft_consensus.cc:2955] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:41.798864  1560 raft_consensus.cc:2955] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:41.820434  1703 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:41.812400  1457 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:41.838866  1632 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:41.842890  1676 consensus_peers.cc:597] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c3a7d726b2f43c28355175906793af9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.845721  1560 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:41.853212  2701 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:41.854498  2676 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:41.874215  2700 raft_consensus.cc:2955] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:41.887285  1553 raft_consensus.cc:2955] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:41.895363  1632 raft_consensus.cc:2955] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:41.910984  1560 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:41.916096  1455 catalog_manager.cc:5697] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 12:59:41.930673  1527 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ca209b0d98c64e1da48e3027b3ea1d9a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.933794  1703 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:41.933944  1636 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:41.936121  2624 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:41.937189  2668 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:41.951272  2668 raft_consensus.cc:2955] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:41.961198  1706 raft_consensus.cc:2955] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:41.968734  1630 raft_consensus.cc:2955] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:41.990381  1459 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:41.992988  1630 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:42.004705  1706 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:42.006175  1560 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:42.006563  2660 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:42.007778  2708 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:42.010769  1602 consensus_peers.cc:597] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7db3e87d77f44ba19ad756277b557c45. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:42.023639  2708 raft_consensus.cc:2955] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:42.027382  1706 raft_consensus.cc:2955] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:42.029937  1560 raft_consensus.cc:2955] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:42.051129  1457 catalog_manager.cc:5697] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:42.323370  2715 ts_tablet_manager.cc:933] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:42.326660  2715 tablet_copy_client.cc:323] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45431
I20260709 12:59:42.328687  1720 tablet_copy_service.cc:140] P a2ba7fc60f9641cdac25f1e016ef91d8: Received BeginTabletCopySession request for tablet c96b268f1f734ec793355ab247e28a75 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:52646)
I20260709 12:59:42.329326  1720 tablet_copy_service.cc:161] P a2ba7fc60f9641cdac25f1e016ef91d8: Beginning new tablet copy session on tablet c96b268f1f734ec793355ab247e28a75 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:52646: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-c96b268f1f734ec793355ab247e28a75
I20260709 12:59:42.334131  2717 ts_tablet_manager.cc:933] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:42.348178  1720 tablet_copy_source_session.cc:215] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.350768  2717 tablet_copy_client.cc:323] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45431
I20260709 12:59:42.353046  2715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c96b268f1f734ec793355ab247e28a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.356492  1720 tablet_copy_service.cc:140] P a2ba7fc60f9641cdac25f1e016ef91d8: Received BeginTabletCopySession request for tablet 2c3a7d726b2f43c28355175906793af9 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:52646)
I20260709 12:59:42.357210  1720 tablet_copy_service.cc:161] P a2ba7fc60f9641cdac25f1e016ef91d8: Beginning new tablet copy session on tablet 2c3a7d726b2f43c28355175906793af9 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:52646: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-2c3a7d726b2f43c28355175906793af9
I20260709 12:59:42.368899  1720 tablet_copy_source_session.cc:215] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.373893  2717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3a7d726b2f43c28355175906793af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.377939  2715 tablet_copy_client.cc:806] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.384572  2715 tablet_copy_client.cc:670] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.389873  2715 tablet_copy_client.cc:538] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:42.401046  2715 tablet_bootstrap.cc:492] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:42.434823  2715 tablet_bootstrap.cc:492] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.435873  2715 tablet_bootstrap.cc:492] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:42.436911  2715 ts_tablet_manager.cc:1403] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.036s	user 0.025s	sys 0.009s
I20260709 12:59:42.440184  2715 raft_consensus.cc:359] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:42.441100  2715 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:42.554651  2715 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:42.572628  2719 ts_tablet_manager.cc:933] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:42.582650  2715 ts_tablet_manager.cc:1434] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.145s	user 0.023s	sys 0.038s
I20260709 12:59:42.585906  2719 tablet_copy_client.cc:323] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 12:59:42.600958  1720 tablet_copy_service.cc:342] P a2ba7fc60f9641cdac25f1e016ef91d8: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-c96b268f1f734ec793355ab247e28a75 received from {username='slave'} at 127.0.0.1:52646
I20260709 12:59:42.602026  1720 tablet_copy_service.cc:434] P a2ba7fc60f9641cdac25f1e016ef91d8: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-c96b268f1f734ec793355ab247e28a75 on tablet c96b268f1f734ec793355ab247e28a75 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:42.617348  2717 tablet_copy_client.cc:806] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.618606  2717 tablet_copy_client.cc:670] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.633903  1571 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet ca209b0d98c64e1da48e3027b3ea1d9a from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 12:59:42.634582  1571 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet ca209b0d98c64e1da48e3027b3ea1d9a from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-ca209b0d98c64e1da48e3027b3ea1d9a
I20260709 12:59:42.651857  1571 tablet_copy_source_session.cc:215] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.657831  2719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca209b0d98c64e1da48e3027b3ea1d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.714123  2719 tablet_copy_client.cc:806] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.715085  2719 tablet_copy_client.cc:670] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.719630  2719 tablet_copy_client.cc:538] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:59:42.730770  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51741.2us
I20260709 12:59:42.736756  2715 ts_tablet_manager.cc:933] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:42.754914  2715 tablet_copy_client.cc:323] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
I20260709 12:59:42.755915  2719 tablet_bootstrap.cc:492] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:42.757829  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet 7db3e87d77f44ba19ad756277b557c45 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 12:59:42.758461  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet 7db3e87d77f44ba19ad756277b557c45 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-7db3e87d77f44ba19ad756277b557c45
I20260709 12:59:42.768425  1646 tablet_copy_source_session.cc:215] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.791622  2352 raft_consensus.cc:1217] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:42.801395  2717 tablet_copy_client.cc:538] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:42.802341  2719 tablet_bootstrap.cc:492] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.807888  2719 tablet_bootstrap.cc:492] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:42.808741  2719 ts_tablet_manager.cc:1403] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.060s	user 0.024s	sys 0.008s
I20260709 12:59:42.820318  2715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7db3e87d77f44ba19ad756277b557c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.824440  2719 raft_consensus.cc:359] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:42.825513  2719 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:42.826905  2719 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:42.831655  2719 ts_tablet_manager.cc:1434] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 12:59:42.828491  2717 tablet_bootstrap.cc:492] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:42.833871  1571 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-ca209b0d98c64e1da48e3027b3ea1d9a received from {username='slave'} at 127.0.0.1:42562
I20260709 12:59:42.834342  1571 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-ca209b0d98c64e1da48e3027b3ea1d9a on tablet ca209b0d98c64e1da48e3027b3ea1d9a with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:42.865319  2717 tablet_bootstrap.cc:492] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.868557  2717 tablet_bootstrap.cc:492] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:42.869498  2717 ts_tablet_manager.cc:1403] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.014s
I20260709 12:59:42.872840  2717 raft_consensus.cc:359] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:42.874194  2717 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:42.874931  2717 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:42.876313  2715 tablet_copy_client.cc:806] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.877058  2715 tablet_copy_client.cc:670] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.878340  2717 ts_tablet_manager.cc:1434] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:59:42.880944  1720 tablet_copy_service.cc:342] P a2ba7fc60f9641cdac25f1e016ef91d8: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-2c3a7d726b2f43c28355175906793af9 received from {username='slave'} at 127.0.0.1:52646
I20260709 12:59:42.881389  1720 tablet_copy_service.cc:434] P a2ba7fc60f9641cdac25f1e016ef91d8: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-2c3a7d726b2f43c28355175906793af9 on tablet 2c3a7d726b2f43c28355175906793af9 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:42.882248  2715 tablet_copy_client.cc:538] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:42.892622  2715 tablet_bootstrap.cc:492] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:42.921551  2715 tablet_bootstrap.cc:492] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.922674  2715 tablet_bootstrap.cc:492] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:42.923527  2715 ts_tablet_manager.cc:1403] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.006s
I20260709 12:59:42.927544  2715 raft_consensus.cc:359] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:42.928596  2715 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:42.929275  2715 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:42.931936  2715 ts_tablet_manager.cc:1434] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:59:42.933826  1646 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-7db3e87d77f44ba19ad756277b557c45 received from {username='slave'} at 127.0.0.1:48246
I20260709 12:59:42.934330  1646 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-7db3e87d77f44ba19ad756277b557c45 on tablet 7db3e87d77f44ba19ad756277b557c45 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:42.961694  2351 raft_consensus.cc:1217] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:43.223420  2676 raft_consensus.cc:1064] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:43.226497  2676 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:43.245488  1629 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.244596  2351 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.249322  1561 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.251114  2701 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:43.258101  2700 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.261102  2700 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:43.282419  2701 raft_consensus.cc:2955] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.284425  2352 raft_consensus.cc:2955] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.305181  1455 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.321245  2352 raft_consensus.cc:1217] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:43.337940  1629 raft_consensus.cc:2955] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.349255  2352 raft_consensus.cc:1217] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:43.361308  1560 raft_consensus.cc:2955] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.439095  2723 raft_consensus.cc:1064] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:43.445861  2723 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 12:59:43.461604  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 78695.9us
I20260709 12:59:43.468644  2352 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.472249  2727 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.476681  1632 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.478645  2723 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.478932  1703 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.480436  2727 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.496733  2727 raft_consensus.cc:2955] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.510619  2351 raft_consensus.cc:2955] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.511204  1703 raft_consensus.cc:2955] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.519304  1629 raft_consensus.cc:2955] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.542461  1458 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:43.556540  1703 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:43.563997  2351 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.565433  1553 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:43.572321  1560 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.581866  2701 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:43.585573  2735 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:43.599591  2735 raft_consensus.cc:2955] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.601504  2350 raft_consensus.cc:2955] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.605666  2352 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.607523  2723 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:43.608539  1560 raft_consensus.cc:2955] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.620709  1703 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.626655  2727 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:43.630689  2693 raft_consensus.cc:2955] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.640532  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c96b268f1f734ec793355ab247e28a75 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:43.643265  1459 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.644479  2350 raft_consensus.cc:2955] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.649406  1703 raft_consensus.cc:2955] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.652724  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ca209b0d98c64e1da48e3027b3ea1d9a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:43.664860  1621 tablet_service.cc:1558] Processing DeleteTablet for tablet c96b268f1f734ec793355ab247e28a75 with delete_type TABLET_DATA_TOMBSTONED (TS ba0548b7d2a94e8598475a46c279fe0e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54730
I20260709 12:59:43.672704  1457 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.701349  2755 tablet_replica.cc:333] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 12:59:43.706883  2755 raft_consensus.cc:2243] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:43.707937  2755 raft_consensus.cc:2272] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:43.710610  1621 tablet_service.cc:1558] Processing DeleteTablet for tablet ca209b0d98c64e1da48e3027b3ea1d9a with delete_type TABLET_DATA_TOMBSTONED (TS ba0548b7d2a94e8598475a46c279fe0e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54730
I20260709 12:59:43.712805  2755 ts_tablet_manager.cc:1916] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:43.731119  2755 ts_tablet_manager.cc:1929] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:43.731609  2755 log.cc:1199] T c96b268f1f734ec793355ab247e28a75 P ba0548b7d2a94e8598475a46c279fe0e: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/wals/c96b268f1f734ec793355ab247e28a75
I20260709 12:59:43.733706  1444 catalog_manager.cc:5028] TS ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): tablet c96b268f1f734ec793355ab247e28a75 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:43.733722  2755 tablet_replica.cc:333] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 12:59:43.737723  2755 raft_consensus.cc:2243] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:43.738431  2755 raft_consensus.cc:2272] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:43.743531  2755 ts_tablet_manager.cc:1916] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:43.753554  2700 raft_consensus.cc:1064] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:43.756822  2700 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:43.800516  2755 ts_tablet_manager.cc:1929] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:43.801091  2755 log.cc:1199] T ca209b0d98c64e1da48e3027b3ea1d9a P ba0548b7d2a94e8598475a46c279fe0e: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/wals/ca209b0d98c64e1da48e3027b3ea1d9a
I20260709 12:59:43.803834  1629 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.805378  1560 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.809132  2350 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.808035  1444 catalog_manager.cc:5028] TS ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): tablet ca209b0d98c64e1da48e3027b3ea1d9a (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:43.827368  2700 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:43.828384  2676 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.829329  2735 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:43.854131  2735 raft_consensus.cc:2955] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.857084  1560 raft_consensus.cc:2955] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.855954  1629 raft_consensus.cc:2955] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.878322  1457 catalog_manager.cc:5697] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.878854  2350 raft_consensus.cc:2955] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.904358  1703 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:43.919085  2350 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.920761  2735 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:43.924382  1632 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.928721  2701 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:43.933524  2700 raft_consensus.cc:2955] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.934732  2730 raft_consensus.cc:1064] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:43.933665  2351 raft_consensus.cc:2955] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.939504  2730 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:43.949285  1629 raft_consensus.cc:2955] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:43.953609  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2c3a7d726b2f43c28355175906793af9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:43.985546  2351 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.987344  1560 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.991825  2761 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:43.993988  1703 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.994815  2730 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:43.996258  2746 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:44.003326  1455 catalog_manager.cc:5697] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:44.048796  1542 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c3a7d726b2f43c28355175906793af9 with delete_type TABLET_DATA_TOMBSTONED (TS d62ce354d09a4be59c29094425d09bc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47096
I20260709 12:59:44.072947  2767 tablet_replica.cc:333] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 12:59:44.074302  2767 raft_consensus.cc:2243] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:44.075145  2767 raft_consensus.cc:2272] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:44.079502  2767 ts_tablet_manager.cc:1916] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:44.123750  2767 ts_tablet_manager.cc:1929] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:44.129457  2767 log.cc:1199] T 2c3a7d726b2f43c28355175906793af9 P d62ce354d09a4be59c29094425d09bc1: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/wals/2c3a7d726b2f43c28355175906793af9
W20260709 12:59:44.129639  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 43938.9us
I20260709 12:59:44.132376  1445 catalog_manager.cc:5028] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): tablet 2c3a7d726b2f43c28355175906793af9 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:44.160416  2761 raft_consensus.cc:2955] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:44.162314  1560 raft_consensus.cc:2955] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:44.182667  2351 raft_consensus.cc:2955] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:44.185740  1455 catalog_manager.cc:5697] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:44.195093  1703 raft_consensus.cc:2955] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:44.222746  1629 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:44.230646  2351 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:44.231240  1703 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:44.232383  2730 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:44.234099  2763 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:44.243243  1703 raft_consensus.cc:2955] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:44.242794  2730 raft_consensus.cc:2955] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:44.244596  2351 raft_consensus.cc:2955] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:44.259605  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7db3e87d77f44ba19ad756277b557c45 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:44.275179  1455 catalog_manager.cc:5697] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:44.290014  1542 tablet_service.cc:1558] Processing DeleteTablet for tablet 7db3e87d77f44ba19ad756277b557c45 with delete_type TABLET_DATA_TOMBSTONED (TS d62ce354d09a4be59c29094425d09bc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47096
I20260709 12:59:44.291035  2767 tablet_replica.cc:333] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 12:59:44.309727  2767 raft_consensus.cc:2243] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:44.310722  2767 raft_consensus.cc:2272] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:44.314713  2767 ts_tablet_manager.cc:1916] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:44.335665  2767 ts_tablet_manager.cc:1929] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:44.336213  2767 log.cc:1199] T 7db3e87d77f44ba19ad756277b557c45 P d62ce354d09a4be59c29094425d09bc1: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/wals/7db3e87d77f44ba19ad756277b557c45
I20260709 12:59:44.337651  1445 catalog_manager.cc:5028] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): tablet 7db3e87d77f44ba19ad756277b557c45 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
W20260709 12:59:45.076221  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54910.1us
W20260709 12:59:45.104843  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 86332.2us
I20260709 12:59:45.373365  1560 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:45.381320  1629 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:45.381664  1527 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 13debc393e8a4121883dda403cb4badf. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.382026  1709 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.382829  2779 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.383826  2772 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.392165  2768 raft_consensus.cc:2955] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.394045  1709 raft_consensus.cc:2955] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.394680  1629 raft_consensus.cc:2955] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.412544  1636 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:45.424499  1457 catalog_manager.cc:5697] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:45.426834  1560 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.428303  1703 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:45.429864  1602 consensus_peers.cc:597] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bc25e0c8bb834899a98a40efe1e66bba. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.431097  2775 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.433146  2770 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:45.456892  2775 raft_consensus.cc:2955] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.469765  1560 raft_consensus.cc:2955] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.490116  1706 raft_consensus.cc:2955] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.498657  1561 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 12:59:45.539748  1527 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f47909c83d54b699ec2b1c2402084b4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.539249  1459 catalog_manager.cc:5697] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.542235  1706 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.543634  1636 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.544530  2725 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.548466  2779 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.567065  2779 raft_consensus.cc:2955] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.573259  1629 raft_consensus.cc:2955] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.574676  1706 raft_consensus.cc:2955] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.597838  1636 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:45.603188  1457 catalog_manager.cc:5697] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 12:59:45.628192  1602 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fb91933caa7241dc9c723195dd6950dc. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.628880  1560 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:45.630092  1703 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:45.641074  2730 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:59:45.650054  2776 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:45.674233  2776 raft_consensus.cc:2955] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.679656  1553 raft_consensus.cc:2955] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.683792  1703 raft_consensus.cc:2955] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:45.787122  1455 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:45.866425  2798 ts_tablet_manager.cc:933] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 12:59:45.869313  1582 debug-util.cc:398] Leaking SignalData structure 0x7b0800118b80 after lost signal to thread 434
W20260709 12:59:45.872210  1582 debug-util.cc:398] Leaking SignalData structure 0x7b0800094760 after lost signal to thread 1493
W20260709 12:59:45.875326  1582 debug-util.cc:398] Leaking SignalData structure 0x7b080032b3a0 after lost signal to thread 1585
W20260709 12:59:45.880420  1582 debug-util.cc:398] Leaking SignalData structure 0x7b080030e980 after lost signal to thread 1660
W20260709 12:59:45.882441  1582 debug-util.cc:398] Leaking SignalData structure 0x7b080020c5a0 after lost signal to thread 1734
W20260709 12:59:45.884649  1582 debug-util.cc:398] Leaking SignalData structure 0x7b080032ee40 after lost signal to thread 2377
I20260709 12:59:45.885449  2798 tablet_copy_client.cc:323] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
I20260709 12:59:45.896348  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet bc25e0c8bb834899a98a40efe1e66bba from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 12:59:45.897158  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet bc25e0c8bb834899a98a40efe1e66bba from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-bc25e0c8bb834899a98a40efe1e66bba
I20260709 12:59:45.912809  2800 ts_tablet_manager.cc:933] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:45.933995  1646 tablet_copy_source_session.cc:215] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:45.941984  2798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc25e0c8bb834899a98a40efe1e66bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:45.949926  2800 tablet_copy_client.cc:323] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 12:59:45.953805  1571 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet 13debc393e8a4121883dda403cb4badf from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 12:59:45.954720  1571 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet 13debc393e8a4121883dda403cb4badf from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-13debc393e8a4121883dda403cb4badf
I20260709 12:59:45.987138  1571 tablet_copy_source_session.cc:215] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:45.996731  2800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13debc393e8a4121883dda403cb4badf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:46.030468  2800 tablet_copy_client.cc:806] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:46.031960  2798 tablet_copy_client.cc:806] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:46.032807  2798 tablet_copy_client.cc:670] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:46.034008  2800 tablet_copy_client.cc:670] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:46.042639  2798 tablet_copy_client.cc:538] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:46.046528  2800 tablet_copy_client.cc:538] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:46.058686  2798 tablet_bootstrap.cc:492] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:46.088138  2800 tablet_bootstrap.cc:492] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:46.154054  2798 tablet_bootstrap.cc:492] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:46.155479  2798 tablet_bootstrap.cc:492] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:46.156456  2798 ts_tablet_manager.cc:1403] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.098s	user 0.030s	sys 0.011s
I20260709 12:59:46.160647  2798 raft_consensus.cc:359] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:46.162480  2798 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:46.163460  2798 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:46.184423  2798 ts_tablet_manager.cc:1434] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.027s	user 0.011s	sys 0.005s
I20260709 12:59:46.187947  2804 ts_tablet_manager.cc:933] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:46.188921  1646 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-bc25e0c8bb834899a98a40efe1e66bba received from {username='slave'} at 127.0.0.1:48246
I20260709 12:59:46.189587  1646 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-bc25e0c8bb834899a98a40efe1e66bba on tablet bc25e0c8bb834899a98a40efe1e66bba with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:46.225881  2800 tablet_bootstrap.cc:492] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:46.227110  2800 tablet_bootstrap.cc:492] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:46.228492  2800 ts_tablet_manager.cc:1403] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.141s	user 0.022s	sys 0.016s
I20260709 12:59:46.233407  2798 ts_tablet_manager.cc:933] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:46.239336  2352 raft_consensus.cc:1217] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:46.257865  2798 tablet_copy_client.cc:323] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
I20260709 12:59:46.262310  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet fb91933caa7241dc9c723195dd6950dc from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 12:59:46.263104  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet fb91933caa7241dc9c723195dd6950dc from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-fb91933caa7241dc9c723195dd6950dc
I20260709 12:59:46.264351  2800 raft_consensus.cc:359] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:46.265324  2800 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:46.266186  2800 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:46.269404  2800 ts_tablet_manager.cc:1434] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.038s	user 0.002s	sys 0.005s
I20260709 12:59:46.275686  2804 tablet_copy_client.cc:323] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 12:59:46.280761  1571 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-13debc393e8a4121883dda403cb4badf received from {username='slave'} at 127.0.0.1:42562
I20260709 12:59:46.281308  1571 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-13debc393e8a4121883dda403cb4badf on tablet 13debc393e8a4121883dda403cb4badf with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:46.281116  1570 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet 2f47909c83d54b699ec2b1c2402084b4 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 12:59:46.283159  1570 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet 2f47909c83d54b699ec2b1c2402084b4 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-2f47909c83d54b699ec2b1c2402084b4
I20260709 12:59:46.286829  1646 tablet_copy_source_session.cc:215] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:46.293356  2798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb91933caa7241dc9c723195dd6950dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:46.322631  1570 tablet_copy_source_session.cc:215] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:46.326596  2804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f47909c83d54b699ec2b1c2402084b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:46.328632  2798 tablet_copy_client.cc:806] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:46.329502  2798 tablet_copy_client.cc:670] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:46.334643  2798 tablet_copy_client.cc:538] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:46.355304  2804 tablet_copy_client.cc:806] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:46.356307  2804 tablet_copy_client.cc:670] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:46.370201  2804 tablet_copy_client.cc:538] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:46.370424  2352 raft_consensus.cc:1217] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:46.376459  2798 tablet_bootstrap.cc:492] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:46.396256  2804 tablet_bootstrap.cc:492] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:46.462049  2798 tablet_bootstrap.cc:492] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:46.463068  2804 tablet_bootstrap.cc:492] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:46.463382  2798 tablet_bootstrap.cc:492] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:46.464332  2804 tablet_bootstrap.cc:492] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:46.464463  2798 ts_tablet_manager.cc:1403] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.088s	user 0.042s	sys 0.009s
I20260709 12:59:46.465224  2804 ts_tablet_manager.cc:1403] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.069s	user 0.032s	sys 0.032s
I20260709 12:59:46.473527  2804 raft_consensus.cc:359] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:46.474610  2804 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:46.473955  2798 raft_consensus.cc:359] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:46.475472  2798 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:46.475646  2804 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:46.476401  2798 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:46.479279  2804 ts_tablet_manager.cc:1434] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:59:46.480114  2798 ts_tablet_manager.cc:1434] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 12:59:46.481329  1570 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-2f47909c83d54b699ec2b1c2402084b4 received from {username='slave'} at 127.0.0.1:42562
I20260709 12:59:46.481941  1570 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-2f47909c83d54b699ec2b1c2402084b4 on tablet 2f47909c83d54b699ec2b1c2402084b4 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:46.487972  1646 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-fb91933caa7241dc9c723195dd6950dc received from {username='slave'} at 127.0.0.1:48246
I20260709 12:59:46.488487  1646 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-fb91933caa7241dc9c723195dd6950dc on tablet fb91933caa7241dc9c723195dd6950dc with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:46.568130   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:46.650871  2352 raft_consensus.cc:1217] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:46.672356  2775 raft_consensus.cc:1064] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:46.686389  2352 raft_consensus.cc:1217] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:59:46.690009  2775 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:46.752871  2352 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.753569  1553 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.755528  1704 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.756582  2770 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:46.761608  2776 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:46.776803  2813 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:46.787307  1704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3897029c4b5f443296641c39af287e98"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:46.788183  1704 raft_consensus.cc:606] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:46.802408  1704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:46.803189  1704 raft_consensus.cc:606] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:46.809369  2775 raft_consensus.cc:2955] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:46.813803  1560 raft_consensus.cc:2955] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:46.825210  1553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:41400
I20260709 12:59:46.826040  1553 raft_consensus.cc:606] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:46.826452  2725 raft_consensus.cc:1064] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:46.829437  2725 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:46.834504  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:46.835386  1703 raft_consensus.cc:606] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:46.849725  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac931d9355284a168be64d7debd009ae"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:46.850515  1709 raft_consensus.cc:606] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:46.852672  1455 catalog_manager.cc:5697] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:46.853358  1709 raft_consensus.cc:2955] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:46.864338  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:46.865099  1703 raft_consensus.cc:606] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:46.867569   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:59:46.869601   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:46.870374  2352 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.873898  2768 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:46.878753  1704 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.881157  2772 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:46.885143  2351 raft_consensus.cc:2955] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:46.895330  1636 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.900275  2818 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:47.001291  1629 raft_consensus.cc:2955] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.001952  2779 raft_consensus.cc:2955] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.006296  2769 raft_consensus.cc:993] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:47.006978  2769 raft_consensus.cc:1081] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:47.008986  2351 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac931d9355284a168be64d7debd009ae"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:59274
I20260709 12:59:47.009660  2351 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:47.010027  2351 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.016696  1704 raft_consensus.cc:2955] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.019835  2351 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:47.025168  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:47.024359  2352 raft_consensus.cc:2955] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.025929  2351 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:47.026983  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:47.027608  1709 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:47.027889  1709 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:47.025967  1632 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.028836  1709 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:47.029877  1709 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.036655  1709 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:47.039611  2318 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 12:59:47.040769  2805 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:47.058390  2775 raft_consensus.cc:1064] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:47.061868  2775 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:47.067400  2779 raft_consensus.cc:993] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:47.068027  2779 raft_consensus.cc:1081] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:47.070242  1704 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46210
I20260709 12:59:47.071089  1704 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:47.071633  1704 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.055470  1632 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:47.081141  2805 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 12:59:47.081694  1709 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:47.083545  2805 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:47.087944  2813 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:47.088863  2351 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:59:47.087436  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 94839.5us
I20260709 12:59:47.094590  1560 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:47.096120  1636 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:47.097707  2776 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:47.101457  2813 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:47.118903  2769 raft_consensus.cc:993] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:47.119499  2769 raft_consensus.cc:1081] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:47.121657  2697 raft_consensus.cc:993] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:47.122792  2825 raft_consensus.cc:1081] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:47.129240  1629 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 12:59:47.129833  1629 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:47.130110  1629 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.133736  1632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 12:59:47.134306  1632 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:47.134673  1704 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.136291  1632 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.137617  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:47.138243  1560 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:47.138526  1560 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:47.139331  1560 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.140429  1560 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.148607  1629 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.150688  1629 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:47.151955  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:47.152628  1561 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.153574  1632 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.155792  1632 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:47.148451  1630 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:47.157275  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:47.158367  1559 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.162009  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:47.162817  1706 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:47.163154  1706 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:47.164016  1706 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.162817  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:47.171455  1710 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:47.171801  1710 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:47.172636  1710 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.174284  1710 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.165468  1706 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.184950  1561 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:47.181196  1703 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.166513  1704 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:47.167584  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:47.187079  1632 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.189234  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:47.190393  2813 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:47.190995  2813 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:47.192076  2813 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.184749  2352 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.194144  2776 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:47.197932  2776 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:47.165275  1560 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:47.209998  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:47.212321  2770 raft_consensus.cc:2955] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.219748  1710 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:47.224678  1632 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:47.226792  2827 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:47.227406  2827 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:47.228556  2827 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.247835  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:47.249197  2775 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:47.250006  2775 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:47.251271  2775 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.214807  1706 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:47.217159  2353 raft_consensus.cc:2955] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.258028  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bc25e0c8bb834899a98a40efe1e66bba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:47.271967  2779 raft_consensus.cc:1064] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:47.275522  2779 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 12:59:47.298887  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 48570.4us
I20260709 12:59:47.303287  1559 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:47.306207  1710 raft_consensus.cc:2955] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.353327  1455 catalog_manager.cc:5697] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.379431  2353 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:59:47.382329  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 56930.4us
I20260709 12:59:47.382074  2768 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:59:47.385681  2839 raft_consensus.cc:2955] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.397820  1559 raft_consensus.cc:2955] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.407637  2825 raft_consensus.cc:993] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:47.408356  2825 raft_consensus.cc:1081] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:47.419440  2697 raft_consensus.cc:993] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:47.420110  2825 raft_consensus.cc:1081] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:47.437067  1703 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:59:47.439236  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 45708.5us
I20260709 12:59:47.444068  2772 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:47.452044  1459 catalog_manager.cc:5697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:47.455935  1630 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.446979  1710 raft_consensus.cc:2955] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.484890  2353 raft_consensus.cc:2955] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 12:59:47.499584  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 118102us
I20260709 12:59:47.505095  1632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3897029c4b5f443296641c39af287e98"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 12:59:47.505993  1632 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:47.506475  1632 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.507681  1455 catalog_manager.cc:5697] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.518458  1630 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 12:59:47.519143  1630 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:47.519508  1630 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.554944  1629 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:47.543507  1459 catalog_manager.cc:5697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:47.556816  2830 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:47.563114  1459 catalog_manager.cc:5697] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:47.563868  1632 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.568017  1632 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:47.568480  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:47.569154  1560 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.574007  1630 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.576107  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:47.576790  1710 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:47.577118  1710 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:47.577275  1630 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:47.577998  1710 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.581453  1710 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.582748  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:47.584079  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:47.584698  1709 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:47.585026  1709 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:47.585793  1709 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.587042  1709 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:59:47.589725  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44743.1us
I20260709 12:59:47.592342  1560 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:47.565467  1459 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:47.593444  1559 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:47.600262  1709 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:47.603715  1706 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:47.605420  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:47.608673  2725 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:59:47.611114  2805 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:47.612229  2775 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:47.612779  2775 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:47.613818  2775 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:47.623351  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:47.627190  1710 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:47.632448  1542 tablet_service.cc:1558] Processing DeleteTablet for tablet bc25e0c8bb834899a98a40efe1e66bba with delete_type TABLET_DATA_TOMBSTONED (TS d62ce354d09a4be59c29094425d09bc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47096
I20260709 12:59:47.634634  1553 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:47.638221  1560 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:47.656559  1559 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:47.668233  2845 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:47.691481  1636 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.642923  2813 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:59:47.678746  2845 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:47.710373  2845 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:47.732569  2848 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:59:47.735942  1457 catalog_manager.cc:5697] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:47.746557  2853 tablet_replica.cc:333] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 12:59:47.730612  2772 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
W20260709 12:59:47.844688  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 61884.5us
I20260709 12:59:47.856730  2352 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.882823  2853 raft_consensus.cc:2243] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:47.884136  2853 raft_consensus.cc:2272] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:47.886967  2852 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.088s	user 0.002s	sys 0.000s
W20260709 12:59:47.921030  2854 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:59:47.921368  1703 raft_consensus.cc:2955] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:47.942836  1560 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:47.909307   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:47.964398  2770 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:47.977947  1709 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:47.985812  1701 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:59:48.004446  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 264669us
I20260709 12:59:48.004707  2853 ts_tablet_manager.cc:1916] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:48.023002  2352 raft_consensus.cc:2955] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 12:59:48.059857  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 330984us
I20260709 12:59:48.079900  1630 raft_consensus.cc:2955] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:48.059105  2867 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:59:48.054657  2855 raft_consensus.cc:2955] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 12:59:48.244884  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 181371us
I20260709 12:59:48.286526  1630 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:48.334326  1704 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:48.350443   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:48.363806  2862 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:59:48.371270  1630 raft_consensus.cc:2955] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:48.360574  2770 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:59:48.446468  2843 raft_consensus.cc:2955] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:48.449241  1458 catalog_manager.cc:5697] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:48.455832  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 95726.9us
I20260709 12:59:48.465323  1458 catalog_manager.cc:5697] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:48.475900  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 66204.8us
I20260709 12:59:48.449637  2773 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.110s
I20260709 12:59:48.510533  2880 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:59:48.611531  1708 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:48.638558  2351 raft_consensus.cc:2955] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:48.670656  2853 ts_tablet_manager.cc:1929] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:48.671339  2853 log.cc:1199] T bc25e0c8bb834899a98a40efe1e66bba P d62ce354d09a4be59c29094425d09bc1: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/wals/bc25e0c8bb834899a98a40efe1e66bba
I20260709 12:59:48.672631  2853 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47096 (request call id 75) took 1041 ms. Trace:
I20260709 12:59:48.673138  2853 rpcz_store.cc:276] 0709 12:59:47.631428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:47.632116 (+   688us) service_pool.cc:224] Handling call
0709 12:59:47.736867 (+104751us) ts_tablet_manager.cc:1151] Deleting tablet bc25e0c8bb834899a98a40efe1e66bba
0709 12:59:47.736961 (+    94us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:59:48.521238 (+784277us) tablet_metadata.cc:665] Metadata flushed
0709 12:59:48.670465 (+149227us) tablet_metadata.cc:665] Metadata flushed
0709 12:59:48.672539 (+  2074us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62511,"tablet-delete.queue_time_us":102932,"thread_start_us":59295,"threads_started":1}
I20260709 12:59:48.741778  1445 catalog_manager.cc:5028] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): tablet bc25e0c8bb834899a98a40efe1e66bba (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:48.704341  2843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:47096 (request call id 74) took 1067 ms. Trace:
I20260709 12:59:48.767488  2843 rpcz_store.cc:276] 0709 12:59:47.630968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:47.631160 (+   192us) service_pool.cc:224] Handling call
0709 12:59:47.654346 (+ 23186us) log.cc:844] Serialized 501 byte log entry
0709 12:59:48.696494 (+1042148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2499,"raft.queue_time_us":4320,"raft.run_cpu_time_us":1476,"raft.run_wall_time_us":51691,"thread_start_us":76847,"threads_started":1}
I20260709 12:59:48.775660  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 13debc393e8a4121883dda403cb4badf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:48.805270  1560 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:48.807132  2862 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:48.830389  1558 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:48.833559  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:48.833927  2873 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 12:59:48.834431  1709 raft_consensus.cc:606] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:48.837373  2873 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:48.859921  2873 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:48.860905  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:48.869287  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:48.881639  2886 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:48.882510  2886 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:48.909020  1559 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:48.933606  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 12:59:48.938520  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 53166.7us
I20260709 12:59:48.940152  2768 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:48.943507  2886 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:48.946982  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:48.947237  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:48.948370  1558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:41400
I20260709 12:59:48.949023  1558 raft_consensus.cc:606] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:48.955613  2862 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:48.963913  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:48.965451  2886 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:59:48.978816  2862 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:59:48.985018  2896 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.082s	user 0.000s	sys 0.003s
I20260709 12:59:48.992218  2824 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:59:49.002559  2895 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
I20260709 12:59:49.019881  1459 catalog_manager.cc:5697] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:49.033154  1630 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:49.077667  1455 catalog_manager.cc:5697] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:49.103461  1694 tablet_service.cc:1558] Processing DeleteTablet for tablet 13debc393e8a4121883dda403cb4badf with delete_type TABLET_DATA_TOMBSTONED (TS a2ba7fc60f9641cdac25f1e016ef91d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:49.115442  1559 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:49.134159  2350 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:49.140053  1701 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:49.140424  1629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:49.143146  1629 raft_consensus.cc:606] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:49.138738  2889 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:59:49.147404  2768 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:49.167047  1701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5838267321348eb83df088988da6449"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:49.167798  1701 raft_consensus.cc:606] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:49.192006  2353 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:49.201956  2844 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:49.204850  1553 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:49.205806  1702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:49.206542  1702 raft_consensus.cc:606] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:49.211086  2772 raft_consensus.cc:993] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:49.211638  2772 raft_consensus.cc:1081] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:49.168344  2865 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 12:59:49.214756  1702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:49.236783  2861 raft_consensus.cc:993] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:49.237566  1708 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46210
I20260709 12:59:49.238312  1708 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:49.237882  2865 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:49.270152  2865 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 12:59:49.265764  1708 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.278301  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:49.252929  1702 raft_consensus.cc:606] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:49.256255  2875 raft_consensus.cc:1081] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:49.303560  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:49.323300  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 12:59:49.292408  2862 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:59:49.363231  1676 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 12:59:49.370078  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 92484.1us
W20260709 12:59:49.377964  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50159.6us
W20260709 12:59:49.385667  2913 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:59:49.403337  2353 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5838267321348eb83df088988da6449"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:59274
I20260709 12:59:49.404114  2353 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:49.404572  2353 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.398700  2861 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:59:49.438158  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 113018us
I20260709 12:59:49.493857  2353 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:49.463928  2912 tablet_replica.cc:333] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 12:59:49.474108  1708 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 12:59:49.497541  2911 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.239s	user 0.002s	sys 0.000s
I20260709 12:59:49.532835  2353 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:49.546000  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:49.546901  1632 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.578020  2350 raft_consensus.cc:2955] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 12:59:49.585928  2914 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.159s	user 0.001s	sys 0.001s
W20260709 12:59:49.594127  2907 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:59:49.596774  1708 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:49.598903  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:49.599617  1628 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.613346  2824 raft_consensus.cc:993] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:49.613991  2824 raft_consensus.cc:1081] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:49.623947  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:49.624738  1555 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:49.625093  1555 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:49.625926  1555 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:49.627231  1555 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.641366  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:49.642148  1710 raft_consensus.cc:606] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:49.648082   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:59:49.648417  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:49.649067  1708 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:49.649343  1708 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:49.650329   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:49.650193  1708 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:49.651588  1708 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.653133  2861 raft_consensus.cc:993] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:49.653450  2876 raft_consensus.cc:993] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:49.653707  2861 raft_consensus.cc:1081] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:49.654028  2876 raft_consensus.cc:1081] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:49.657439  2353 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:59274
I20260709 12:59:49.658223  2353 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:49.658604  2353 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.624393  2872 raft_consensus.cc:2955] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:49.674914  2845 raft_consensus.cc:2955] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:49.676951  2351 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:59274
I20260709 12:59:49.677641  2351 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:49.678033  2351 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.698947  2349 raft_consensus.cc:2955] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:49.701023  1632 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:49.711552  2353 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:49.717764  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:49.719255  2351 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:49.724140  2352 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:59274
I20260709 12:59:49.724855  2353 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:49.724961  2352 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:49.723096  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:49.726462  2917 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:49.727872  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:49.732141  1710 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:49.729349  1708 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:49.732177  2352 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.732779  1710 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 12:59:49.735498  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 183031us
I20260709 12:59:49.763098  2348 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:49.736545  1636 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.868777  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:49.869550  1559 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.880117  2352 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:49.884519  1628 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:49.889533  1710 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:49.894685  1710 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.906133  2351 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:49.906919  2352 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:49.873325  2917 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 12:59:49.908919  1559 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:49.926709  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:49.910778  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:49.929317  1701 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:49.929661  1701 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:49.911553  2912 raft_consensus.cc:2243] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:49.936093  1701 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 12:59:49.938162  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 234591us
I20260709 12:59:49.940106  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2f47909c83d54b699ec2b1c2402084b4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:49.910411  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:49.942036  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fb91933caa7241dc9c723195dd6950dc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:49.951612  1710 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:49.913909  1555 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
W20260709 12:59:49.961531  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 266172us
I20260709 12:59:49.962608  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 12:59:49.965220  1701 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.934772  2912 raft_consensus.cc:2272] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:49.963759  1708 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:49.998668  1704 raft_consensus.cc:2955] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:50.003716  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:50.004776  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:49.994616  2805 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:50.036446  1632 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:49.999634  1708 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 12:59:50.039654  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 71206.5us
I20260709 12:59:50.057394  1632 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:50.045282  2876 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:50.092614  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:50.047919  2805 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 12:59:50.033708  2917 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:50.110150  2876 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:50.125715  2934 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:50.109004  1708 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:50.130297  2933 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:50.131103  2912 ts_tablet_manager.cc:1916] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:50.126541  2805 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 12:59:50.166988  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 74451.8us
I20260709 12:59:50.151804  2933 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 12:59:50.134845  2876 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:50.171723  1701 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
W20260709 12:59:50.172487  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:59:50.155869  2934 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 12:59:50.182685  1636 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:50.183837  2885 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 12:59:50.184425  2885 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:50.199018  2934 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:50.216675  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:50.229064  1553 raft_consensus.cc:2955] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:50.288393  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 12:59:50.288707  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 91236.5us
I20260709 12:59:50.237576  2933 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:50.306310  2921 raft_consensus.cc:993] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:50.294299  2885 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 12:59:50.306825  2921 raft_consensus.cc:1081] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:50.329391  1601 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:50.331816  2862 raft_consensus.cc:993] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:50.332043  2927 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:59:50.347980  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59665.9us
W20260709 12:59:50.353389  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 65431.7us
I20260709 12:59:50.360774  1632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 12:59:50.361316  1632 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:50.361704  1632 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.364377  2926 raft_consensus.cc:1081] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:50.379230  1632 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:50.382797  1632 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:50.385253  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:50.385915  1701 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:50.386211  1701 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:50.386926  1701 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:50.387701  1701 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.388252  1708 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.394799  1701 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:50.395949  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:50.396704  2885 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:50.397177  2885 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:50.404645  1706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46224
I20260709 12:59:50.404754  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:50.405280  1706 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:50.405332  1554 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.405748  1706 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.398041  2885 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:50.411828  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 12:59:50.433977  1458 catalog_manager.cc:5697] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:50.436510  1455 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 200) took 1121 ms. Trace:
I20260709 12:59:50.437065  1706 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:50.441074  1705 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:50.442725  1706 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:50.444133  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:50.436935  1455 rpcz_store.cc:276] 0709 12:59:49.313123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:49.313302 (+   179us) service_pool.cc:224] Handling call
0709 12:59:50.434233 (+1120931us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95022,"reported_tablets":11}
I20260709 12:59:50.454043  1553 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.462888  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:50.464838  1632 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:50.465370  1632 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:50.466527  1632 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:50.475797  1457 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.478878  1553 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:50.480203  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:50.478780  2847 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 12:59:50.487111  2847 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:50.491492  2847 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 12:59:50.495179  1632 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.497406  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:50.492818  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 12:59:50.502008  1554 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:50.486704  2912 ts_tablet_manager.cc:1929] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:50.506956  1708 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:50.513118  2923 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:50.512451  2912 log.cc:1199] T 13debc393e8a4121883dda403cb4badf P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/wals/13debc393e8a4121883dda403cb4badf
I20260709 12:59:50.518419  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:50.519337  2912 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46194 (request call id 71) took 1417 ms. Trace:
I20260709 12:59:50.520009  1674 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:50.518481  2923 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:50.521075  2847 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:59:50.530637  1632 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:50.519750  2912 rpcz_store.cc:276] 0709 12:59:49.102414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:49.103003 (+   589us) service_pool.cc:224] Handling call
0709 12:59:49.174196 (+ 71193us) ts_tablet_manager.cc:1151] Deleting tablet 13debc393e8a4121883dda403cb4badf
0709 12:59:49.174264 (+    68us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:59:49.284553 (+110289us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400077454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276e2ff9 00007fda2c6f0482 00007fda2c7212d0 00007fda2c70783d 00007fda2c7077e9 00007fda2c707779 00007fda2c707741 00007fda2c70696d 000000000038f354 000000000038f199
0709 12:59:50.366750 (+1082197us) tablet_metadata.cc:665] Metadata flushed
0709 12:59:50.486561 (+119811us) tablet_metadata.cc:665] Metadata flushed
0709 12:59:50.519236 (+ 32675us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":371990,"spinlock_wait_cycles":247459584,"tablet-delete.queue_time_us":65635,"thread_start_us":41951,"threads_started":1}
I20260709 12:59:50.525861  2923 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:50.541592  1445 catalog_manager.cc:5028] TS a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): tablet 13debc393e8a4121883dda403cb4badf (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:50.554638  1628 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:50.555950  2933 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:59:50.477618  2929 consensus_queue.cc:1175] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:59:50.582612  1455 catalog_manager.cc:5697] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:50.584674  1455 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index 3 to 4, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.598241  1708 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:50.599740  2917 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:50.610334  1553 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:50.611943  2923 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:50.635802  1621 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f47909c83d54b699ec2b1c2402084b4 with delete_type TABLET_DATA_TOMBSTONED (TS ba0548b7d2a94e8598475a46c279fe0e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54730
I20260709 12:59:50.662838  1627 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:50.668951  2861 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:50.683424  2965 tablet_replica.cc:333] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 12:59:50.684365   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:50.685751  2965 raft_consensus.cc:2243] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:50.686496  2965 raft_consensus.cc:2272] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:50.691470  2965 ts_tablet_manager.cc:1916] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:50.716701  2965 ts_tablet_manager.cc:1929] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:50.717095  2965 log.cc:1199] T 2f47909c83d54b699ec2b1c2402084b4 P ba0548b7d2a94e8598475a46c279fe0e: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/wals/2f47909c83d54b699ec2b1c2402084b4
I20260709 12:59:50.720338  1444 catalog_manager.cc:5028] TS ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): tablet 2f47909c83d54b699ec2b1c2402084b4 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:50.721055  1694 tablet_service.cc:1558] Processing DeleteTablet for tablet fb91933caa7241dc9c723195dd6950dc with delete_type TABLET_DATA_TOMBSTONED (TS a2ba7fc60f9641cdac25f1e016ef91d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:50.726379  2912 tablet_replica.cc:333] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 12:59:50.725776  1458 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.727635  2912 raft_consensus.cc:2243] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:50.728578  2912 raft_consensus.cc:2272] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:50.728080  1458 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.737221  1458 catalog_manager.cc:5697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.738386  1553 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:50.739058  1458 catalog_manager.cc:5697] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.741145  2917 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:50.768519  1708 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:50.773522  1457 catalog_manager.cc:5697] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 2, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.758204  2912 ts_tablet_manager.cc:1916] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:50.778654  2805 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:50.781164  1708 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:50.783246  2933 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:50.818001  1628 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:50.819860  2917 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:50.821987  2912 ts_tablet_manager.cc:1929] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:59:50.825062  2912 log.cc:1199] T fb91933caa7241dc9c723195dd6950dc P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/wals/fb91933caa7241dc9c723195dd6950dc
I20260709 12:59:50.829809  1628 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:50.840444  2917 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:50.844635  1445 catalog_manager.cc:5028] TS a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): tablet fb91933caa7241dc9c723195dd6950dc (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:50.871918  1706 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:50.877677  2917 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:50.937139  1553 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:50.941416   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:50.957880  2887 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:50.963810  1706 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:50.972361  1552 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:50.983234  2949 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:50.986740  1628 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:50.988348  2875 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.010874  2887 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.032615  1628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:51.033269  1628 raft_consensus.cc:606] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:51.044329  1553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 12:59:51.044947  1553 raft_consensus.cc:606] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:51.048966  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:51.049614  1627 raft_consensus.cc:606] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:51.084371  1628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:51.084962  1628 raft_consensus.cc:606] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:51.092218  1628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:51.092829  1628 raft_consensus.cc:606] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:51.094718   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:51.096235   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:51.145031  2887 raft_consensus.cc:993] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:51.145596  2919 raft_consensus.cc:1081] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:51.150604  1706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46224
I20260709 12:59:51.151166  1706 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:51.151552  1706 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.164341  1706 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:51.167174  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:51.167454  1706 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:51.167847  1552 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.169389  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:51.169999  1627 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:51.170306  1627 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:51.171058  1627 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:51.172289  1627 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.175072  1552 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:51.176506  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:51.177404  2949 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:51.178126  2949 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:51.178989  1627 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:51.179177  2949 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:51.190945  2862 raft_consensus.cc:993] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:51.192837  2862 raft_consensus.cc:1081] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:51.194985  1702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46224
I20260709 12:59:51.195514  1702 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:51.195850  1702 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:51.196552  1456 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 2, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:51.203274  1702 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:51.205272  1702 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:51.206689  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:51.206681  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:51.207387  1552 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:51.207473  1627 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:51.208213  1627 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:51.208937  1627 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:51.210157  1627 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:51.216069  1552 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 12:59:51.217324  1627 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 12:59:51.218408  1675 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:51.220106  2949 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:51.220803  2949 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:51.221592  2949 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:51.234122  1456 catalog_manager.cc:5697] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 2 to 3, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:51.363273  2862 raft_consensus.cc:993] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:51.363811  2887 raft_consensus.cc:1081] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Signalling peer 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:51.365460  2352 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:59270
I20260709 12:59:51.366112  2352 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:51.366555  2352 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.373162  2352 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:51.375495  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:51.375674  2352 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:51.375701  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:51.376307  1552 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.376515  1627 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:51.377187  1627 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:51.378026  1627 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:51.381311  1627 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.386761  1552 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:51.387843  2318 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:51.388888  2805 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:51.389348  2805 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 12:59:51.391577  2805 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:51.395507  1627 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:51.404356  1456 catalog_manager.cc:5697] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:51.487036  2887 raft_consensus.cc:993] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:51.487617  2862 raft_consensus.cc:1081] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Signalling peer 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:51.489264  2352 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:59270
I20260709 12:59:51.489802  2352 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:51.490200  2352 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.496785  2937 raft_consensus.cc:993] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:51.497280  2937 raft_consensus.cc:1081] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:51.500073  1628 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54748
I20260709 12:59:51.500613  1628 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:51.500959  1628 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.502007  2352 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:51.504480  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:51.504480  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:51.505154  1627 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:51.505260  1702 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.505535  1627 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:51.506327  1627 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:51.507330  1627 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.508359  1628 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:51.513346  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:51.513921  1552 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:51.514122  1627 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:51.514991  2352 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:51.515221  2317 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:51.514250  1552 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:51.516175  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:51.516360  2805 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:51.516742  1706 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.516870  2805 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 12:59:51.516373  1552 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:51.517900  2805 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:51.518725  1552 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.514719  1702 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:51.526278  1552 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:51.532047  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:51.532939  2862 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:51.533515  2862 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:51.534354  2862 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:51.523916  1706 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:51.537797  1628 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:51.540710  1456 catalog_manager.cc:5697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:51.547992  1458 catalog_manager.cc:5697] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:51.591813  1553 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:51.593137  2949 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.608994  1628 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:51.610838  2948 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.732508  1628 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:51.735997  2865 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:51.770095  1553 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:51.772466  2865 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:51.845608  1553 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 12:59:51.853206  1527 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 22f9010f94804fcaacae8539ddfc64a7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:51.854049  1702 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:51.855505  2877 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.855876  1627 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:51.857265  2889 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.867925  2938 raft_consensus.cc:2955] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:51.872352  1706 raft_consensus.cc:2955] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:51.872793  1627 raft_consensus.cc:2955] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:51.888109  1702 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:51.898072  1458 catalog_manager.cc:5697] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:51.914340  1632 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:51.919936  2917 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:51.921782  1553 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:51.923821  2988 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:51.926149  1676 consensus_peers.cc:597] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70981cbf9c1143309f0f78706fe5931d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:51.929062  1632 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:51.944682  1627 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:51.945438  2949 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:51.949065  2805 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:51.955108  1702 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:51.956772  2862 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.968624  1554 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:51.971136  2887 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:51.982609  1554 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:51.984349  2805 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:51.991523  2865 raft_consensus.cc:2955] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:52.043143  1552 raft_consensus.cc:2955] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:52.089305  1628 raft_consensus.cc:2955] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:52.100440   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:59:52.147395  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66142.4us
I20260709 12:59:52.184510  1635 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:52.185223  1456 catalog_manager.cc:5697] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 12:59:52.202904  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 103668us
W20260709 12:59:52.206712  1602 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6c211dcb5c947719413e384c6a72769. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:52.241631  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 194067us
I20260709 12:59:52.251502  1701 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:52.267750  1554 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:59:52.287002  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 200463us
I20260709 12:59:52.313858  3010 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:52.322965  1701 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:52.326529  3002 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:52.346074  3015 ts_tablet_manager.cc:933] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 12:59:52.355093  2887 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:52.386003  3005 raft_consensus.cc:2955] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:52.402136  3015 tablet_copy_client.cc:323] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 12:59:52.401249  1708 raft_consensus.cc:2955] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:52.412024  1570 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet 22f9010f94804fcaacae8539ddfc64a7 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 12:59:52.412720  1570 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet 22f9010f94804fcaacae8539ddfc64a7 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-22f9010f94804fcaacae8539ddfc64a7
I20260709 12:59:52.436044  1635 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:52.437167  3020 ts_tablet_manager.cc:933] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:52.450126  1457 catalog_manager.cc:5697] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:52.457808  3020 tablet_copy_client.cc:323] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45431
I20260709 12:59:52.465643  1720 tablet_copy_service.cc:140] P a2ba7fc60f9641cdac25f1e016ef91d8: Received BeginTabletCopySession request for tablet 70981cbf9c1143309f0f78706fe5931d from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:52646)
I20260709 12:59:52.466442  1720 tablet_copy_service.cc:161] P a2ba7fc60f9641cdac25f1e016ef91d8: Beginning new tablet copy session on tablet 70981cbf9c1143309f0f78706fe5931d from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:52646: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-70981cbf9c1143309f0f78706fe5931d
I20260709 12:59:52.470428  1570 tablet_copy_source_session.cc:215] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:52.474915  3015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f9010f94804fcaacae8539ddfc64a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:52.477785  1559 raft_consensus.cc:2955] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:52.506506  1720 tablet_copy_source_session.cc:215] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:52.504357   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:52.508769  3015 tablet_copy_client.cc:806] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:52.509670  3015 tablet_copy_client.cc:670] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
W20260709 12:59:52.511683  1602 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e2094a808984c84823c1e0dad0a5de0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:52.512583  3020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70981cbf9c1143309f0f78706fe5931d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:52.513245  1701 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:52.516057  1554 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:52.519652  3005 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:52.521893  3005 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.534013  3020 tablet_copy_client.cc:806] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:52.534732  3020 tablet_copy_client.cc:670] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:52.542321  3015 tablet_copy_client.cc:538] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:52.560098  3015 tablet_bootstrap.cc:492] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:52.565160  1706 raft_consensus.cc:2955] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:52.570008  3020 tablet_copy_client.cc:538] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:52.559126  2887 raft_consensus.cc:2955] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:52.607671  1554 raft_consensus.cc:2955] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:52.611584  3020 tablet_bootstrap.cc:492] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:52.643182  1706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0adeaf85877a4103bbd070927413d267"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:52.644201  1706 raft_consensus.cc:606] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:52.651319  1457 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:52.660027  1706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:52.660809  1706 raft_consensus.cc:606] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:52.671439  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 12:59:52.672386  1559 raft_consensus.cc:606] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:52.694947  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:41400
I20260709 12:59:52.695698  1559 raft_consensus.cc:606] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 12:59:52.706095  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 78815.1us
I20260709 12:59:52.759814  3025 ts_tablet_manager.cc:933] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 12:59:52.795446  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 100375us
W20260709 12:59:52.808408  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 99905.8us
W20260709 12:59:52.835501  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 62145.5us
I20260709 12:59:52.826618  3041 raft_consensus.cc:993] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:52.856020  1628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:52.856735  1628 raft_consensus.cc:606] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:52.847864  3041 raft_consensus.cc:1081] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:52.945698   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:52.947889   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 12:59:53.062626  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 132671us
I20260709 12:59:53.019035  3025 tablet_copy_client.cc:323] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
W20260709 12:59:53.189436  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 200537us
W20260709 12:59:53.207479  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 205050us
W20260709 12:59:53.246022  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 293537us
I20260709 12:59:53.259580  3015 tablet_bootstrap.cc:492] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:53.260983  3015 tablet_bootstrap.cc:492] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:53.271193  3015 ts_tablet_manager.cc:1403] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.711s	user 0.036s	sys 0.013s
I20260709 12:59:53.271304  3051 raft_consensus.cc:993] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:53.237263  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet e6c211dcb5c947719413e384c6a72769 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 12:59:53.277238  3051 raft_consensus.cc:1081] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:53.318673  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet e6c211dcb5c947719413e384c6a72769 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-e6c211dcb5c947719413e384c6a72769
I20260709 12:59:53.320787  3015 raft_consensus.cc:359] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:53.322395  3015 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:53.323680  3015 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 12:59:53.427922  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 202816us
I20260709 12:59:53.429940  1646 tablet_copy_source_session.cc:215] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:59:53.483415  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 259817us
W20260709 12:59:53.490129  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 238457us
I20260709 12:59:53.500028  1701 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46210
W20260709 12:59:53.531404  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 277160us
I20260709 12:59:53.531996  1701 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.532457  1701 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:59:53.543712  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46582us
I20260709 12:59:53.542673  3015 ts_tablet_manager.cc:1434] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.271s	user 0.008s	sys 0.006s
W20260709 12:59:53.587401  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 340149us
I20260709 12:59:53.609740  2353 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0adeaf85877a4103bbd070927413d267"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:59274
I20260709 12:59:53.610671  2353 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.611199  2353 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.613322  2351 raft_consensus.cc:1217] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:59:53.622857  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 62593.7us
W20260709 12:59:53.624064  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 376528us
I20260709 12:59:53.661321  1701 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:53.665732  1701 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:53.667441  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:53.668120  1553 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.667268  2353 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:53.668457  1553 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:53.671350  2353 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:53.673796  1570 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-22f9010f94804fcaacae8539ddfc64a7 received from {username='slave'} at 127.0.0.1:42562
I20260709 12:59:53.674273  3031 raft_consensus.cc:993] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:53.681314  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:53.682173  1628 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.682308  3068 ts_tablet_manager.cc:933] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:53.674321  1570 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-22f9010f94804fcaacae8539ddfc64a7 on tablet 22f9010f94804fcaacae8539ddfc64a7 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:53.695578  3076 raft_consensus.cc:993] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:53.696200  3059 raft_consensus.cc:1081] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
W20260709 12:59:53.703501  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 54242.2us
I20260709 12:59:53.706055  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:53.706728  1701 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.707077  1701 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:53.707913  1701 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:53.705597  1553 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:53.710314  1553 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.711413  1628 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:53.713696  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:53.714591  3034 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.715086  3034 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:53.716003  3034 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:53.718182  2352 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:53.720237  3047 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:59:53.723966  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50182.7us
I20260709 12:59:53.731698  1553 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:53.736227  1701 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:59:53.741348  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 63884.8us
I20260709 12:59:53.748610  1701 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:53.751920  3068 tablet_copy_client.cc:323] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
W20260709 12:59:53.755470  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 85753.4us
I20260709 12:59:53.758131  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 12:59:53.753643  3040 raft_consensus.cc:1081] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:53.767620  3025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6c211dcb5c947719413e384c6a72769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:53.770785  3042 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.771476  3042 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 12:59:53.772428  3042 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:53.778622  3020 tablet_bootstrap.cc:492] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:53.779767  3020 tablet_bootstrap.cc:492] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:53.780644  3020 ts_tablet_manager.cc:1403] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 1.169s	user 0.023s	sys 0.029s
I20260709 12:59:53.783385  1706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46224
I20260709 12:59:53.783955  1706 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.784338  1706 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.823396  1706 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:53.783159  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:53.827122  1627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54748
I20260709 12:59:53.833114  1627 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.833570  1627 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.833760  1457 catalog_manager.cc:5697] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:53.848205  1706 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:53.848104  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:53.849257  1630 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.849560  1630 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:53.848047  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:53.854000  1553 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.854116  1630 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:53.861868  1553 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:53.864409  1627 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:53.867175  3048 raft_consensus.cc:993] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:53.867437  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:53.867695  3048 raft_consensus.cc:1081] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:53.868697  3048 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.851720  1554 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.880373  1554 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 12:59:53.883333  1632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 12:59:53.883996  1632 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.884356  1632 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.886585  1630 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.895227  1632 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:53.900544  1632 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:53.869222  3048 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:53.903304  3048 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:53.920356  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet 4e2094a808984c84823c1e0dad0a5de0 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 12:59:53.921185  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet 4e2094a808984c84823c1e0dad0a5de0 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-4e2094a808984c84823c1e0dad0a5de0
I20260709 12:59:53.922552  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:53.923254  1705 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.923578  1705 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:53.924636  1705 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:53.926080  1705 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.925158  3020 raft_consensus.cc:359] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:53.926723  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:53.938436  1628 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:53.938637  1553 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.948271  1646 tablet_copy_source_session.cc:215] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:53.949101   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:53.949121  3038 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:53.950634  1627 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:53.950631  3038 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.954316  3068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2094a808984c84823c1e0dad0a5de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:53.954506  3038 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:53.937320  1630 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:53.958551  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:53.957876  1553 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:53.960768  3038 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:53.962131  3038 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.964357  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:53.964942  1706 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.963837  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:53.975755  2919 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.976364  2919 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:53.977340  2919 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:53.938194  3020 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:53.986292  3020 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:53.993808  3020 ts_tablet_manager.cc:1434] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.213s	user 0.002s	sys 0.008s
I20260709 12:59:54.019367  1706 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:54.016785  1456 catalog_manager.cc:5697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:54.024439  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:54.029651  3040 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:54.030351  3040 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:54.031349  3040 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:54.047181  3025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59270 (request call id 256) took 1445 ms. Trace:
I20260709 12:59:54.047639  3025 rpcz_store.cc:276] 0709 12:59:52.602012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:52.602190 (+   178us) service_pool.cc:224] Handling call
0709 12:59:52.766838 (+164648us) ts_tablet_manager.cc:934] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
0709 12:59:53.734289 (+967451us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:59:53.906400 (+172111us) tablet_metadata.cc:665] Metadata flushed
0709 12:59:53.906533 (+   133us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:59:54.045557 (+139024us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:59:54.046670 (+  1113us) tablet_replica.cc:206] Creating consensus instance
0709 12:59:54.047015 (+   345us) ts_tablet_manager.cc:989] Replica registered.
0709 12:59:54.047081 (+    66us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":86734,"tablet-copy.queue_time_us":109817,"thread_start_us":197204,"threads_started":2}
I20260709 12:59:54.049595  1552 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:54.062052  1720 tablet_copy_service.cc:342] P a2ba7fc60f9641cdac25f1e016ef91d8: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-70981cbf9c1143309f0f78706fe5931d received from {username='slave'} at 127.0.0.1:52646
I20260709 12:59:54.062611  1720 tablet_copy_service.cc:434] P a2ba7fc60f9641cdac25f1e016ef91d8: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-70981cbf9c1143309f0f78706fe5931d on tablet 70981cbf9c1143309f0f78706fe5931d with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:54.068598  3025 tablet_copy_client.cc:806] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
W20260709 12:59:54.353096  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 275274us
I20260709 12:59:54.374284  1705 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:54.392230  1459 catalog_manager.cc:5697] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:54.392606  3025 tablet_copy_client.cc:670] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
W20260709 12:59:54.447875  3084 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:59:54.536345  3071 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 12:59:54.536973  3071 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:54.561293  1552 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:54.598896  3071 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 12:59:54.595098  3025 tablet_copy_client.cc:538] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:54.611056  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 12:59:54.617125  3068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59270 (request call id 260) took 1407 ms. Trace:
I20260709 12:59:54.631654  1705 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.625552  2917 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:59:54.651263  1705 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.695384  2347 raft_consensus.cc:1217] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:54.695779  1706 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:54.633062  3068 rpcz_store.cc:276] 0709 12:59:53.209697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:53.212612 (+  2915us) service_pool.cc:224] Handling call
0709 12:59:53.682754 (+470142us) ts_tablet_manager.cc:934] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
0709 12:59:53.952112 (+269358us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:59:54.392789 (+440677us) tablet_metadata.cc:665] Metadata flushed
0709 12:59:54.404157 (+ 11368us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:59:54.616288 (+212131us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:59:54.616630 (+   342us) tablet_replica.cc:206] Creating consensus instance
0709 12:59:54.616994 (+   364us) ts_tablet_manager.cc:989] Replica registered.
0709 12:59:54.617060 (+    66us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":5433,"tablet-copy.queue_time_us":417293,"thread_start_us":321770,"threads_started":2}
I20260709 12:59:54.699632  3068 tablet_copy_client.cc:806] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:54.700274  3068 tablet_copy_client.cc:670] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:54.708035  3093 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:54.728201   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:54.736330  1630 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.749583  3068 tablet_copy_client.cc:538] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:54.749259  3086 raft_consensus.cc:1064] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:54.782956  1630 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.797626  3025 tablet_bootstrap.cc:492] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:54.799708  3076 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 12:59:54.800395  3076 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:54.821781  3076 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:54.826100  3012 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:54.828604  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 12:59:54.829385  1553 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.831815  1557 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.832065  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:54.833640  3012 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:54.834009  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:54.835233  3094 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:54.835964  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 12:59:54.846066  3031 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:54.844676  3095 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:59:54.855026  1557 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.872342  1552 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.877276  3033 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:59:54.880908  2865 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:59:54.908720  3086 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:54.890074  3085 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:59:54.931874  1457 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 225) took 1128 ms. Trace:
W20260709 12:59:54.934465  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 57475.9us
I20260709 12:59:54.938179  1675 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:54.945744  3068 tablet_bootstrap.cc:492] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 12:59:54.942649  3109 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:55.040551  1456 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 128) took 1034 ms. Trace:
I20260709 12:59:54.932691  1457 rpcz_store.cc:276] 0709 12:59:53.803064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:53.803391 (+   327us) service_pool.cc:224] Handling call
0709 12:59:54.931774 (+1128383us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:59:54.059731 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:59:54.537128 (+477397us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b5c001ffe0c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d8cc331 00007fda2d8cc221 00007fda2d8c6f6c 00007fda2b124d69 00007fda2b1100a6 00007fda2b10fd58 00007fda2b1149f1 00007fda2b1149a9 00007fda2b114939 00007fda2b114901 00007fda2b113b2d
0709 12:59:54.537208 (+    80us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:59:54.537244 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:59:54.537270 (+    26us) tablet.cc:662] Decoding operations
0709 12:59:54.542651 (+  5381us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:59:54.545184 (+  2533us) write_op.cc:641] Partition lock acquired for write op
0709 12:59:54.551318 (+  6134us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:59:54.564426 (+ 13108us) tablet.cc:701] Row locks acquired
0709 12:59:54.584294 (+ 19868us) write_op.cc:260] PREPARE: finished
0709 12:59:54.595308 (+ 11014us) write_op.cc:270] Start()
0709 12:59:54.606398 (+ 11090us) write_op.cc:276] Timestamp: P: 1783601994595241 usec, L: 0
0709 12:59:54.606438 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 12:59:54.607326 (+   888us) log.cc:844] Serialized 541 byte log entry
0709 12:59:54.795261 (+187935us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b18001e4de0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 12:59:54.895667 (+100406us) op_driver.cc:464] REPLICATION: finished
0709 12:59:54.924460 (+ 28793us) write_op.cc:301] APPLY: starting
0709 12:59:54.924561 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 12:59:54.924675 (+   114us) tablet.cc:1370] finished BulkCheckPresence
0709 12:59:54.924695 (+    20us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:59:54.924963 (+   268us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:59:54.925159 (+   196us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:59:54.925225 (+    66us) write_op.cc:312] APPLY: finished
0709 12:59:54.925451 (+   226us) log.cc:844] Serialized 23 byte log entry
0709 12:59:54.925649 (+   198us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:59:54.925725 (+    76us) write_op.cc:454] Released schema lock
0709 12:59:54.925764 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 12:59:54.926095 (+   331us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:59:54.926122 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":28571,"apply.run_cpu_time_us":1639,"apply.run_wall_time_us":1639,"num_ops":1,"prepare.queue_time_us":216860,"prepare.run_cpu_time_us":24464,"prepare.run_wall_time_us":664944,"raft.queue_time_us":202167,"raft.run_cpu_time_us":27640,"raft.run_wall_time_us":119364,"replication_time_us":289068,"spinlock_wait_cycles":8375040,"thread_start_us":343281,"threads_started":4,"wal-append.queue_time_us":131619}]]}
W20260709 12:59:55.057703  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 164662us
I20260709 12:59:55.041277  1456 rpcz_store.cc:276] 0709 12:59:54.006048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:54.006161 (+   113us) service_pool.cc:224] Handling call
0709 12:59:55.040449 (+1034288us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:59:54.768117 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:59:54.768720 (+   603us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:59:54.768751 (+    31us) write_op.cc:435] Acquired schema lock
0709 12:59:54.768768 (+    17us) tablet.cc:662] Decoding operations
0709 12:59:54.769129 (+   361us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:59:54.769177 (+    48us) write_op.cc:641] Partition lock acquired for write op
0709 12:59:54.769206 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:59:54.783479 (+ 14273us) tablet.cc:701] Row locks acquired
0709 12:59:54.783500 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:59:54.783648 (+   148us) write_op.cc:270] Start()
0709 12:59:54.783786 (+   138us) write_op.cc:276] Timestamp: P: 1783601994783597 usec, L: 0
0709 12:59:54.783812 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:59:54.784586 (+   774us) log.cc:844] Serialized 536 byte log entry
0709 12:59:54.924655 (+140069us) op_driver.cc:464] REPLICATION: finished
0709 12:59:54.926424 (+  1769us) write_op.cc:301] APPLY: starting
0709 12:59:54.926503 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 12:59:54.926605 (+   102us) tablet.cc:1370] finished BulkCheckPresence
0709 12:59:54.926703 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:59:54.926943 (+   240us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:59:54.927140 (+   197us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:59:54.927210 (+    70us) write_op.cc:312] APPLY: finished
0709 12:59:54.927440 (+   230us) log.cc:844] Serialized 23 byte log entry
0709 12:59:54.927605 (+   165us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:59:54.927696 (+    91us) write_op.cc:454] Released schema lock
0709 12:59:54.927744 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:59:55.037539 (+109795us) write_op.cc:489] Releasing partition, row and schema locks
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":1409280,"child_traces":[["op",{"apply.queue_time_us":1581,"apply.run_cpu_time_us":3519,"apply.run_wall_time_us":93750,"num_ops":1,"prepare.queue_time_us":723679,"prepare.run_cpu_time_us":2823,"prepare.run_wall_time_us":16800,"replication_time_us":140707,"thread_start_us":31814,"threads_started":1}]]}
W20260709 12:59:55.185144  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 120588us
I20260709 12:59:55.188985  3092 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:59:55.248932  3107 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
W20260709 12:59:55.250582  3111 log.cc:927] Time spent T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:59:55.225576  3104 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
W20260709 12:59:55.353746  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 67477.8us
W20260709 12:59:55.436929  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 66609.2us
W20260709 12:59:55.495908  3113 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 12:59:55.497997  3106 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
W20260709 12:59:55.643447  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 171887us
W20260709 12:59:55.659384  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 80570us
W20260709 12:59:55.756636  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 112233us
I20260709 12:59:55.772957  1708 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:55.787719  2348 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:55.791662  1458 catalog_manager.cc:5697] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 1 to 2, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 12:59:55.816141  3071 raft_consensus.cc:1064] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:55.817538  3036 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:55.819199  3071 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:55.869416  3141 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:59:55.935935  3025 tablet_bootstrap.cc:492] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:55.951972  3025 tablet_bootstrap.cc:492] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 12:59:55.959949  3025 ts_tablet_manager.cc:1403] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 1.175s	user 0.038s	sys 0.017s
W20260709 12:59:55.960718  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:59:55.992494  3025 raft_consensus.cc:359] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:56.003378  3025 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:56.004597  3025 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:56.051369  1459 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 221) took 1666 ms. Trace:
W20260709 12:59:56.087833  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 93052.7us
I20260709 12:59:56.204741  3025 ts_tablet_manager.cc:1434] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.244s	user 0.008s	sys 0.000s
I20260709 12:59:56.013899  3068 tablet_bootstrap.cc:492] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:56.201766  1459 rpcz_store.cc:276] 0709 12:59:54.384508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:54.384671 (+   163us) service_pool.cc:224] Handling call
0709 12:59:56.050766 (+1666095us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:59:54.785154 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:59:54.785760 (+   606us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:59:54.785795 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:59:54.785810 (+    15us) tablet.cc:662] Decoding operations
0709 12:59:54.786190 (+   380us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:59:54.786246 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 12:59:54.786279 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:59:54.786529 (+   250us) tablet.cc:701] Row locks acquired
0709 12:59:54.786552 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:59:54.786755 (+   203us) write_op.cc:270] Start()
0709 12:59:54.799950 (+ 13195us) write_op.cc:276] Timestamp: P: 1783601994786671 usec, L: 0
0709 12:59:54.799986 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:59:54.816484 (+ 16498us) log.cc:844] Serialized 541 byte log entry
0709 12:59:55.049442 (+232958us) op_driver.cc:464] REPLICATION: finished
0709 12:59:55.171090 (+121648us) write_op.cc:301] APPLY: starting
0709 12:59:55.183596 (+ 12506us) tablet.cc:1367] starting BulkCheckPresence
0709 12:59:55.215145 (+ 31549us) tablet.cc:1370] finished BulkCheckPresence
0709 12:59:55.237423 (+ 22278us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:59:55.270082 (+ 32659us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:59:55.361857 (+ 91775us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:59:55.370219 (+  8362us) write_op.cc:312] APPLY: finished
0709 12:59:55.450068 (+ 79849us) log.cc:844] Serialized 23 byte log entry
0709 12:59:55.458810 (+  8742us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:59:55.465048 (+  6238us) write_op.cc:454] Released schema lock
0709 12:59:55.469095 (+  4047us) write_op.cc:341] FINISH: Updating metrics
0709 12:59:55.543766 (+ 74671us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:59:55.545953 (+  2187us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":312056,"reported_tablets":1,"spinlock_wait_cycles":75136,"child_traces":[["op",{"apply.queue_time_us":6878,"apply.run_cpu_time_us":3115,"apply.run_wall_time_us":319830,"mutex_wait_us":1542,"num_ops":1,"prepare.queue_time_us":379610,"prepare.run_cpu_time_us":3341,"prepare.run_wall_time_us":31635,"replication_time_us":236997}]]}
I20260709 12:59:56.317027  3068 tablet_bootstrap.cc:492] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
W20260709 12:59:56.333465  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43592.4us
I20260709 12:59:56.338464  3068 ts_tablet_manager.cc:1403] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 1.393s	user 0.036s	sys 0.026s
I20260709 12:59:56.357718  2353 raft_consensus.cc:1217] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
W20260709 12:59:56.382465  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 406618us
I20260709 12:59:56.454303  1627 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:56.455379  3133 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 12:59:56.456187  3133 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:56.513764  1701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0adeaf85877a4103bbd070927413d267"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:56.514858  1701 raft_consensus.cc:606] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:56.515319  1701 raft_consensus.cc:614] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:59:56.453243  3068 raft_consensus.cc:359] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:56.551550  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:56.552295  1627 raft_consensus.cc:606] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:56.546496  3068 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 12:59:56.584362  3068 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:56.594854  1633 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:56.601156  2348 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:56.612176  3087 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:56.612668  1708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:56.613334  1708 raft_consensus.cc:606] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:56.621248  3068 ts_tablet_manager.cc:1434] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.277s	user 0.008s	sys 0.006s
I20260709 12:59:56.661279  1646 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-e6c211dcb5c947719413e384c6a72769 received from {username='slave'} at 127.0.0.1:48246
I20260709 12:59:56.662081  1646 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-e6c211dcb5c947719413e384c6a72769 on tablet e6c211dcb5c947719413e384c6a72769 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:56.684267  3133 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:56.709641  3074 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:56.773085  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 12:59:56.779381  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 12:59:56.791800  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 52581.8us
I20260709 12:59:56.802145  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:56.803177  1627 raft_consensus.cc:606] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:56.803792  1646 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-4e2094a808984c84823c1e0dad0a5de0 received from {username='slave'} at 127.0.0.1:48246
I20260709 12:59:56.804311  1646 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-4e2094a808984c84823c1e0dad0a5de0 on tablet 4e2094a808984c84823c1e0dad0a5de0 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:56.813136  3076 raft_consensus.cc:2955] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:56.833760  3087 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:59:56.858148  3062 raft_consensus.cc:993] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:56.865263  3062 raft_consensus.cc:1081] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:56.879688  1601 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:56.883972  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 12:59:56.884825  1559 raft_consensus.cc:606] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:56.887692   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:56.890916   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:56.912868  1709 raft_consensus.cc:2955] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 12:59:56.928364  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 53532.3us
I20260709 12:59:56.924522  3133 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:56.940948  3033 raft_consensus.cc:993] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:56.946094  3033 raft_consensus.cc:1081] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:56.950138  1708 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46224
I20260709 12:59:56.950851  1708 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.951211  1708 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.972285  1708 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:56.973073  1458 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 221) took 1320 ms. Trace:
I20260709 12:59:56.981513  1708 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:56.982515  1457 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:56.980439  1634 raft_consensus.cc:2955] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 12:59:56.992319  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 63142.4us
W20260709 12:59:57.001762  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 71632us
I20260709 12:59:56.973582  1458 rpcz_store.cc:276] 0709 12:59:55.652116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:55.652256 (+   140us) service_pool.cc:224] Handling call
0709 12:59:56.972854 (+1320598us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:59:56.319365 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:59:56.603697 (+284332us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b5c001ffe0c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d8cc331 00007fda2d8cc221 00007fda2d8c6f6c 00007fda2b124d69 00007fda2b1100a6 00007fda2b10fd58 00007fda2b1149f1 00007fda2b1149a9 00007fda2b114939 00007fda2b114901 00007fda2b113b2d
0709 12:59:56.603773 (+    76us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:59:56.603815 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:59:56.603829 (+    14us) tablet.cc:662] Decoding operations
0709 12:59:56.608914 (+  5085us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:59:56.609811 (+   897us) write_op.cc:641] Partition lock acquired for write op
0709 12:59:56.609854 (+    43us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:59:56.610285 (+   431us) tablet.cc:701] Row locks acquired
0709 12:59:56.610337 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:59:56.610561 (+   224us) write_op.cc:270] Start()
0709 12:59:56.610848 (+   287us) write_op.cc:276] Timestamp: P: 1783601996610495 usec, L: 0
0709 12:59:56.610891 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 12:59:56.611953 (+  1062us) log.cc:844] Serialized 541 byte log entry
0709 12:59:56.876214 (+264261us) op_driver.cc:464] REPLICATION: finished
0709 12:59:56.957892 (+ 81678us) write_op.cc:301] APPLY: starting
0709 12:59:56.958043 (+   151us) tablet.cc:1367] starting BulkCheckPresence
0709 12:59:56.958184 (+   141us) tablet.cc:1370] finished BulkCheckPresence
0709 12:59:56.958226 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:59:56.965832 (+  7606us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:59:56.966140 (+   308us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:59:56.966201 (+    61us) write_op.cc:312] APPLY: finished
0709 12:59:56.966559 (+   358us) log.cc:844] Serialized 23 byte log entry
0709 12:59:56.967579 (+  1020us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:59:56.967671 (+    92us) write_op.cc:454] Released schema lock
0709 12:59:56.967727 (+    56us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":137274,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":81301,"apply.run_cpu_time_us":2452,"apply.run_wall_time_us":22132,"num_ops":1,"prepare.queue_time_us":473460,"prepare.run_cpu_time_us":96190,"prepare.run_wall_time_us":441421,"raft.queue_time_us":35921,"raft.run_cpu_time_us":1628,"raft.run_wall_time_us":50861,"replication_time_us":264964,"spinlock_wait_cycles":2897280,"thread_start_us":643969,"threads_started":4,"wal-append.queue_time_us":188719}]]}
I20260709 12:59:57.014183  1631 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 12:59:57.015021  1631 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.015616  1631 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:57.027302  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:57.027890  2347 raft_consensus.cc:1217] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:57.028235  1629 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.029582  2348 raft_consensus.cc:2955] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.028658  1629 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 12:59:57.058887  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 48096.7us
I20260709 12:59:57.057444  1631 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:57.057444  1629 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:57.085675  1631 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:57.143947  3146 raft_consensus.cc:993] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:57.144493  3146 raft_consensus.cc:1081] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 12:59:57.094427  1629 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.097821  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:57.150173  1708 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:57.150492  1708 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:57.151213  1708 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:57.152431  1708 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:57.098042  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:57.154857  1560 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:57.118920  3147 raft_consensus.cc:1064] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:57.161705  3147 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:57.184430  1709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46224
I20260709 12:59:57.185149  1709 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.185640  1709 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:57.193222  2347 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.195559  3146 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.203634  1705 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.138135  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:57.214852  1559 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.215188  3134 raft_consensus.cc:993] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:57.130316  3169 raft_consensus.cc:2955] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.134298  1456 catalog_manager.cc:5697] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:57.215708  3134 raft_consensus.cc:1081] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:57.220417  1631 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54748
I20260709 12:59:57.221056  1631 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.221457  1631 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:59:57.129400  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 81784us
I20260709 12:59:57.232226  1629 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:57.235297  1636 raft_consensus.cc:2955] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.240895  1560 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 12:59:57.232334  3146 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:59:57.244813  1708 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 12:59:57.246642  2352 raft_consensus.cc:2955] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.262066  1559 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 12:59:57.262048  1709 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:57.263127  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:57.275183  1709 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 12:59:57.276799  2919 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:57.278412  2919 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:57.285012  1674 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:57.305512  1555 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.307094  3131 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.279696  2919 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:57.307757  3131 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:57.308899  3131 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:57.314522  2862 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.315873  1558 raft_consensus.cc:2955] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.287739  1631 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:57.354172  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:57.354873  1561 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.355222  1561 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:57.356254  1561 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 12:59:57.357930  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 66676.5us
I20260709 12:59:57.361057  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:57.357268  1561 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.364368  1555 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:57.368443  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:57.377828  1634 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:57.378229  1634 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:57.379027  1634 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:57.380651  1634 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:57.391582  1634 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 12:59:57.401688  1561 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:57.397114  1555 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 12:59:57.403406  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:57.404942  3092 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.405507  3092 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:57.406471  3092 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:57.417167  1675 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:57.422469  3147 raft_consensus.cc:2955] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.432154  3071 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:57.432876  3071 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:57.434315  3071 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:57.434762  1706 raft_consensus.cc:2955] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.432694  1554 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:57.489403  1631 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 12:59:57.513453  2346 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:57.516176  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 12:59:57.516392  2877 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.516988  2347 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.569746  1560 raft_consensus.cc:2955] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.601866  3009 raft_consensus.cc:2955] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.602139  1455 catalog_manager.cc:5697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.622951  1702 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:57.643117  2348 raft_consensus.cc:2955] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.646912  2352 raft_consensus.cc:2955] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.652133  2877 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.660564  1710 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:57.666453  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 22f9010f94804fcaacae8539ddfc64a7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:59:57.670547  3177 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:59:57.693341  1634 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:57.695142  3169 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:57.696745  2349 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:57.697824  1706 raft_consensus.cc:2955] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.702968  3170 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.705149  3170 raft_consensus.cc:2955] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.725644  2352 raft_consensus.cc:2955] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.729635  2347 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:57.728717  1631 raft_consensus.cc:2955] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.730139  1560 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:57.737512  3136 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:57.743562  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 70981cbf9c1143309f0f78706fe5931d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:57.759459  1456 catalog_manager.cc:5697] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:57.761374  1561 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:57.763197  3133 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:57.769042  1459 catalog_manager.cc:5697] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.789111  1455 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 2 to 3, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 3 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.790709  1621 tablet_service.cc:1558] Processing DeleteTablet for tablet 22f9010f94804fcaacae8539ddfc64a7 with delete_type TABLET_DATA_TOMBSTONED (TS ba0548b7d2a94e8598475a46c279fe0e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54730
I20260709 12:59:57.791155  1455 catalog_manager.cc:5697] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.797780  1634 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:57.801219  3170 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:57.828938  3188 tablet_replica.cc:333] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 12:59:57.829116  1706 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:57.845356  3188 raft_consensus.cc:2243] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:57.847875  3188 raft_consensus.cc:2272] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:57.832727  2348 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Advancing to term 2
I20260709 12:59:57.847646  2862 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:57.875792  3188 ts_tablet_manager.cc:1916] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:57.884992  2348 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:57.887238  3169 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:57.887735  2347 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:57.889438  2862 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:57.830505  1459 catalog_manager.cc:5697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 2 to 3, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:57.892010  1459 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 2, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.909966   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:57.924810  3136 raft_consensus.cc:1064] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 12:59:57.928313  3136 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:57.941052  3188 ts_tablet_manager.cc:1929] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:57.941788  3188 log.cc:1199] T 22f9010f94804fcaacae8539ddfc64a7 P ba0548b7d2a94e8598475a46c279fe0e: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/wals/22f9010f94804fcaacae8539ddfc64a7
I20260709 12:59:57.942132  1554 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:57.944104  1444 catalog_manager.cc:5028] TS ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): tablet 22f9010f94804fcaacae8539ddfc64a7 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:57.947897  2919 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:57.953323  2349 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.948668  1456 catalog_manager.cc:5697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.957264  1631 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.958413  3169 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:57.963624  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:57.964318  1710 raft_consensus.cc:606] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:59:57.966478  3088 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.974498  1542 tablet_service.cc:1558] Processing DeleteTablet for tablet 70981cbf9c1143309f0f78706fe5931d with delete_type TABLET_DATA_TOMBSTONED (TS d62ce354d09a4be59c29094425d09bc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47096
I20260709 12:59:57.976956  3071 raft_consensus.cc:993] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:57.977572  3071 raft_consensus.cc:1081] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Signalling peer 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 12:59:57.978601  3071 raft_consensus.cc:2955] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:57.983268  1554 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.985576  1558 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:57.985812  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:58.000329  1634 raft_consensus.cc:2955] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:58.008945  3196 tablet_replica.cc:333] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 12:59:58.010181  3088 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:58.011984  3088 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:58.027386  1560 raft_consensus.cc:2955] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:58.034026  3196 raft_consensus.cc:2243] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:58.036223  3196 raft_consensus.cc:2272] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:58.036417  1710 raft_consensus.cc:606] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Received request to transfer leadership
W20260709 12:59:58.042887  1445 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 4e2094a808984c84823c1e0dad0a5de0 already in progress [suppressed 5 similar messages]
I20260709 12:59:58.042059  2352 raft_consensus.cc:2955] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:58.045220   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:58.055989  2349 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:59274
I20260709 12:59:58.075515  3196 ts_tablet_manager.cc:1916] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:58.077380  1706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:58.078990  1631 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:58.081542  1706 raft_consensus.cc:606] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:59:58.088673  3071 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:58.096158  1458 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 2 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:58.112424  2349 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:58.112911  2349 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:58.167106  1634 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 12:59:58.173890  1507 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:59:58.174477  1507 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 4e2094a808984c84823c1e0dad0a5de0 already in progress
W20260709 12:59:58.174772  1507 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 4e2094a808984c84823c1e0dad0a5de0 already in progress
W20260709 12:59:58.222347  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 66680.3us
I20260709 12:59:58.233902  1705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:58.234794  1705 raft_consensus.cc:606] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:58.202561  2349 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 12:59:58.245669   431 auto_leader_rebalancer.cc:438] leader transfer for tablet 4e2094a808984c84823c1e0dad0a5de0 from a2ba7fc60f9641cdac25f1e016ef91d8 to 95fc6a4cf5874b7eb91280dffcd3b3fb failed: Remote error: Service unavailable: leadership transfer for 4e2094a808984c84823c1e0dad0a5de0 already in progress
I20260709 12:59:58.259552  2349 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:58.262094  3196 ts_tablet_manager.cc:1929] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:58.262833  3196 log.cc:1199] T 70981cbf9c1143309f0f78706fe5931d P d62ce354d09a4be59c29094425d09bc1: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/wals/70981cbf9c1143309f0f78706fe5931d
I20260709 12:59:58.264263  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:58.264979  1558 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:58.279572  1558 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 12:59:58.292315  2352 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:58.295240  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:58.295915  1631 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:58.297955  1445 catalog_manager.cc:5028] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): tablet 70981cbf9c1143309f0f78706fe5931d (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:58.301381  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:58.302232  1705 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:58.302543  1705 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:58.303412  1705 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:58.305111  1705 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:58.314213  1631 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 12:59:58.316673  1706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:58.317323  1706 raft_consensus.cc:606] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:58.321053  2317 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:58.295258  3092 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:59:58.322057  3205 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:58.322579  3205 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 12:59:58.323565  3205 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:58.352878  1705 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
W20260709 12:59:58.358678  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 87165us
I20260709 12:59:58.362849  3091 raft_consensus.cc:2955] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:58.370566  1558 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:58.366732  2352 raft_consensus.cc:2955] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:58.374370  1554 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "946eea23a2334d678895281941dd9926"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 12:59:58.375124  1554 raft_consensus.cc:606] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 12:59:58.387782  1458 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 3, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 3 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:58.407651  3147 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:58.422698  3182 raft_consensus.cc:993] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:58.423286  3182 raft_consensus.cc:1081] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 12:59:58.425458  1634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "946eea23a2334d678895281941dd9926"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54748
I20260709 12:59:58.426141  1634 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:58.426525  1634 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:58.444991  3169 raft_consensus.cc:993] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 12:59:58.451371  3169 raft_consensus.cc:1081] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Signalling peer d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 12:59:58.460758  1558 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:47110
I20260709 12:59:58.463405  1558 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:58.463975  1558 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:58.472270  1457 catalog_manager.cc:5697] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 4 to 5, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 12:59:58.460040  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55728.9us
I20260709 12:59:58.485958  1635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:58.493814  1558 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:58.501428  1634 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:58.505995  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e6c211dcb5c947719413e384c6a72769 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:58.518636  1634 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 12:59:58.521667  1694 tablet_service.cc:1558] Processing DeleteTablet for tablet e6c211dcb5c947719413e384c6a72769 with delete_type TABLET_DATA_TOMBSTONED (TS a2ba7fc60f9641cdac25f1e016ef91d8 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:58.523146  1635 raft_consensus.cc:606] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:58.523255  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 12:59:58.524238  1635 raft_consensus.cc:630] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Rejecting request to transfer leadership because tablet server a2ba7fc60f9641cdac25f1e016ef91d8 is not a voter in the active config
I20260709 12:59:58.524338  2352 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:58.530146  1561 raft_consensus.cc:2955] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:58.531670  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 12:59:58.532289  1634 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:58.537431   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:59:58.539616   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:58.540439  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 12:59:58.540818  3211 tablet_replica.cc:333] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 12:59:58.541137  1706 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:58.541545  1706 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 12:59:58.542388  1706 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:58.543551  1706 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:58.537170  1558 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 12:59:58.557435  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 12:59:58.558185  1554 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:58.558538  1554 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:58.559491  1554 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:58.561276  1554 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:58.563659  1706 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 12:59:58.565927  2352 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:58.567829  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 12:59:58.569947  3210 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:58.570454  3210 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 12:59:58.571659  3210 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:58.561365  1634 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 12:59:58.574425  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 12:59:58.575021  3211 raft_consensus.cc:2243] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:58.576076  3211 raft_consensus.cc:2272] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:58.576694  3130 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:58.577318  3130 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 12:59:58.581641  3211 ts_tablet_manager.cc:1916] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:58.599452  1554 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 12:59:58.602962  3130 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 12:59:58.614694  3211 ts_tablet_manager.cc:1929] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:59:58.615383  3211 log.cc:1199] T e6c211dcb5c947719413e384c6a72769 P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/wals/e6c211dcb5c947719413e384c6a72769
I20260709 12:59:58.617375  1445 catalog_manager.cc:5028] TS a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): tablet e6c211dcb5c947719413e384c6a72769 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:58.637844  1457 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:58.651403  1455 catalog_manager.cc:5697] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 3 to 4, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:58.739091  1554 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:59:58.740933  3183 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:58.905112  1706 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:59:58.906910  3205 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:58.912493  1634 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:59:58.924011  3213 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:59.003367  1705 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:59.005354  3182 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:59.025429  1634 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:59.033218  3134 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:59.085668  1561 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:59.087779  3209 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:59.106809  2352 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:59.108503  3209 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:59.540637   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:59.616509  2352 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 12:59:59.622586  1634 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:59:59.626169  3213 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:59:59.636417  1561 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:59:59.636027  3183 raft_consensus.cc:2955] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:59.637996  3205 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:59:59.643401  1561 raft_consensus.cc:2955] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:59.658057  1446 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4e2094a808984c84823c1e0dad0a5de0 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:59.657169  1455 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 4 to 6, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New cstate: current_term: 3 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:59.660027   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:59.685354  1634 raft_consensus.cc:2955] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 12:59:59.792969  1635 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:59.823004  1636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:59.823716  1636 raft_consensus.cc:606] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:59.829012  1634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:59.829757  1634 raft_consensus.cc:606] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 12:59:59.878882  1694 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e2094a808984c84823c1e0dad0a5de0 with delete_type TABLET_DATA_TOMBSTONED (TS a2ba7fc60f9641cdac25f1e016ef91d8 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:46194
I20260709 12:59:59.884171  3231 tablet_replica.cc:333] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 12:59:59.887945  3231 raft_consensus.cc:2243] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:59.888463  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 12:59:59.889249  1710 raft_consensus.cc:606] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 12:59:59.890065  3231 raft_consensus.cc:2272] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:59.897027  1558 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:59.898296  1561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:41400
I20260709 12:59:59.899320  1561 raft_consensus.cc:606] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:59.898802  1705 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:59.899945  3227 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:59.903443  1635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 12:59:59.904134  1635 raft_consensus.cc:606] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 12:59:59.905625   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:59.906749  3226 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:59.907858   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 12:59:59.908411  1602 consensus_peers.cc:597] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ce507f8325ba41dda6865b9492bfd0a7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:59.910506  3231 ts_tablet_manager.cc:1916] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:59.921789  3209 raft_consensus.cc:2955] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:59.923791  1705 raft_consensus.cc:2955] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:59.937719  1558 raft_consensus.cc:2955] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:59.953228  3231 ts_tablet_manager.cc:1929] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:59:59.953819  3231 log.cc:1199] T 4e2094a808984c84823c1e0dad0a5de0 P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/wals/4e2094a808984c84823c1e0dad0a5de0
I20260709 12:59:59.955684  1445 catalog_manager.cc:5028] TS a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): tablet 4e2094a808984c84823c1e0dad0a5de0 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 12:59:59.962417  1459 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 12:59:59.978156  1636 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 12:59:59.999482  3169 raft_consensus.cc:993] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 13:00:00.000097  3169 raft_consensus.cc:1081] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 13:00:00.002789  1561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:47110
I20260709 13:00:00.004432  1705 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:00:00.004541  1602 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6c08bb7c6294cf9b33ad3b85e34831d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:00.006388  3226 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:00.009703  1561 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.010233  1561 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.015350  1560 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:00:00.017452  3226 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:00.022118  1709 raft_consensus.cc:2955] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:00.024188  1561 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:00.026777  1561 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:00.021118  3210 raft_consensus.cc:2955] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:00.027719  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:00.027877  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:00.028508  1636 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.028535  1706 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:00.029062  1706 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:00.029984  1706 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:00.031113  3209 raft_consensus.cc:993] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 13:00:00.031217  1706 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.031848  3227 raft_consensus.cc:1081] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Signalling peer 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 13:00:00.034039  2352 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:59270
I20260709 13:00:00.034634  2352 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.034972  2352 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.047506  2352 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:00.048921  1706 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:00.050453  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:00.051355  1636 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:00.051488  3182 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:00.051497  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:00.052363  1554 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.052474  3182 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 2 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:00.053536  3182 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:00.062184  2352 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:00.066530  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:00.067238  1636 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:00.067549  1636 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:00.068411  1636 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:00.076989  1636 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.078428  1560 raft_consensus.cc:2955] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:00.083747  1456 catalog_manager.cc:5697] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.102941  1558 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:00.114038  1554 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:00.134786  1636 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:00.142102  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:00.146580  3205 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:00.147185  3205 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:00.148124  3205 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:00.154665  1706 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:00.156282  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 49239.5us
W20260709 13:00:00.157866  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51864.3us
I20260709 13:00:00.166139  3238 raft_consensus.cc:993] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:00.166754  3238 raft_consensus.cc:1081] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:00.167217  1634 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:00.169466  1706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46210
I20260709 13:00:00.170125  1706 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.170594  1706 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.184063  3239 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:00.193221  1527 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 83a61af0ea7d47c1bc0ee6a94a2d1a7f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:00.194712  1456 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.199344  3227 raft_consensus.cc:993] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:00.200093  3227 raft_consensus.cc:1081] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:00.202672  1710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46224
I20260709 13:00:00.203778  1710 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.204208  1710 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.200490  1706 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:00.209116  3134 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:00.218406  1459 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:00.234091  1710 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:00.236811  1710 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:00.239183  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:00.241801  1560 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.239744  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:00.245967  1634 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:00.251549  3240 raft_consensus.cc:2955] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:00.256220  1706 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:00.246471  1634 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:00.257968  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:00.259370  2349 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.261354  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:00.262077  1554 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:00.262398  1554 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:00.262791  1634 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:00.263473  3210 raft_consensus.cc:993] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:00.269939  3210 raft_consensus.cc:1081] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:00.271589  1554 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:00.273104  1554 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.278167  1702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46224
I20260709 13:00:00.278875  1702 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.279246  1702 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.285171  1634 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.264410  1706 raft_consensus.cc:2955] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:00.267987  1560 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:00.304109  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:00.307431  3131 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:00.293296  2349 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:00.308092  3131 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:00.319188  3131 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:00.323278  1702 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:00.324443  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:00.317891  3245 ts_tablet_manager.cc:933] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:00.317425  1710 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:00.328837  3062 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:00.330794  1554 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:00.336091  1702 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:00.336164  1634 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:00.338272  1636 raft_consensus.cc:2955] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:00.343271  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:00.344003  1631 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:00.344312  1631 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:00.348572  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:00.349192  1558 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.351912  3245 tablet_copy_client.cc:323] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
I20260709 13:00:00.354326  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet ce507f8325ba41dda6865b9492bfd0a7 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 13:00:00.354890  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet ce507f8325ba41dda6865b9492bfd0a7 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-ce507f8325ba41dda6865b9492bfd0a7
I20260709 13:00:00.329437  3062 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:00.345083  1631 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:00.372133  1631 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.321645  1455 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.361321  3062 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:00.407210  1459 catalog_manager.cc:5697] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 2, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:00.409706  1646 tablet_copy_source_session.cc:215] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:00.414573  3245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce507f8325ba41dda6865b9492bfd0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:00.416110  1558 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:00.419111  1631 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:00.422502  1634 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:00.425992  1676 consensus_peers.cc:597] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1457f31daeb646a9824ac8068949eddd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:00.426528  1675 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:00.428261  3248 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:00.428788  3248 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:00.428958  3250 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:00.429797  3248 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:00.447561  1631 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:00.450570  3240 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:00.464255  3245 tablet_copy_client.cc:806] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:00.465173  3245 tablet_copy_client.cc:670] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:00.470639  3245 tablet_copy_client.cc:538] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:00.471820  1558 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:00.489408  3245 tablet_bootstrap.cc:492] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 13:00:00.490091  3169 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:00.512176  1706 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:00.515137  3239 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:00.553418  1558 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:00.555243  3245 tablet_bootstrap.cc:492] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:00.556542  3245 tablet_bootstrap.cc:492] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 13:00:00.558516  3245 ts_tablet_manager.cc:1403] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.069s	user 0.031s	sys 0.004s
I20260709 13:00:00.565272  3213 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:00.570598  1459 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.575762  3259 ts_tablet_manager.cc:933] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:00.574782  1459 catalog_manager.cc:5697] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 2 to 3, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:00.595376  3248 raft_consensus.cc:2955] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:00.623198  1634 raft_consensus.cc:2955] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:00.621162  3245 raft_consensus.cc:359] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:00.647293  3245 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 13:00:00.650756  3259 tablet_copy_client.cc:323] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
I20260709 13:00:00.648437  3245 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:00.653662  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet e6c08bb7c6294cf9b33ad3b85e34831d from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 13:00:00.654258  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet e6c08bb7c6294cf9b33ad3b85e34831d from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-e6c08bb7c6294cf9b33ad3b85e34831d
I20260709 13:00:00.660408  1558 raft_consensus.cc:2955] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:00.683169  3245 ts_tablet_manager.cc:1434] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.110s	user 0.006s	sys 0.003s
I20260709 13:00:00.686388  1646 tablet_copy_source_session.cc:215] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:00.689792  1636 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:00.699277  1456 catalog_manager.cc:5697] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.698065  1645 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-ce507f8325ba41dda6865b9492bfd0a7 received from {username='slave'} at 127.0.0.1:48246
I20260709 13:00:00.701148  1645 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-ce507f8325ba41dda6865b9492bfd0a7 on tablet ce507f8325ba41dda6865b9492bfd0a7 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:00.701723  3213 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:00.709719  3245 ts_tablet_manager.cc:933] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 13:00:00.729811  3259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6c08bb7c6294cf9b33ad3b85e34831d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:00.734843  3245 tablet_copy_client.cc:323] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:33149
I20260709 13:00:00.737689  1570 tablet_copy_service.cc:140] P d62ce354d09a4be59c29094425d09bc1: Received BeginTabletCopySession request for tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:42562)
I20260709 13:00:00.738287  1570 tablet_copy_service.cc:161] P d62ce354d09a4be59c29094425d09bc1: Beginning new tablet copy session on tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:42562: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-83a61af0ea7d47c1bc0ee6a94a2d1a7f
I20260709 13:00:00.750407  1570 tablet_copy_source_session.cc:215] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:00.764557  3245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:00.776679  3259 tablet_copy_client.cc:806] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:00.777367  3259 tablet_copy_client.cc:670] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:00.780784  1554 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:00.782707  3248 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:00.791615  1631 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:00.794390  3250 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:00.799274  3259 tablet_copy_client.cc:538] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:00.820394  3259 tablet_bootstrap.cc:492] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 13:00:00.867861  1634 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.882218  2349 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:00.887511  3248 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:00.913233   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:00.931828  1558 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:00.938530  3062 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:00.941215  3131 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:00:00.990841  3269 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:00:01.001223  2352 raft_consensus.cc:1217] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:01.054278  3267 ts_tablet_manager.cc:933] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:01.071085  1561 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:00:01.124706  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 45063.5us
I20260709 13:00:01.133951  3131 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:00:01.241561  3245 tablet_copy_client.cc:806] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:01.242415  3245 tablet_copy_client.cc:670] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:01.210412  3267 tablet_copy_client.cc:323] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45431
I20260709 13:00:01.259652  3245 tablet_copy_client.cc:538] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:01.259675  3259 tablet_bootstrap.cc:492] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:01.261020  3259 tablet_bootstrap.cc:492] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 13:00:01.261839  3259 ts_tablet_manager.cc:1403] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.442s	user 0.040s	sys 0.001s
I20260709 13:00:01.265630  3259 raft_consensus.cc:359] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:01.266739  3259 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 13:00:01.267011  1720 tablet_copy_service.cc:140] P a2ba7fc60f9641cdac25f1e016ef91d8: Received BeginTabletCopySession request for tablet 1457f31daeb646a9824ac8068949eddd from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:52646)
I20260709 13:00:01.267768  1720 tablet_copy_service.cc:161] P a2ba7fc60f9641cdac25f1e016ef91d8: Beginning new tablet copy session on tablet 1457f31daeb646a9824ac8068949eddd from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:52646: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-1457f31daeb646a9824ac8068949eddd
I20260709 13:00:01.287859  1720 tablet_copy_source_session.cc:215] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:01.306403  3245 tablet_bootstrap.cc:492] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 13:00:01.308514  3267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1457f31daeb646a9824ac8068949eddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:01.352291   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:01.357347  3245 tablet_bootstrap.cc:492] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:01.358960  3245 tablet_bootstrap.cc:492] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 13:00:01.359792  3245 ts_tablet_manager.cc:1403] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.055s	user 0.023s	sys 0.007s
I20260709 13:00:01.365628  3245 raft_consensus.cc:359] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:01.366739  3245 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 13:00:01.367759  3245 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:01.375160  3245 ts_tablet_manager.cc:1434] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.003s
I20260709 13:00:01.377802  1570 tablet_copy_service.cc:342] P d62ce354d09a4be59c29094425d09bc1: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-83a61af0ea7d47c1bc0ee6a94a2d1a7f received from {username='slave'} at 127.0.0.1:42562
I20260709 13:00:01.378316  1570 tablet_copy_service.cc:434] P d62ce354d09a4be59c29094425d09bc1: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-83a61af0ea7d47c1bc0ee6a94a2d1a7f on tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:01.267632  3259 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:01.394192  3259 ts_tablet_manager.cc:1434] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.132s	user 0.010s	sys 0.000s
I20260709 13:00:01.421461  1646 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-e6c08bb7c6294cf9b33ad3b85e34831d received from {username='slave'} at 127.0.0.1:48246
I20260709 13:00:01.422055  1646 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-e6c08bb7c6294cf9b33ad3b85e34831d on tablet e6c08bb7c6294cf9b33ad3b85e34831d with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:01.489126  2348 raft_consensus.cc:1217] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:00:01.499557  3267 tablet_copy_client.cc:806] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:01.505429  3267 tablet_copy_client.cc:670] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:01.524456  3267 tablet_copy_client.cc:538] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:01.538934  3276 raft_consensus.cc:1064] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:00:01.543016  3276 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:01.569947  2348 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:01.571552  3276 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:01.574630  1561 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:01.584193  3267 tablet_bootstrap.cc:492] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 13:00:01.585160  1710 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:01.586316  1702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:00:01.588259  3276 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:01.592317  1702 raft_consensus.cc:606] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:01.592921  3133 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:01.597651  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:00:01.598567  1710 raft_consensus.cc:606] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:01.614302  1561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:00:01.615386  1561 raft_consensus.cc:606] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 2 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:01.618558  3282 raft_consensus.cc:993] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:01.620699  3280 raft_consensus.cc:1081] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 2 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:01.623641  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:00:01.624414  1710 raft_consensus.cc:606] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:01.626187   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:01.628363   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:01.636144  1710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46210
I20260709 13:00:01.637658  1710 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:01.638036  1710 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.643729  3276 raft_consensus.cc:2955] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:01.648181  1710 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:01.656001  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:01.656692  1561 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:01.656983  1561 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:01.651542  1702 raft_consensus.cc:2955] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:01.657734  1561 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:01.658865  1561 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.658679  1560 raft_consensus.cc:2955] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:01.665846  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:01.666563  1631 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.677872  3267 tablet_bootstrap.cc:492] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:01.682698  3267 tablet_bootstrap.cc:492] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 13:00:01.683612  3267 ts_tablet_manager.cc:1403] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 0.106s	user 0.022s	sys 0.009s
I20260709 13:00:01.684434  1710 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:01.694994  2348 raft_consensus.cc:2955] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:01.698001  1561 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:01.699759  1675 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:01.700779  3248 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:01.701463  3248 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:01.701916  3267 raft_consensus.cc:359] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:01.702373  3248 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:01.702917  3267 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 13:00:01.703677  3267 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:01.706434  3267 ts_tablet_manager.cc:1434] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:00:01.715564  1720 tablet_copy_service.cc:342] P a2ba7fc60f9641cdac25f1e016ef91d8: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-1457f31daeb646a9824ac8068949eddd received from {username='slave'} at 127.0.0.1:52646
I20260709 13:00:01.716117  1720 tablet_copy_service.cc:434] P a2ba7fc60f9641cdac25f1e016ef91d8: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-1457f31daeb646a9824ac8068949eddd on tablet 1457f31daeb646a9824ac8068949eddd with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:01.724743  1456 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:01.747287  1631 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:01.778409  2348 raft_consensus.cc:1217] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:01.818769  3279 raft_consensus.cc:993] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:00:01.819346  3279 raft_consensus.cc:1081] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:00:01.821707  1627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 13:00:01.826231  1627 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:01.829043  1627 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.846990  1634 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:01.891814  1710 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:01.894184  3133 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:01.896467  2348 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:01.913992  1627 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:01.916775  1627 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:01.918798  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:01.917557  1455 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 2 to 3, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:01.921177  1710 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.921561  1710 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:01.922804  1710 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:01.924568  1710 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.921355  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:01.932063  1561 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.939146  3273 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:01.942858  3209 raft_consensus.cc:1064] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:00:01.946484  3209 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:01.949313  3271 raft_consensus.cc:2955] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:01.965658  1702 raft_consensus.cc:2955] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:01.974295  1710 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:01.975801  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:01.977124  3273 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:01.978296  3273 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:01.979490  3273 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:02.002354  1561 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:02.032102  3279 raft_consensus.cc:993] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:00:02.066504  3248 raft_consensus.cc:1081] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:00:02.096370  1455 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:02.129858  1627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 13:00:02.130935  1627 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:02.131546  1627 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.144486  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ce507f8325ba41dda6865b9492bfd0a7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:00:02.158185  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 76595.5us
I20260709 13:00:02.157240  1456 catalog_manager.cc:5697] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:02.193281  1627 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:02.226876  1627 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:02.232766  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 68239.5us
W20260709 13:00:02.368016  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 98551.5us
W20260709 13:00:02.407935  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 137979us
I20260709 13:00:02.424607  3298 raft_consensus.cc:993] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 13:00:02.428462  3298 raft_consensus.cc:1081] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer d62ce354d09a4be59c29094425d09bc1 to start an election
W20260709 13:00:02.479454  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 75937.8us
I20260709 13:00:02.481910  2350 raft_consensus.cc:2955] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:02.500334  2348 raft_consensus.cc:1217] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:02.501735  1555 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:47110
I20260709 13:00:02.502420  1555 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:02.502818  1555 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.520131  1627 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:02.521874  1561 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:02.524531  1555 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:02.530503  1631 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:02.535220  3330 raft_consensus.cc:1064] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:00:02.539490  2348 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:02.553361  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:02.554055  1636 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.558979  3279 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:00:02.565493  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 375399us
I20260709 13:00:02.573164  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:02.573866  1705 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:02.574190  1705 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:02.575407  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:02.576117  1560 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.577350  1706 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:02.566507  1636 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:02.563059  3312 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:02.582408  1705 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:02.585708  3248 consensus_queue.cc:1059] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:02.590003  1705 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.590112  1555 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:02.590515  3319 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:02.598338  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:02.602340  3325 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:02.602932  3325 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 2 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:02.610687  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:02.588835  3330 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:02.632301  1558 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:02.639818  3304 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:02.643098  1701 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:02.603924  3325 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:02.632969  1706 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:02.643797  1706 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:02.644519  1705 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:02.657064  1560 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:02.667433  1706 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:02.667888  1636 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:02.674556  1706 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.682893  3338 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:02.687695  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:02.698616  1561 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:02.702837  1706 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:02.703805  3133 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:02.727466  2348 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:02.731774  3324 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:02.732447  1705 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:02.734933  3306 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:02.714803  3133 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:02.741503  3133 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:02.744029  3338 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:02.748126  3323 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:02.797271   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:02.807862  1559 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 2 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:02.890441  3292 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.055s	user 0.005s	sys 0.003s
W20260709 13:00:03.319483  1710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.022s	sys 0.030s
W20260709 13:00:03.319938  1710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.022s	sys 0.030s
W20260709 13:00:03.313222  3349 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.432s	user 0.002s	sys 0.004s
W20260709 13:00:03.526587  3354 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:00:03.346079  3209 raft_consensus.cc:2955] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:00:03.533430  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.010s	sys 0.054s
W20260709 13:00:03.533906  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.010s	sys 0.054s
I20260709 13:00:03.574616  3324 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:03.598743  3324 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:03.601141  3353 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:00:03.649171  3324 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:03.661315  1455 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 255) took 1574 ms. Trace:
I20260709 13:00:03.698300  2346 raft_consensus.cc:2955] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:03.699892  3334 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:03.700700  3334 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:03.668159  3355 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:00:03.661949  1455 rpcz_store.cc:276] 0709 13:00:02.086860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:02.092566 (+  5706us) service_pool.cc:224] Handling call
0709 13:00:02.243459 (+150893us) spinlock_profiling.cc:243] Waited 3.2 ms on lock 0x7b54001799a8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2b044161 00007fda2b0ce3af 00007fda2b0cfb11 00007fda2d5d02d1 00007fda2d5d0b2f 00007fda2d48fd4b 00007fda2d597c3e 00007fda2ba582a4 00007fda2ba58232 00007fda2ba58161 00007fda2ba580dc
0709 13:00:03.661247 (+1417788us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:02.418665 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:02.419307 (+   642us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:02.419338 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:00:02.419358 (+    20us) tablet.cc:662] Decoding operations
0709 13:00:02.419759 (+   401us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:02.419818 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:02.419853 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:02.420057 (+   204us) tablet.cc:701] Row locks acquired
0709 13:00:02.420077 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:00:02.420229 (+   152us) write_op.cc:270] Start()
0709 13:00:02.420357 (+   128us) write_op.cc:276] Timestamp: P: 1783602002420177 usec, L: 0
0709 13:00:02.420378 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:00:02.455169 (+ 34791us) log.cc:844] Serialized 536 byte log entry
0709 13:00:02.603780 (+148611us) op_driver.cc:464] REPLICATION: finished
0709 13:00:02.709138 (+105358us) write_op.cc:301] APPLY: starting
0709 13:00:02.709235 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:02.709354 (+   119us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:02.713249 (+  3895us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:02.759565 (+ 46316us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:02.776553 (+ 16988us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:02.779669 (+  3116us) write_op.cc:312] APPLY: finished
0709 13:00:02.813001 (+ 33332us) log.cc:844] Serialized 23 byte log entry
0709 13:00:02.825085 (+ 12084us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:02.827015 (+  1930us) write_op.cc:454] Released schema lock
0709 13:00:02.856056 (+ 29041us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:02.862095 (+  6039us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:02.862125 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":45022,"reported_tablets":1,"spinlock_wait_cycles":7366912,"child_traces":[["op",{"apply.queue_time_us":105127,"apply.run_cpu_time_us":4739,"apply.run_wall_time_us":152929,"num_ops":1,"prepare.queue_time_us":174112,"prepare.run_cpu_time_us":3168,"prepare.run_wall_time_us":36837,"replication_time_us":183252,"thread_start_us":104673,"threads_started":1}]]}
W20260709 13:00:03.730468  3352 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.258s	user 0.003s	sys 0.000s
I20260709 13:00:03.755590  3334 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:03.773406  1542 tablet_service.cc:1558] Processing DeleteTablet for tablet ce507f8325ba41dda6865b9492bfd0a7 with delete_type TABLET_DATA_TOMBSTONED (TS d62ce354d09a4be59c29094425d09bc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47096
W20260709 13:00:03.777812  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 83746.9us
I20260709 13:00:03.791683  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:03.798235  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:03.808008  1558 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:00:03.825398  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 128299us
I20260709 13:00:03.825611  1710 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:03.843319  1634 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:03.846346  3271 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:03.854696  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:03.862110  1634 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:00:03.865545  3356 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.405s	user 0.001s	sys 0.001s
I20260709 13:00:03.867179  3389 tablet_replica.cc:333] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:00:03.869911  3385 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:03.870815  3385 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:03.909189  1561 raft_consensus.cc:2955] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:03.913587  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:03.914606  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 49610.9us
W20260709 13:00:03.972790  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 52522us
I20260709 13:00:03.990190  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:03.986508  1527 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:03.984584  1705 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:03.983644  3375 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:03.999833  3375 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:03.932215  3385 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.007874  3387 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:04.008586  3387 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:04.015230  3334 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:04.022346  3317 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.026276  3296 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:04.027024  3296 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:04.034248  3387 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:03.988976  3374 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:00:03.998142  3319 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:04.051112  3305 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:04.023793  3324 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:00:04.031621  3338 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:04.051900  3305 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.060667  3338 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:04.064926  3382 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:04.064971  3328 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:00:04.047756  3304 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:04.066624  1456 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 256) took 1922 ms. Trace:
I20260709 13:00:04.022462  3334 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.080592  3327 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:04.070824  3282 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:00:04.085400  3304 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.092270  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:04.099711  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.085808  3271 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:04.102351  1708 raft_consensus.cc:2955] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:04.101068  3271 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:04.052301  3319 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.085322  1456 rpcz_store.cc:276] 0709 13:00:02.144108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:02.147511 (+  3403us) service_pool.cc:224] Handling call
0709 13:00:04.066554 (+1919043us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:02.205440 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:02.308882 (+103442us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:02.311480 (+  2598us) write_op.cc:435] Acquired schema lock
0709 13:00:02.327440 (+ 15960us) tablet.cc:662] Decoding operations
0709 13:00:02.327918 (+   478us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:02.328232 (+   314us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:02.328363 (+   131us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:02.328644 (+   281us) tablet.cc:701] Row locks acquired
0709 13:00:02.328666 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:00:02.328828 (+   162us) write_op.cc:270] Start()
0709 13:00:02.328972 (+   144us) write_op.cc:276] Timestamp: P: 1783602002328779 usec, L: 0
0709 13:00:02.328998 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:00:02.377016 (+ 48018us) log.cc:844] Serialized 541 byte log entry
0709 13:00:02.469953 (+ 92937us) op_driver.cc:464] REPLICATION: finished
0709 13:00:02.693660 (+223707us) write_op.cc:301] APPLY: starting
0709 13:00:02.695809 (+  2149us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:02.708473 (+ 12664us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:02.757726 (+ 49253us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:02.779239 (+ 21513us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:02.815267 (+ 36028us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:02.821227 (+  5960us) write_op.cc:312] APPLY: finished
0709 13:00:02.838574 (+ 17347us) log.cc:844] Serialized 23 byte log entry
0709 13:00:02.862703 (+ 24129us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:02.862789 (+    86us) write_op.cc:454] Released schema lock
0709 13:00:02.862836 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:02.863274 (+   438us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:02.863302 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":793994,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":204657,"apply.run_cpu_time_us":3830,"apply.run_wall_time_us":173514,"num_ops":1,"prepare.queue_time_us":141,"prepare.run_cpu_time_us":5811,"prepare.run_wall_time_us":200137,"replication_time_us":128852,"thread_start_us":96488,"threads_started":1}]]}
I20260709 13:00:04.088719  3382 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:04.110157  3325 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:04.109329  3375 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:04.115654  3376 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:04.117365  3209 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:04.118453  3320 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:04.168756  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 52308.8us
I20260709 13:00:04.171010  3338 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:04.171800  3338 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:04.161530  3377 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:04.172874  3338 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.158362  3323 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:04.119621  3334 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:04.110988  3327 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:04.152056  3325 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:04.173297  3377 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:04.180666  3379 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:04.182832  3394 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:00:04.182845  3340 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:04.168862  3209 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 13:00:04.170276  3376 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.318907  3279 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.316535  3304 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.319497  3279 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.332929  3271 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:04.338832  3325 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:04.329087  3304 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:04.328730  3339 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:04.356166  3376 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.357396  3364 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:04.179622  3323 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.331323  3296 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:04.358079  3393 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:00:04.185915  3320 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.378834  3320 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.389992  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.393142  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.324239  3379 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.328920  3340 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:04.430832  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.368563  3304 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.448254  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.419674  3382 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:04.361732  3364 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:04.365056  3339 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.439348  3338 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.443621  3319 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:04.480574  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:04.481366  1636 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:04.483080  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:04.483089  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:04.483753  1636 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:04.468592  3209 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 13:00:04.485146  3370 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.485258  3377 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:04.485942  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:04.485738  3370 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.484413  3364 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.490185  3370 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.496872  3379 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.498582  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:04.498706  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.499680  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:04.506575  3279 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:04.501351  3209 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:04.513729  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.514842  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:04.519878  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.522778  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:04.502501  3334 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:04.502800  3327 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:04.502646  3319 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.530450  3319 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:04.527588  3340 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.530902  3319 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.538545  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:04.520131  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.542469  3340 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.540587  1703 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:04.516731  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:04.548178  1674 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:04.550177  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:04.517915  3367 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:04.551083  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:04.523321  3330 raft_consensus.cc:2955] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:04.557873  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:04.529098  3334 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:04.564128  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:04.557075  3390 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.567787  3389 raft_consensus.cc:2243] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.570542  1459 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 168) took 1836 ms. Trace:
I20260709 13:00:04.571020  1459 rpcz_store.cc:276] 0709 13:00:02.734540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:02.758724 (+ 24184us) service_pool.cc:224] Handling call
0709 13:00:04.570466 (+1811742us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1167368,"reported_tablets":1}
I20260709 13:00:04.574095  3389 raft_consensus.cc:2272] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:04.573980  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:04.574915  3340 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.575300  3340 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.575626  3340 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.560400  3367 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.518380  3209 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:04.578277  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.591286  3367 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.591322  3209 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:04.560999  3402 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:00:04.564888  3323 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.595765  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:04.597019  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.605597  2350 raft_consensus.cc:2955] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:04.613862  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:04.620935  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.631809  3389 ts_tablet_manager.cc:1916] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:04.578972  3334 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:04.644289  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.650477  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.647917  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.659955  3339 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.666925  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:04.675696  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:04.677630  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:04.699224  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.700887  1602 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:04.708966  1602 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:04.707443  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:04.714423  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:04.719898  3376 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.722157  3379 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.724432  3319 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:04.725521  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:04.726408  3305 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.726555  1710 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:04.729075  1456 catalog_manager.cc:5697] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 2 to 3, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 3 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:04.730823  1456 catalog_manager.cc:5697] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index 3 to 4, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:04.707821  3377 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.736838  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:04.738574  1561 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:04.744306  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.751622  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:04.753461  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:04.759857  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:04.713702  1707 raft_consensus.cc:2955] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:04.774783  3340 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.795374  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:04.805613  3397 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.784143  3280 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.786311  3319 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:04.775843  3317 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.813297  3304 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.816901   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:04.819756  1701 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:04.820799  1633 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:04.824553  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:04.830226  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:04.827371  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:04.850147  3340 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:04.832298  3373 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.837406  3304 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.831965  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:04.858203  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:04.868366  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 163981us
I20260709 13:00:04.861747  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:04.852835  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:04.900024  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:04.874413  3337 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.852517  3334 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.879504  3327 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:00:04.929945  1702 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:04.890271  3311 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:00:04.934085  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:04.913951  3337 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:04.938051  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:04.929193  1627 raft_consensus.cc:2955] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:04.975426  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:04.975783  3334 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:04.960446  3280 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:04.977023  3280 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:04.977955  3280 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:04.979233  3280 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:04.983906  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:04.987761  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:04.945828  3337 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.967712  3372 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.044718  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.045996  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:05.070446  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 52128.3us
I20260709 13:00:05.071151  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.081707  3330 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:05.082788  3366 raft_consensus.cc:1064] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:00:05.082947  3330 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:05.083356  3330 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:05.087829  3389 ts_tablet_manager.cc:1929] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:05.090602  3389 log.cc:1199] T ce507f8325ba41dda6865b9492bfd0a7 P d62ce354d09a4be59c29094425d09bc1: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/wals/ce507f8325ba41dda6865b9492bfd0a7
I20260709 13:00:05.102541  3389 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47096 (request call id 79) took 1329 ms. Trace:
I20260709 13:00:05.103462  3389 rpcz_store.cc:276] 0709 13:00:03.773039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:03.773197 (+   158us) service_pool.cc:224] Handling call
0709 13:00:03.858124 (+ 84927us) ts_tablet_manager.cc:1151] Deleting tablet ce507f8325ba41dda6865b9492bfd0a7
0709 13:00:03.858202 (+    78us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:00:04.797692 (+939490us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:05.086792 (+289100us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:05.102426 (+ 15634us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":704490,"tablet-delete.queue_time_us":83883,"thread_start_us":55789,"threads_started":1}
I20260709 13:00:05.098331  3330 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.106556  3330 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:05.110033  1445 catalog_manager.cc:5028] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): tablet ce507f8325ba41dda6865b9492bfd0a7 (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 13:00:05.117306  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.126014  3364 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:05.126561  3364 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:05.170329  3364 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:05.175163  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.180716  3377 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:05.180842  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.182564  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.183138  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.183558  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.183197  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.191892  3357 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:05.194168  3357 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.188278  3377 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.201239  3363 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:05.207147  3363 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.207844  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:05.214241  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:05.215579  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:05.225405  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.232340  3357 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:05.230760  3377 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:05.220019  3317 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:05.242624  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:05.253979  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.261618  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:05.262081  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:05.265136  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:05.275288  3304 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.276206  1633 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:05.244452  3317 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.282258  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:05.290861  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:05.291303  1561 raft_consensus.cc:2410] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:05.293769  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:05.295331  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.304255  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:05.293128  3280 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.264472  3366 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:05.303742  3404 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:05.300621  3311 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.298615  3378 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:05.306903  3310 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:05.318863  3310 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.316398  3404 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 13:00:05.350605  3364 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.354385  3363 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:05.362084  1558 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:05.362265  3337 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:05.363539  2350 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:05.317310  3377 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:05.365504  3377 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.367287  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:05.367761  3377 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:05.375444  1559 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:05.376758  3337 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:05.379515  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:05.382094  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:05.383886  3404 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:05.384526  3404 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:05.386137  3310 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:05.386790  3317 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:05.385069  3404 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.390476  3404 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:05.392532  3312 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:05.393693  3312 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:05.394987  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:05.395583  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:05.395756  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:05.396828  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:05.397976  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:05.399839  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:05.401122  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:05.401309  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:05.402102  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:05.404631  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:05.394027  3312 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:05.406236  3404 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:05.407426  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.406291  3312 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:05.409822  3312 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.410665  1675 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.411916  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.412688  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:05.413578  3336 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.414321  3336 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:05.415007  1526 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:05.416574  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:05.417308  3311 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.418049  3311 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.419464  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:05.423040  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.423473  3310 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.428479  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:05.431735  3337 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.432109  3322 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.436867  1675 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:05.440073  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.440755  3408 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:05.442412  3379 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.445281  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.450436  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.449849  3320 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.453418  3367 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.454129  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.455284  3307 raft_consensus.cc:1027] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: attempt to promote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:00:05.460352  3323 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.461357  1627 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:05.438516  3377 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:05.462677  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.458573  2350 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:05.459575  1628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 13:00:05.463691  1628 raft_consensus.cc:606] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:05.452773  3408 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:05.467355  3408 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.469384  1458 catalog_manager.cc:5697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:05.469815  3307 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:05.474539  3366 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:05.478853  1628 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:05.482362  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.483788  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.485795  1704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:00:05.486533  1704 raft_consensus.cc:606] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.490356  1561 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:05.488916  3367 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.502753  1704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:00:05.503435  1704 raft_consensus.cc:606] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:05.506919  3323 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.505066  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.507673  3298 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:05.487905  3320 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:05.512166  1458 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 257) took 2207 ms. Trace:
I20260709 13:00:05.516397  1456 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:05.516398  1704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:00:05.522782  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.523007  1704 raft_consensus.cc:606] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.525192   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:05.528400  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.521162  1458 rpcz_store.cc:276] 0709 13:00:03.304413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:03.304584 (+   171us) service_pool.cc:224] Handling call
0709 13:00:05.512062 (+2207478us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:05.472720 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:05.473273 (+   553us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:05.473307 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:00:05.473319 (+    12us) tablet.cc:662] Decoding operations
0709 13:00:05.473829 (+   510us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:05.474533 (+   704us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:05.474561 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:05.474770 (+   209us) tablet.cc:701] Row locks acquired
0709 13:00:05.474787 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:00:05.474960 (+   173us) write_op.cc:270] Start()
0709 13:00:05.475087 (+   127us) write_op.cc:276] Timestamp: P: 1783602005474899 usec, L: 0
0709 13:00:05.475107 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:00:05.475748 (+   641us) log.cc:844] Serialized 541 byte log entry
0709 13:00:05.477446 (+  1698us) op_driver.cc:464] REPLICATION: finished
0709 13:00:05.478797 (+  1351us) write_op.cc:301] APPLY: starting
0709 13:00:05.478881 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:05.478979 (+    98us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:05.479074 (+    95us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:05.492760 (+ 13686us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:05.496663 (+  3903us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:05.496720 (+    57us) write_op.cc:312] APPLY: finished
0709 13:00:05.496925 (+   205us) log.cc:844] Serialized 23 byte log entry
0709 13:00:05.507970 (+ 11045us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:05.508785 (+   815us) write_op.cc:454] Released schema lock
0709 13:00:05.508841 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:05.509207 (+   366us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:05.509237 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":754521,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":1115,"apply.run_cpu_time_us":1875,"apply.run_wall_time_us":30369,"num_ops":1,"prepare.queue_time_us":128,"prepare.run_cpu_time_us":2619,"prepare.run_wall_time_us":3317,"replication_time_us":2221}]]}
I20260709 13:00:05.534647   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:05.538095  1601 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.544134  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.550103  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:05.551165  3209 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.528100  3312 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.536527  3320 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.545706  3367 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.542847  3323 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.555401  2348 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:05.549706  3305 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.563524  3304 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:00:05.591382  3340 raft_consensus.cc:993] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:05.592046  3340 raft_consensus.cc:1081] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:05.600708  1704 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46224
I20260709 13:00:05.601609  1558 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:05.602383  1704 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:05.602835  1704 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.610608  3305 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:00:05.652468  3366 raft_consensus.cc:993] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 13:00:05.653746  3298 raft_consensus.cc:1081] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Signalling peer d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 13:00:05.653120  3366 raft_consensus.cc:2955] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:05.656782  1561 raft_consensus.cc:2955] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:05.659838  1554 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:47110
I20260709 13:00:05.656009  1704 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.670632  1554 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:05.671334  1554 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.687847  3320 raft_consensus.cc:2955] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:05.691630  2348 raft_consensus.cc:2955] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:05.697664  1704 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:05.698949  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:05.699887  1627 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:05.700330  1627 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:05.706694  2350 raft_consensus.cc:2955] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:05.701084  1627 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.715200  1627 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.715531  1629 raft_consensus.cc:2955] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:05.731678  1627 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:05.733844  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:05.735112  1554 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:05.738072  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 45105.8us
I20260709 13:00:05.754197  3307 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:05.763608  3307 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:05.768575  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:05.769301  1560 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.762825  1458 catalog_manager.cc:5697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 4 to 5, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:05.794790  1554 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:05.797448  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:05.798216  1635 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.795964  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:05.799786  1704 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:05.800038  1704 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:05.806353  1560 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:05.800719  1704 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.810137  1704 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.830554  1560 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:05.837090  1704 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:05.839214  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:05.843772  1635 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:05.883234  1444 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e6c08bb7c6294cf9b33ad3b85e34831d with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:05.844780  3307 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.916796  3432 raft_consensus.cc:993] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:00:05.917825  3307 raft_consensus.cc:1081] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:00:05.919371  1635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 13:00:05.919973  1635 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:05.920334  1635 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.923194  3408 raft_consensus.cc:993] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 13:00:05.923868  3408 raft_consensus.cc:1081] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Signalling peer d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 13:00:05.926301  1555 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:47110
I20260709 13:00:05.927114  1555 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:05.927676  1555 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.929891  1635 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.931939  1635 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:05.944535  1554 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:05.950129  3409 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:05.952948  1455 catalog_manager.cc:5697] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:05.956458  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:05.957157  1708 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:05.957449  1708 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:05.958280  1708 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:05.959697  1708 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.959937  1555 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:05.965219  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:05.966006  1559 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.973816  1555 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:06.002521  1456 catalog_manager.cc:5697] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 4 to 5, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:00:06.008531  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 169418us
I20260709 13:00:06.010407  3311 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:06.010955  3311 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:06.011907  3311 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:06.013839  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 170572us
I20260709 13:00:06.015620  1555 raft_consensus.cc:2955] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:06.023749  2350 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:06.023962  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:06.024526  2347 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.033942  3358 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:06.035925  2347 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:06.038079  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:06.038731  1704 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:06.039033  1704 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:06.039794  1704 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:06.041115  1704 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.043148  1701 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:06.057444  1559 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:06.058696  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:06.068838  3317 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:06.070006  3317 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:06.070976  3317 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:06.077430  1708 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:06.063417  1458 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 3 to 4, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:06.088471  1694 tablet_service.cc:1558] Processing DeleteTablet for tablet e6c08bb7c6294cf9b33ad3b85e34831d with delete_type TABLET_DATA_TOMBSTONED (TS a2ba7fc60f9641cdac25f1e016ef91d8 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:46194
I20260709 13:00:06.089866  3358 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:06.097674  3377 raft_consensus.cc:2955] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:06.099820  2348 raft_consensus.cc:2955] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:06.113162  1635 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:06.135737  1458 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:06.149827  1708 raft_consensus.cc:2955] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:06.174870  1621 tablet_service.cc:1558] Processing DeleteTablet for tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f with delete_type TABLET_DATA_TOMBSTONED (TS ba0548b7d2a94e8598475a46c279fe0e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54730
I20260709 13:00:06.188508  1705 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:06.205269  1704 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:06.219455  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:06.220455  3209 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:06.220993  3209 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:06.222122  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:06.221969  3209 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:06.266047  3438 tablet_replica.cc:333] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:00:06.267359  3438 raft_consensus.cc:2243] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:06.269088  3438 raft_consensus.cc:2272] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:06.271235  1635 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:06.271724  2347 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:06.277807  3408 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:06.292111  1633 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:06.293349  3366 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:06.309983  3366 raft_consensus.cc:2955] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:06.313769  3336 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:06.311471  3438 ts_tablet_manager.cc:1916] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:06.318365  2348 raft_consensus.cc:2955] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:06.324311  3439 tablet_replica.cc:333] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:00:06.328362  3439 raft_consensus.cc:2243] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:06.329968  3439 raft_consensus.cc:2272] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:00:06.377389  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 114428us
I20260709 13:00:06.401935  1458 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 3 to 4, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:06.415355  1456 catalog_manager.cc:5697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 3 to 4, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 4 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:06.448879  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1457f31daeb646a9824ac8068949eddd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:06.449932  3357 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.451115  3357 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:06.456605  3358 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:06.464634  3336 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:06.466102  3336 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:06.472429  1635 raft_consensus.cc:2955] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:06.473173  3336 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:06.487103  3439 ts_tablet_manager.cc:1916] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:06.490348  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:06.492049  1633 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:06.497651  3357 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:06.522279  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:06.523345  1627 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:06.536285   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:06.500427  3358 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:06.539861  3358 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:06.549680  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 64537.4us
W20260709 13:00:06.570423  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44197.8us
I20260709 13:00:06.573662  1458 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 2 to 3, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:06.583221  1561 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:06.595605  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:06.596848  1704 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:06.598577  1701 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:06.600561  1707 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:06.634379  2350 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:06.621445  1456 catalog_manager.cc:5697] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: config changed from index 3 to 4, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:06.626704  3311 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:00:06.629659  3357 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:00:06.639046  1627 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:06.630087  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:06.642185  2347 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:06.630034  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:06.644762  3444 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:06.645433  3444 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.645855  3444 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:06.658618  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:06.667222  3438 ts_tablet_manager.cc:1929] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:00:06.668038  3438 log.cc:1199] T e6c08bb7c6294cf9b33ad3b85e34831d P a2ba7fc60f9641cdac25f1e016ef91d8: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-2-root/wals/e6c08bb7c6294cf9b33ad3b85e34831d
I20260709 13:00:06.689582  3357 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:00:06.694926  1542 tablet_service.cc:1558] Processing DeleteTablet for tablet 1457f31daeb646a9824ac8068949eddd with delete_type TABLET_DATA_TOMBSTONED (TS d62ce354d09a4be59c29094425d09bc1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47096
I20260709 13:00:06.695518  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:06.703043  1635 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:06.703538  1633 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:06.705629  3334 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:00:06.714659  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:06.721782  3330 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:06.723570  3330 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:06.724674  3330 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.733772  1445 catalog_manager.cc:5028] TS a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): tablet e6c08bb7c6294cf9b33ad3b85e34831d (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 13:00:06.726064  3358 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:00:06.793428  1701 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:06.811225  3366 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:06.813275  1560 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:06.845947  3450 tablet_replica.cc:333] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:00:06.847520  3450 raft_consensus.cc:2243] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:06.816442  3444 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:06.865067  3450 raft_consensus.cc:2272] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:06.881392  3450 ts_tablet_manager.cc:1916] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:06.883977  1557 raft_consensus.cc:2368] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4.
I20260709 13:00:06.892071  3448 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.892978  3448 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:06.893374  3330 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:06.900336  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:06.901950  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:06.902917  2346 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:06.902791  3305 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:00:06.913179  1628 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:06.915335  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:06.906778  3448 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:06.928417  3311 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:06.929620  3444 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:06.930832  3439 ts_tablet_manager.cc:1929] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:06.912443  3330 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:06.931399  3439 log.cc:1199] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P ba0548b7d2a94e8598475a46c279fe0e: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-1-root/wals/83a61af0ea7d47c1bc0ee6a94a2d1a7f
I20260709 13:00:06.939574  3447 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:00:06.938643  3311 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:06.943034  3311 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:06.946017  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:06.947065  1703 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:06.959745  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:06.960752  1628 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:06.964116  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59712.1us
I20260709 13:00:07.033156  1444 catalog_manager.cc:5028] TS ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): tablet 83a61af0ea7d47c1bc0ee6a94a2d1a7f (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 13:00:07.052531  1628 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:07.063611  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:07.064637  1708 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
W20260709 13:00:07.071623  3457 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:00:07.073683  3312 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.074412  3312 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:07.076849  3312 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:07.084456  2318 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:07.089442  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:07.092561  3444 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:07.094419  3444 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:07.095573  3444 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:07.097898  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:07.137400  1703 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:07.145277  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:07.146459  1627 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:07.150900  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:07.151818  1704 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
W20260709 13:00:07.194778  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 61409.6us
I20260709 13:00:07.195358  3418 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:07.195853  3418 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.196228  3418 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:07.197355  1675 leader_election.cc:400] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4.
I20260709 13:00:07.211741  1675 leader_election.cc:403] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:07.232447  3471 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4.
W20260709 13:00:07.327260  3466 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.207s	user 0.002s	sys 0.000s
I20260709 13:00:07.356978  3418 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:07.373054  2348 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:07.377513  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:07.378286  1702 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:07.379715  3418 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:07.384075  1456 catalog_manager.cc:5697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:07.388098  1456 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 2 to 3, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:07.397738  1702 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
W20260709 13:00:07.473040  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 61253.4us
I20260709 13:00:07.475603  3337 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.476238  3337 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:07.484774  2318 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:07.486847  3419 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:07.486965  3337 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:07.487495  3419 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:07.488493  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:07.489612  1633 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:07.488504  3419 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:07.497987  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:07.501048  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:07.516153  1628 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:07.529979  3310 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:07.531243  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:07.531956  1628 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:07.550441  3450 ts_tablet_manager.cc:1929] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:07.551031  3450 log.cc:1199] T 1457f31daeb646a9824ac8068949eddd P d62ce354d09a4be59c29094425d09bc1: Deleting WAL directory at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0/minicluster-data/ts-0-root/wals/1457f31daeb646a9824ac8068949eddd
I20260709 13:00:07.552922  1445 catalog_manager.cc:5028] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): tablet 1457f31daeb646a9824ac8068949eddd (table test-workload [id=6f942a41aa0443b6bd9ab8bf0b57562f]) successfully deleted
I20260709 13:00:07.553840  2317 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:07.556383  3448 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:07.556893  3448 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.557432  3448 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:07.549178  1633 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:07.576407  3480 raft_consensus.cc:2764] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:07.585441  1633 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:07.592705  1702 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:07.594269   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:07.590965  3311 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:00:07.607638  3448 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:07.609652  1455 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:07.611358  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:07.612468  3448 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:07.620725  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:07.669188  3378 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:00:07.690130  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:07.692071  3419 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:07.693825  2352 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:07.725333  3474 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:07.726701  3474 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:07.727258  3474 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:07.728893  3474 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:07.730566  3474 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:07.766580  1636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 13:00:07.767367  1636 raft_consensus.cc:606] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:07.779623  1561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:00:07.780339  1561 raft_consensus.cc:606] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:07.823562  1561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:00:07.824753  1561 raft_consensus.cc:606] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:07.838441  1560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:00:07.839152  1560 raft_consensus.cc:606] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:07.840529   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:07.842792   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:07.856364  3377 raft_consensus.cc:993] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:07.857236  3474 raft_consensus.cc:1081] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:07.859292  1704 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46210
I20260709 13:00:07.860122  1704 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:07.860564  1704 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:07.889670  1704 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:07.893005  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:07.893858  2352 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:07.901167  2352 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:07.893005  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:07.902277  1560 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:07.902594  1560 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:07.903402  1560 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:07.904609  1560 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:07.909198  1702 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:07.911350  3305 raft_consensus.cc:993] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:07.912019  3305 raft_consensus.cc:1081] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:07.913591  1704 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:07.914532  1701 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46224
I20260709 13:00:07.915169  1701 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:07.915555  1701 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.916668  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:07.918087  3366 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:07.918651  3366 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:07.919598  3366 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:07.924321  1701 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:07.939107  1701 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:07.939409  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:07.940090  1561 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.943053  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:07.943701  1633 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:07.944048  1633 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:07.945919  1633 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:07.949275  1633 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.955580  1561 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:07.957039  1675 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:07.958036  3390 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:07.958511  3390 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:07.959486  3390 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:07.977538  1455 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:07.985491  1560 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:07.990973  1633 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:07.995678  1636 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:07.997546  3409 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:07.999575  1560 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:08.001292  1676 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 637e660a66be4924866287fa6150aaea. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:08.002820  3409 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:08.007354  1455 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 2 to 3, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:08.023795  1560 raft_consensus.cc:2955] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:08.031554  3377 raft_consensus.cc:993] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:00:08.032367  3311 raft_consensus.cc:1081] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:00:08.014943  3390 raft_consensus.cc:2955] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:08.041143  1636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54748
I20260709 13:00:08.041953  1636 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.042389  1636 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:08.051996  1710 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:08.053001  1455 catalog_manager.cc:5697] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:08.062433  1633 raft_consensus.cc:2955] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:08.074296  1629 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:08.074391  1636 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:08.082525  3419 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:08.087344  1636 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:08.088764  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:08.094895  1560 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:08.095274  1560 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:08.096035  1560 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:08.097512  1560 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:08.091169  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:08.105954  1710 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:08.106853  1560 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:08.114584  1702 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:08.115933  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:08.130007  3446 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:08.126410  3419 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:00:08.225708  1636 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:08.200592  3446 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:08.258762  3496 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.128s	user 0.002s	sys 0.000s
I20260709 13:00:08.260531  3446 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:08.290964  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 145645us
I20260709 13:00:08.298727  1710 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:08.330590  3366 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
W20260709 13:00:08.376334  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 57121.3us
W20260709 13:00:08.378762  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 52512.9us
W20260709 13:00:08.408362  1676 consensus_peers.cc:597] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8785cfce0d0042d3855cf9a7edaf1f7d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:08.449939  1561 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:08.455102  1557 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:09.174852   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:00:09.280741  3514 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.816s	user 0.001s	sys 0.000s
W20260709 13:00:09.281837  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 224648us
I20260709 13:00:09.283699  3513 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:09.284276  3513 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } }
I20260709 13:00:09.288005  3513 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:09.307788  3419 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.308486  3419 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.311921  3419 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.326187  3520 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:09.327312  3520 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.331146  3520 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.336323  3519 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:09.337184  3519 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.340312  3519 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.343822  3518 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.342281  3465 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.344420  3518 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.345146  3465 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.345870  3519 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:09.346405  3519 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.350088  3519 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.350903  3465 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.349929  3518 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:09.359761  1636 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.003s	sys 0.004s
W20260709 13:00:09.360184  1636 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.003s	sys 0.004s
I20260709 13:00:09.361783  3504 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:09.362622  3504 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.365826  3504 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.368621  3524 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:09.372455  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 906451us
I20260709 13:00:09.372095  3524 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.374729  3419 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.378966  3465 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:09.378865  3419 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.379602  3465 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.381654  3419 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:09.382203  3465 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.382797  3517 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.383074  3465 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:09.383325  3517 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.383684  3465 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.377393  3524 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.386888  3465 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.387477  3517 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.407086  3503 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.408005  3525 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.407871  3503 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.408667  3525 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.410391  3503 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.411077  3525 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.412743  3501 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.413295  3501 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.415817  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.422631  3501 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.425463  3404 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:09.426138  3404 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.428592  3404 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.431023  3404 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:09.433645  3404 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.439878  3404 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.445915  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.450779  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.451637  2352 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:09.453671  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.455622  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.458748  3474 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:09.459398  3474 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.462944  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.470930  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.472713  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.474646  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.476039  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.478492  1456 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 268) took 1394 ms. Trace:
I20260709 13:00:09.478927  1456 rpcz_store.cc:276] 0709 13:00:08.084191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.084453 (+   262us) service_pool.cc:224] Handling call
0709 13:00:09.478386 (+1393933us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":670091,"reported_tablets":1}
I20260709 13:00:09.479641  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.481586  1455 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 266) took 1383 ms. Trace:
I20260709 13:00:09.481978  1455 rpcz_store.cc:276] 0709 13:00:08.098587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.099148 (+   561us) service_pool.cc:224] Handling call
0709 13:00:09.481521 (+1382373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1299965,"reported_tablets":1}
I20260709 13:00:09.487946  3474 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.494663  1527 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:09.495527  3515 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:09.495937  3515 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:09.496340  3515 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:09.496075  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.498423  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.501858  3504 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:09.502498  3504 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.504751  3504 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.507181  3529 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:09.507447  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.507746  3529 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.510016  3529 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.513178  3513 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:09.513862  3513 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.516151  3513 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.517589  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.520308  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.520742  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.524775  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.526891  3515 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.528321  1560 raft_consensus.cc:2368] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5.
I20260709 13:00:09.529763  3515 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.530629  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.533999  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.534708  1633 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:09.535953  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:09.536778  2347 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:09.537273  3390 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.537684  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.537992  3390 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.539077  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.540814  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.540817  3527 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.536070  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.542332  3527 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.544945  3513 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:09.545784  3527 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.545631  3513 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.547767  3532 ts_tablet_manager.cc:933] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.548276  3529 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.548915  3529 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.550508  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.551263  3529 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.552392  3474 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.552922  3474 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.555347  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.556813  3390 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.557432  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:09.558001  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.558481  3513 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.559396  3474 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.576299  1633 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:09.576591  1702 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:09.562057  2347 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:09.578676  1527 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:09.578931  3531 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:09.579612  3474 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:09.580185  3474 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:09.562747  1601 leader_election.cc:400] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5.
I20260709 13:00:09.563496  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.581187  3474 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.565873  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.566749  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.567044  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.568904  2346 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:09.585018  1636 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:09.585916  1601 leader_election.cc:403] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:09.587795  3527 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:09.588707  3528 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.588570  3527 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.589237  3528 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.590170  3531 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.591254  3527 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.591683  3528 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:09.591866  3527 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.592303  3528 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.592453  3527 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.593869  3531 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.571692  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.594237  2317 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:09.593895  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.573797  3532 tablet_copy_client.cc:323] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45431
I20260709 13:00:09.596782  1554 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:09.600844  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.601999  1633 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:09.604939  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.622433  3513 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.605902  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.611819  2350 raft_consensus.cc:2368] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5.
I20260709 13:00:09.623004  3513 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.613739  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.615710  3518 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.618465  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.618335  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:09.592897  3528 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.625350  1707 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:09.624768  3529 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5.
I20260709 13:00:09.628540  3520 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.627187  3515 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:09.632218  3515 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.634380  3390 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:09.635489  1525 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:09.636437  3522 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:09.639123  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:09.644153  3530 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:09.645200  3522 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.638769  3531 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.646271  3531 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.647401  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:09.648260  3522 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.648851  3531 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.640422  3515 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.650749  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.650844  3535 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.641230  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.652000  3535 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.653324  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.654546  3535 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:09.655931  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.656594  1702 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:09.640066  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.640954  1455 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 4 to 5, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 5 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:09.643411  3513 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.643541  3480 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:09.651537  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.667634  3533 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.667405  3534 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.668366  3533 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.669158  3534 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.669848  3480 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:09.679173  1602 leader_election.cc:400] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5.
I20260709 13:00:09.684175  1633 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:09.689587  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:09.691293  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.720494  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.724342  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.736896  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.672895  3390 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:00:09.719762  3480 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.697902  3515 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.698006  3535 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:09.740818  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 79548.6us
I20260709 13:00:09.741145  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.742252  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:09.743286  3541 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.715377  3527 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:09.716483  3533 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.740571  3528 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.756840  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.756747  1459 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:09.736063  3534 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:09.754518  3541 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:09.748620  3311 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:09.754822  3535 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.764559  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.758298  3533 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:00:09.776119  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.776376  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.767710  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.767831  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.769665  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.770601  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.771715  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.781152  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.776674  1702 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:09.785447  1526 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:09.774521  3311 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:09.796967  3516 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.797662  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.789191  3526 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:00:09.800268  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.803371  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:09.803740  3535 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.804688  3521 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.793658  3311 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:09.806173  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.810146  3541 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:09.813791  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:09.815470  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:09.816613  3404 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:09.839612  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51329.9us
I20260709 13:00:09.830780  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:09.848272  3474 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:09.854228  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:09.856799  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.860635  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.862018  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:09.863428  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:09.868559  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 84092.9us
I20260709 13:00:09.869436  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:09.870271  3503 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.870756  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:09.873677  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:09.875932  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:09.883008  3552 ts_tablet_manager.cc:933] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:09.888406  3503 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.888720  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.873132  3521 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:09.890472  3521 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:09.890955  3521 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:09.874002  3474 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.892879  3474 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.897011  3504 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:09.905189  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:09.908437  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.911031  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:09.912307  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 155153us
I20260709 13:00:09.897431  3519 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:09.916242  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.914004  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.913015  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:09.899103  3522 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.926976  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:09.930801  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.939878  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.908671  3504 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.912858  3465 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.946830  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:09.948513  3552 tablet_copy_client.cc:323] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45431
I20260709 13:00:09.969944  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.947381  3504 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.941839  3541 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:09.974320  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:09.941937  3549 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.980195  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.986358  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.971551  3524 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:00:09.989249  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 46278.5us
I20260709 13:00:09.990823  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:09.978415  3474 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:09.999356  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:09.983860  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:09.994585  3524 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:09.988577  3525 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.003103  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:10.002693  3524 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.985384  3541 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:09.990922  3515 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:10.039793  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 75831.1us
I20260709 13:00:10.072952  3474 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:10.073678  3474 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:10.073941  1720 tablet_copy_service.cc:140] P a2ba7fc60f9641cdac25f1e016ef91d8: Received BeginTabletCopySession request for tablet 637e660a66be4924866287fa6150aaea from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:52646)
I20260709 13:00:10.081410  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:10.087241  3520 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:10.087427  1719 tablet_copy_service.cc:140] P a2ba7fc60f9641cdac25f1e016ef91d8: Received BeginTabletCopySession request for tablet 8785cfce0d0042d3855cf9a7edaf1f7d from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:52646)
W20260709 13:00:10.097674  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 123996us
I20260709 13:00:10.091051  3525 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.086067  3515 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:10.145445  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 45561.2us
I20260709 13:00:10.081858  3519 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:10.148272  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.158814  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.134606  3390 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:10.090929  3517 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:10.091461  3526 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:10.164170  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.089340  3541 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:10.166195  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 91798.3us
I20260709 13:00:10.004218  3551 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.156s
I20260709 13:00:10.088083  1719 tablet_copy_service.cc:161] P a2ba7fc60f9641cdac25f1e016ef91d8: Beginning new tablet copy session on tablet 8785cfce0d0042d3855cf9a7edaf1f7d from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:52646: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-8785cfce0d0042d3855cf9a7edaf1f7d
I20260709 13:00:10.172523  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.173820  3526 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:10.184243  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:10.177525  1675 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.089848  3513 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:10.162719  3535 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.136158  3311 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:10.087929  3520 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:10.198916  1675 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.191171  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:10.169996  3525 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:10.230813  1557 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:10.195971  3390 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:10.236128  3311 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:10.236935  1582 debug-util.cc:398] Leaking SignalData structure 0x7b08000b6360 after lost signal to thread 434
W20260709 13:00:10.238288  1582 debug-util.cc:398] Leaking SignalData structure 0x7b08002f7300 after lost signal to thread 1493
I20260709 13:00:10.074219  3539 raft_consensus.cc:2955] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 13:00:10.238734  1582 debug-util.cc:398] Leaking SignalData structure 0x7b0800314a80 after lost signal to thread 1507
I20260709 13:00:10.169629  3519 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:10.190281  3527 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:10.240151  1582 debug-util.cc:398] Leaking SignalData structure 0x7b08000c49e0 after lost signal to thread 1585
W20260709 13:00:10.270463  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 71609.5us
I20260709 13:00:10.271306  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:10.271733  1582 debug-util.cc:398] Leaking SignalData structure 0x7b08001f9b20 after lost signal to thread 1660
I20260709 13:00:10.254522  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:10.272733  1627 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:10.273118  1582 debug-util.cc:398] Leaking SignalData structure 0x7b080037d780 after lost signal to thread 1734
W20260709 13:00:10.274784  1582 debug-util.cc:398] Leaking SignalData structure 0x7b0800389aa0 after lost signal to thread 2377
I20260709 13:00:10.247380  2350 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:10.244611  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:10.256132  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:10.237686  3525 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:10.267889  3519 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:10.200688  3541 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.292462  1675 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.299260  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:10.312849  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 123642us
I20260709 13:00:10.315038  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.183650  3517 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:10.326012  1675 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.286302  3534 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.269820  1720 tablet_copy_service.cc:161] P a2ba7fc60f9641cdac25f1e016ef91d8: Beginning new tablet copy session on tablet 637e660a66be4924866287fa6150aaea from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:52646: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-637e660a66be4924866287fa6150aaea
I20260709 13:00:10.213899  3513 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:10.307947  3528 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:10.399705  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 85180us
I20260709 13:00:10.309171  3474 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:10.406486  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.409116  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:10.409881  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:10.413033  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:10.413641  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:10.420446  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.350426  3515 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:10.427023  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:10.428607  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.434635  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:10.454835  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:10.455821  3546 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.456681  3565 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:10.457377  3560 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:10.458042  3560 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:10.460430  3560 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:10.460966  3522 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:10.461702  3567 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.461058  3545 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.461093  3571 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.463546  3562 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:10.463804   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:10.457263  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:10.464172  3562 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:10.462584  3567 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.465853  3567 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:10.466845  3562 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
W20260709 13:00:10.469683  3572 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:00:10.470726  3517 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:10.472826  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:10.461853  3585 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.490692  1636 raft_consensus.cc:2955] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:10.485160  1634 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:10.465080  3504 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:10.537837  3524 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:10.539381  1631 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:10.540535  3524 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:10.543165  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 111820us
I20260709 13:00:10.457257  3565 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:10.642519  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:10.660893  3521 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.769s	user 0.012s	sys 0.000s
I20260709 13:00:10.668090  1627 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:10.623301  3526 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:10.669911  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:10.655123  3529 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.002s	sys 0.010s
I20260709 13:00:10.650712  3513 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:10.682247  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 270524us
W20260709 13:00:10.677919  3529 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.003s	sys 0.010s
I20260709 13:00:10.668478  3504 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:10.677608  3571 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:10.714044  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.715776  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.718456  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:10.724478  3528 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.739398  1720 tablet_copy_source_session.cc:215] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:10.745563  3565 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:10.676323  3521 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:10.751144  3521 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:10.734747  3390 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.708119  3579 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:00:10.760113  3591 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:10.753420  3539 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.003s
W20260709 13:00:10.769901  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45788.3us
W20260709 13:00:10.761682  3582 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:00:10.775946  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:10.783406  3605 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:10.786432  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:10.789271  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:10.790048  1555 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:10.795006  1459 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 268) took 1090 ms. Trace:
I20260709 13:00:10.804873  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:10.810133  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:10.807459  3604 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:00:10.814272  3532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 637e660a66be4924866287fa6150aaea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:10.795409  1459 rpcz_store.cc:276] 0709 13:00:09.705081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.705301 (+   220us) service_pool.cc:224] Handling call
0709 13:00:10.794924 (+1089623us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:09.872074 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:09.886251 (+ 14177us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:09.886290 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:00:09.886310 (+    20us) tablet.cc:662] Decoding operations
0709 13:00:09.886731 (+   421us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:09.886781 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:09.886820 (+    39us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:09.887024 (+   204us) tablet.cc:701] Row locks acquired
0709 13:00:09.887058 (+    34us) write_op.cc:260] PREPARE: finished
0709 13:00:09.887214 (+   156us) write_op.cc:270] Start()
0709 13:00:09.887333 (+   119us) write_op.cc:276] Timestamp: P: 1783602009887158 usec, L: 0
0709 13:00:09.887357 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:00:09.888101 (+   744us) log.cc:844] Serialized 536 byte log entry
0709 13:00:10.275456 (+387355us) op_driver.cc:464] REPLICATION: finished
0709 13:00:10.536615 (+261159us) write_op.cc:301] APPLY: starting
0709 13:00:10.536706 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:10.536823 (+   117us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:10.536938 (+   115us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:10.537236 (+   298us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:10.537413 (+   177us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:10.537511 (+    98us) write_op.cc:312] APPLY: finished
0709 13:00:10.537742 (+   231us) log.cc:844] Serialized 23 byte log entry
0709 13:00:10.538524 (+   782us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:10.538612 (+    88us) write_op.cc:454] Released schema lock
0709 13:00:10.538655 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:10.539003 (+   348us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:10.539039 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":260928,"apply.run_cpu_time_us":1751,"apply.run_wall_time_us":2348,"num_ops":1,"prepare.queue_time_us":112106,"prepare.run_cpu_time_us":2997,"prepare.run_wall_time_us":16290,"replication_time_us":387938,"thread_start_us":75509,"threads_started":1}]]}
I20260709 13:00:10.784148  3605 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:10.826691  3539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:52636 (request call id 10) took 2780 ms. Trace:
I20260709 13:00:10.830492  3605 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:10.831393  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:10.831658  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:10.832167  1552 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:10.836387  3525 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:10.839887  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:10.840853  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:10.843562  1675 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.848392  3538 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:10.850137  1633 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:10.852170  3525 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:10.852638  3525 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:10.853044  3525 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.857515  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:10.859642  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:10.860142  2317 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:10.842722  1561 raft_consensus.cc:2955] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:10.864866  3599 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:10.895680  3583 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.909315  1552 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:10.920019  1636 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:10.920596  1455 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 267) took 1287 ms. Trace:
I20260709 13:00:10.921986  1627 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:10.924937  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:10.916105  3599 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:10.921002  1455 rpcz_store.cc:276] 0709 13:00:09.633134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.634038 (+   904us) service_pool.cc:224] Handling call
0709 13:00:10.920523 (+1286485us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:09.757072 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:09.767499 (+ 10427us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:09.767537 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:00:09.767554 (+    17us) tablet.cc:662] Decoding operations
0709 13:00:09.769571 (+  2017us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:09.769650 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:09.769686 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:09.769898 (+   212us) tablet.cc:701] Row locks acquired
0709 13:00:09.769916 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:00:09.770056 (+   140us) write_op.cc:270] Start()
0709 13:00:09.770179 (+   123us) write_op.cc:276] Timestamp: P: 1783602009770011 usec, L: 0
0709 13:00:09.770204 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:00:09.770933 (+   729us) log.cc:844] Serialized 541 byte log entry
0709 13:00:10.001395 (+230462us) op_driver.cc:464] REPLICATION: finished
0709 13:00:10.533130 (+531735us) write_op.cc:301] APPLY: starting
0709 13:00:10.533246 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:10.533375 (+   129us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:10.533539 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:10.533804 (+   265us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:10.534009 (+   205us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:10.534069 (+    60us) write_op.cc:312] APPLY: finished
0709 13:00:10.534321 (+   252us) log.cc:844] Serialized 23 byte log entry
0709 13:00:10.534530 (+   209us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:10.534625 (+    95us) write_op.cc:454] Released schema lock
0709 13:00:10.534675 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:10.536204 (+  1529us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:10.536239 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":210817,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":528356,"apply.run_cpu_time_us":2012,"apply.run_wall_time_us":3097,"num_ops":1,"prepare.queue_time_us":82825,"prepare.run_cpu_time_us":10763,"prepare.run_wall_time_us":114589,"raft.queue_time_us":342389,"raft.run_cpu_time_us":18934,"raft.run_wall_time_us":410751,"replication_time_us":217650,"thread_start_us":357888,"threads_started":4,"wal-append.queue_time_us":98183}]]}
I20260709 13:00:10.945323  3520 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:10.923795  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:10.964859  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:10.965448  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:10.963824  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 47775.8us
I20260709 13:00:10.989797  1701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:00:10.827207  3539 rpcz_store.cc:276] 0709 13:00:08.046566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.049264 (+  2698us) service_pool.cc:224] Handling call
0709 13:00:08.064527 (+ 15263us) log.cc:844] Serialized 506 byte log entry
0709 13:00:08.101928 (+ 37401us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b2c000732a8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe2a0a 00007fda25fe2693 00007fda25fe2631 00007fda27716b52 00007fda27716cf5 00007fda27653691 00007fda27633eaa 00007fda2763ad53 00007fda2763acd9 00007fda2763ac69
0709 13:00:08.264265 (+162337us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b1800176700. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27716d01 00007fda27653691 00007fda27633eaa 00007fda2763ad53 00007fda2763acd9 00007fda2763ac69 00007fda2763ac31 00007fda27639dfd 000000000038f354 000000000038f199
0709 13:00:08.306801 (+ 42536us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b1800176700. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27716d01 00007fda27653691 00007fda27633eaa 00007fda2763ad53 00007fda2763acd9 00007fda2763ac69 00007fda2763ac31 00007fda27639dfd 000000000038f354 000000000038f199
0709 13:00:10.826569 (+2519768us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10258,"raft.queue_time_us":2382,"raft.run_cpu_time_us":12847,"raft.run_wall_time_us":504551,"spinlock_wait_cycles":134611840,"thread_start_us":8783,"threads_started":1,"wal-append.queue_time_us":62929,"wal-append.run_cpu_time_us":15548,"wal-append.run_wall_time_us":1287234}
I20260709 13:00:10.987824  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:11.039512  1701 raft_consensus.cc:606] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:10.966391  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:11.048132  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:11.050598  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 126901us
I20260709 13:00:10.920102  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:10.967298  3390 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:10.966675  3599 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.072088  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:11.070808  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:11.082190  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:11.067129  3593 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.075048  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:11.072031  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:11.089742  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:11.104005  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:11.071111  3390 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:11.087313  3605 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:11.089098  3311 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:11.138484  3541 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:11.139534  3541 raft_consensus.cc:993] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 13:00:11.119230  3311 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:11.140472  3311 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.139973  3541 raft_consensus.cc:1081] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Signalling peer d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 13:00:11.142166  3390 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:11.093818  3522 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:00:11.144615  3390 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.145268  3525 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:11.146764  1632 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:11.150250  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:11.114399  3516 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:00:11.100111  3520 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:00:11.100154  3563 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:00:11.135716  3605 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:11.135119  3546 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:11.198596  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48452.7us
I20260709 13:00:11.193120  3581 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.201072  3581 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:11.203948  3581 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:11.204841  3532 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59274 (request call id 286) took 1669 ms. Trace:
I20260709 13:00:11.198676  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:11.206787  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:11.207178  3513 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.214922  1703 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:11.212827  3580 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.197314  3546 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:11.227744  3605 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:11.228775  3546 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:11.229833  3525 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:11.211951  3581 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.236351  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:11.238266  3311 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:11.243139  3311 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:11.244488  1719 tablet_copy_source_session.cc:215] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:11.250069  1719 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:52646 (request call id 262) took 1177 ms. Trace:
I20260709 13:00:11.247840  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:11.252030  1636 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:11.259891  3571 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:11.261143  3571 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:11.261577  3571 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:11.268695  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:11.290592  3563 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.301929  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:11.250619  1719 rpcz_store.cc:276] 0709 13:00:10.072322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.074435 (+  2113us) service_pool.cc:224] Handling call
0709 13:00:11.242209 (+1167774us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b540005fda0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27651dc0 00007fda276e44a5 00007fda276e43ee 00007fda2c67a036 00007fda2c683a3e 00007fda23f4727b 00007fda2c6839ba 00007fda2c67eacc 00007fda2c679c7b 00007fda2c66e8d1
0709 13:00:11.243990 (+  1781us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54001860d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276e43f7 00007fda2c67a036 00007fda2c683a3e 00007fda23f4727b 00007fda2c6839ba 00007fda2c67eacc 00007fda2c679c7b 00007fda2c66e8d1 00007fda2b7224b4 00007fda2b722442
0709 13:00:11.249957 (+  5967us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":746261760}
I20260709 13:00:11.307307  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:11.314009  3571 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:11.316614  3571 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.306564  1555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:00:11.318186  1555 raft_consensus.cc:606] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:11.319928  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:11.205360  3532 rpcz_store.cc:276] 0709 13:00:09.535885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.541429 (+  5544us) service_pool.cc:224] Handling call
0709 13:00:09.548162 (+  6733us) ts_tablet_manager.cc:934] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
0709 13:00:10.799923 (+1251761us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:00:11.030064 (+230141us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:11.030224 (+   160us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:00:11.192892 (+162668us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:00:11.193266 (+   374us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:11.204668 (+ 11402us) ts_tablet_manager.cc:989] Replica registered.
0709 13:00:11.204740 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":358,"tablet-copy.queue_time_us":5523,"thread_start_us":30021,"threads_started":2}
I20260709 13:00:11.322408  3520 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:11.323374  3557 raft_consensus.cc:993] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:11.323887  3557 raft_consensus.cc:1081] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:11.328972  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:11.363924  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:11.365669  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:11.322413  3532 tablet_copy_client.cc:806] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:11.360930  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:11.477260  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 113708us
I20260709 13:00:11.485747  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:11.490571  3581 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.493585  3599 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:11.494493  3552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8785cfce0d0042d3855cf9a7edaf1f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:11.495355  1629 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:11.495841  3599 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:11.485821  3311 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.497771  1628 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:11.498693  3603 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:11.499229  3603 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:11.499930  1629 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:11.500200  3603 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:11.503078  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:11.504510  3549 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:11.504923  3549 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:11.505800  3549 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:11.519351  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.521448  1629 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:00:11.493170  3557 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:11.525712  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47214.7us
I20260709 13:00:11.529062  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:11.534392  3549 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:11.508730  3532 tablet_copy_client.cc:670] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:11.551422  3516 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.553102  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:11.556834  3599 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.556649  3474 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
W20260709 13:00:11.608354  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 122241us
I20260709 13:00:11.544426  3557 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:11.662513  3610 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.126s	user 0.000s	sys 0.002s
I20260709 13:00:11.633289  3557 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:11.690608  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:11.676975  1455 catalog_manager.cc:5697] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index -1 to 2, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:11.676358  1628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:48418
W20260709 13:00:11.698535  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 168120us
W20260709 13:00:11.700829  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.720728  1635 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:11.728356  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.729110  3614 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:11.730801  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 223898us
I20260709 13:00:11.733074  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:11.739501  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 215235us
I20260709 13:00:11.734023  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:11.751349  1558 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:47110
W20260709 13:00:11.757416  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44276.1us
I20260709 13:00:11.765035  1557 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:11.771729  1701 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46210
I20260709 13:00:11.772616  1701 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:11.773114  1701 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.771823  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:11.755278  3614 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:11.823860  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:11.795003  3311 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:00:11.831625  3590 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:11.832871  3608 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:00:11.834317  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.763973  3551 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:00:11.832927  1628 raft_consensus.cc:606] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:11.840437  1635 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:11.779827  3539 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:11.841542  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:11.765427  3604 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 13:00:11.855942  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:11.858814  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:11.861009  1701 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:11.879866  1706 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:00:11.889658  1706 pending_rounds.cc:85] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 2
I20260709 13:00:11.847175  3613 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.894200  1705 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:11.890839  1706 pending_rounds.cc:107] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:00:11.900691  1701 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:11.846446  3590 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:11.973994  1706 raft_consensus.cc:2889] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:11.887887  3525 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:12.031013  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 134198us
I20260709 13:00:12.033547  3596 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:11.892103  3311 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.035916  3311 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:12.036698  3596 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:12.041981  3596 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:12.048214  3596 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:12.050714  3596 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.061580  1554 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:00:11.889605  3539 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:12.078416  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 196356us
I20260709 13:00:12.048941  3603 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.090342  3590 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:12.093720  3571 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.777s	user 0.003s	sys 0.014s
I20260709 13:00:12.098421  3539 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:12.100240  3549 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.002s
W20260709 13:00:12.104471  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 202699us
I20260709 13:00:12.111367  3552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59274 (request call id 297) took 2292 ms. Trace:
W20260709 13:00:12.059376  3516 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.002s
I20260709 13:00:12.123859  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:12.131852  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:12.132675  1627 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:12.134398  3563 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.134984  3614 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:12.137614  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:12.141002  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51683.6us
I20260709 13:00:12.144408  3626 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:12.122509  3516 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.146633  1558 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.147051  1558 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.150365  1707 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:12.157698  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:12.159976  3532 tablet_copy_client.cc:538] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:12.162325  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:12.168814  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 405163us
I20260709 13:00:12.179193  1558 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:12.181238  3515 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:12.189983  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:12.193035  1676 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 61a6c336f56b4c248aac90993fb477cf. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:12.198465  1628 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.139612  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:12.201274  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61772.4us
I20260709 13:00:12.206285  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:12.207345  2348 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.208386  1553 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 3. Ops: []
I20260709 13:00:12.208874  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:12.209756  1556 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:12.216866  3522 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:12.208155  3626 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.224210  3539 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:12.262104  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.268872  2348 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:12.270835  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:12.121487  3552 rpcz_store.cc:276] 0709 13:00:09.819083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.819316 (+   233us) service_pool.cc:224] Handling call
0709 13:00:09.883592 (+ 64276us) ts_tablet_manager.cc:934] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
0709 13:00:11.493028 (+1609436us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:00:11.846147 (+353119us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:11.848561 (+  2414us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:00:12.110525 (+261964us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:00:12.110857 (+   332us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:12.111191 (+   334us) ts_tablet_manager.cc:989] Replica registered.
0709 13:00:12.111265 (+    74us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":33584,"tablet-copy.queue_time_us":62696,"thread_start_us":119414,"threads_started":2}
I20260709 13:00:12.279737  3528 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:12.283819  3552 tablet_copy_client.cc:806] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:12.284710  3552 tablet_copy_client.cc:670] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:12.267043  3539 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:12.288331  3539 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:12.313452  3532 tablet_bootstrap.cc:492] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 13:00:12.315800  3549 raft_consensus.cc:993] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: : Instructing follower 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 13:00:12.316421  3549 raft_consensus.cc:1081] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Signalling peer 95fc6a4cf5874b7eb91280dffcd3b3fb to start an election
I20260709 13:00:12.319491  3552 tablet_copy_client.cc:538] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:12.323648  2352 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:59270
I20260709 13:00:12.307080  1561 raft_consensus.cc:2955] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:12.287087  3528 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:12.324512  2352 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.324942  2352 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.327473  3528 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:12.303580  3626 raft_consensus.cc:2955] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:12.340005  1558 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:12.341671  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:12.342295  1629 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.383458  1708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:00:12.387632  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:12.557574  3632 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.172s	user 0.005s	sys 0.014s
I20260709 13:00:12.558719  3604 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:12.559326  3596 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.508s	user 0.022s	sys 0.001s
I20260709 13:00:12.566669  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:12.568043  3538 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:12.569645  3390 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.568871  3538 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:12.570310  3390 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:12.572588  3390 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:12.572804  3538 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:12.570267  1554 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:12.575515  3538 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:12.576223  3538 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:12.591622  2352 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:12.602243  2352 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:12.604241  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:12.617120  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:12.618013  1708 raft_consensus.cc:606] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:12.618868  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:12.619452  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 1892) took 1353 ms (client timeout 1568 ms). Trace:
I20260709 13:00:12.619616  3616 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:12.619956  1707 rpcz_store.cc:269] 0709 13:00:11.265719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.265859 (+   140us) service_pool.cc:224] Handling call
0709 13:00:12.619248 (+1353389us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b5400224554. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:12.619358 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1827157888}
I20260709 13:00:12.622062  3532 tablet_bootstrap.cc:700] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent opening tablet: real 0.135s	user 0.003s	sys 0.003s
W20260709 13:00:12.625226  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.600481  1629 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:12.608103  3539 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:12.672098  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:12.923360  1704 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.923561  3552 tablet_bootstrap.cc:492] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
W20260709 13:00:12.929734  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 298765us
I20260709 13:00:12.930985  3590 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.939088  1704 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:12.947283   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:12.920971  3538 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:12.670418  3604 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:12.950198  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:12.964370  3557 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.965062  3557 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:12.965644  3515 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:12.966246  3515 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:12.967408  3557 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:12.968269  3515 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:12.969568  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 296103us
I20260709 13:00:12.761716  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:12.974820  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:12.974471  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:12.975039  3557 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:12.970292  3604 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:12.983848  3604 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:12.997378  3565 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.006273  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:13.001670  3557 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:13.007946  3639 ts_tablet_manager.cc:933] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:13.018105  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:13.010989  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:13.023970  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 72327.4us
I20260709 13:00:12.998173  3565 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.026774  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:13.019212  3557 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.020393  3515 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:13.062362  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:13.008841  3626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:52636 (request call id 11) took 1796 ms. Trace:
I20260709 13:00:13.075320  1629 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 3. Ops: []
I20260709 13:00:13.075938  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:13.084487  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 75286.7us
I20260709 13:00:13.060588  3390 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:13.122098  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 67050.5us
I20260709 13:00:13.109047  3515 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:13.103966  3390 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:13.117837  3539 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:13.145224  3390 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:13.134001  3531 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:00:13.156719  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:13.136492  3565 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:13.160833  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:13.160199  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:13.162228  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:13.151964  3649 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.071465  3626 rpcz_store.cc:276] 0709 13:00:11.208959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.209109 (+   150us) service_pool.cc:224] Handling call
0709 13:00:11.251625 (+ 42516us) log.cc:844] Serialized 506 byte log entry
0709 13:00:11.500223 (+248598us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b54001c0fa0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27653142 00007fda27633eaa 00007fda2763ad53 00007fda2763acd9 00007fda2763ac69 00007fda2763ac31 00007fda27639dfd 000000000038f354 000000000038f199 00007fda245ed475
0709 13:00:11.730283 (+230060us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b1800344d40. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda27633b0b 00007fda276b8b63 00007fda276e0752 00007fda2c6a0d29
0709 13:00:13.004969 (+1274686us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":479224,"raft.queue_time_us":256720,"raft.run_cpu_time_us":2706,"raft.run_wall_time_us":24350,"spinlock_wait_cycles":30013952,"thread_start_us":224640,"threads_started":1,"wal-append.queue_time_us":283944,"wal-append.run_cpu_time_us":3854,"wal-append.run_wall_time_us":1134738}
I20260709 13:00:13.169128  3649 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.172627  3649 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:13.172871  1627 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
W20260709 13:00:13.174633  3640 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:00:13.185724  3515 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:13.198027  3637 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.258s	user 0.000s	sys 0.004s
I20260709 13:00:13.216347  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:13.225607  3577 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:13.240358  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:13.242132  1561 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:13.255697  1633 raft_consensus.cc:2955] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:13.259584  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:13.260031  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:13.260591  1560 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:13.262957  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:13.264200  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:13.264991  1561 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:13.250349  3577 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:13.270082  3552 tablet_bootstrap.cc:700] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent opening tablet: real 0.115s	user 0.004s	sys 0.000s
I20260709 13:00:13.283354  1630 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:13.285151  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:13.259480  3615 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:13.294945  3639 tablet_copy_client.cc:323] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:45431
I20260709 13:00:13.303714  1555 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 3. Ops: []
I20260709 13:00:13.318182  3615 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.297294  3644 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:13.270473  3577 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:13.326099  1675 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:13.334867  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:13.353386  1675 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:13.353982  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 195151us
I20260709 13:00:13.333678  3528 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:13.366019  3528 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.360070  1701 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:13.381013  1701 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:13.334122  3644 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:13.337447  3615 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:13.391917  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 1870) took 1221 ms. Trace:
I20260709 13:00:13.394695  1554 rpcz_store.cc:276] 0709 13:00:12.157269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.157419 (+   150us) service_pool.cc:224] Handling call
0709 13:00:13.378833 (+1221414us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400230d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:13.378996 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":923680000}
I20260709 13:00:13.397200  1701 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.387612  3631 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:13.414275  1701 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.426173  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:13.427901  1635 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:13.430951  3644 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.446283  3390 raft_consensus.cc:993] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:00:13.446914  3390 raft_consensus.cc:1081] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:00:13.452307  1633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 13:00:13.452930  1633 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:13.453258  1633 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.419253  3528 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:13.467746  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:13.481115  1701 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:13.491284  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:13.497797  1633 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.492071  1553 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:13.514919  1719 tablet_copy_service.cc:140] P a2ba7fc60f9641cdac25f1e016ef91d8: Received BeginTabletCopySession request for tablet 61a6c336f56b4c248aac90993fb477cf from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:52646)
I20260709 13:00:13.515684  1719 tablet_copy_service.cc:161] P a2ba7fc60f9641cdac25f1e016ef91d8: Beginning new tablet copy session on tablet 61a6c336f56b4c248aac90993fb477cf from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:52646: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-61a6c336f56b4c248aac90993fb477cf
I20260709 13:00:13.521373  1525 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:13.522656  3557 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.533581   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:13.485365  3515 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.540939  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:13.502765  3551 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 3 is higher.
I20260709 13:00:13.500754  1633 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:13.545423  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:13.545828  1628 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:13.552052  1628 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:13.511888  3531 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:13.555107  2353 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.567803  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:13.565980  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:13.540966  3515 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.560132  1628 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:13.575599  1628 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.576143  1719 tablet_copy_source_session.cc:215] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:00:13.577665  3526 consensus_queue.cc:1175] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:13.582278  1455 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 268) took 2489 ms. Trace:
I20260709 13:00:13.582793  1455 rpcz_store.cc:276] 0709 13:00:11.092755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.092973 (+   218us) service_pool.cc:224] Handling call
0709 13:00:13.582194 (+2489221us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:11.887549 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:12.042464 (+154915us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:12.042511 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:00:12.043348 (+   837us) tablet.cc:662] Decoding operations
0709 13:00:12.043746 (+   398us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:12.043804 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:12.043840 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:12.044042 (+   202us) tablet.cc:701] Row locks acquired
0709 13:00:12.044061 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:00:12.044216 (+   155us) write_op.cc:270] Start()
0709 13:00:12.044358 (+   142us) write_op.cc:276] Timestamp: P: 1783602012044169 usec, L: 0
0709 13:00:12.044383 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:00:12.045154 (+   771us) log.cc:844] Serialized 595 byte log entry
0709 13:00:12.377311 (+332157us) op_driver.cc:464] REPLICATION: finished
0709 13:00:12.658976 (+281665us) write_op.cc:301] APPLY: starting
0709 13:00:12.662918 (+  3942us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:12.702140 (+ 39222us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:12.760631 (+ 58491us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:12.761110 (+   479us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:12.761400 (+   290us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:12.761506 (+   106us) write_op.cc:312] APPLY: finished
0709 13:00:12.761835 (+   329us) log.cc:844] Serialized 23 byte log entry
0709 13:00:12.762194 (+   359us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:12.762325 (+   131us) write_op.cc:454] Released schema lock
0709 13:00:12.762402 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:12.762973 (+   571us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:12.763016 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":417453,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":253548,"apply.run_cpu_time_us":3338,"apply.run_wall_time_us":125105,"num_ops":1,"prepare.queue_time_us":149881,"prepare.run_cpu_time_us":17945,"prepare.run_wall_time_us":244096,"raft.queue_time_us":75519,"raft.run_cpu_time_us":19789,"raft.run_wall_time_us":213444,"replication_time_us":332759,"thread_start_us":231398,"threads_started":4,"wal-append.queue_time_us":168371}]]}
I20260709 13:00:13.589128  1456 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 269) took 2322 ms. Trace:
I20260709 13:00:13.593277  1456 rpcz_store.cc:276] 0709 13:00:11.267079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.267303 (+   224us) service_pool.cc:224] Handling call
0709 13:00:13.589029 (+2321726us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":239903,"reported_tablets":4}
I20260709 13:00:13.595917  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:13.597085  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:13.605232  1628 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:13.606335  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 320) took 1002 ms. Trace:
I20260709 13:00:13.608317  3532 tablet_bootstrap.cc:492] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:13.606729  1628 rpcz_store.cc:276] 0709 13:00:12.603524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.603927 (+   403us) service_pool.cc:224] Handling call
0709 13:00:13.545698 (+941771us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b54004a3e20. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27651dc0 00007fda276dde18 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6
0709 13:00:13.606267 (+ 60569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1745609984}
I20260709 13:00:13.853402  3504 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:13.853158  3649 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:13.890023  3504 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.890125  3649 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.890659  3504 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.890722  3649 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:13.852908  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 255848us
W20260709 13:00:13.893404  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 288707us
W20260709 13:00:13.893781  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 326043us
W20260709 13:00:13.852609  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 282538us
I20260709 13:00:13.609826  1635 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:13.867255  1458 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 270) took 1619 ms. Trace:
I20260709 13:00:13.881956  3515 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:13.602336  3647 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:13.895629  1458 rpcz_store.cc:276] 0709 13:00:12.248012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.250964 (+  2952us) service_pool.cc:224] Handling call
0709 13:00:13.867159 (+1616195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24663,"reported_tablets":6}
I20260709 13:00:13.915488  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:13.916249  1633 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:13.904619  3647 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:13.922818  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.917454  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:13.929723  3390 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:13.930744  3390 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.932703  3646 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:13.933359  3646 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.937137  3643 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:13.929876  3538 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:13.940323  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:13.933781  3558 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:13.933182  3390 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:13.948580  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:13.949623  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:13.956415  1701 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.956889  1701 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:13.914098  3549 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:13.961997  3549 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.912881  3669 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:13.928260  3647 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.970523  3652 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:13.949859  3532 tablet_bootstrap.cc:492] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 13:00:13.982642  1701 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.984730  1701 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:13.984926  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 67594.9us
I20260709 13:00:13.935817  3646 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:13.937786  3643 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.965372  3549 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:13.946725  3311 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:13.974085  3669 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:14.012415  1676 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:14.022464  1701 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:13.946765  3538 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.020955  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:14.015375  3669 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:14.021080  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:14.041095  1525 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:13.954509  3558 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:14.009219  3544 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:14.060446  2351 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:14.066570  1559 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.033900  3551 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:14.036803  3532 ts_tablet_manager.cc:1403] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 1.724s	user 0.049s	sys 0.045s
I20260709 13:00:14.021659  3311 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:13.989637  3652 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.055724  3557 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:14.074015  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 147568us
I20260709 13:00:14.057169  3604 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:14.090703  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 171193us
I20260709 13:00:14.108750  3311 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:14.109380  1628 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:14.113440  3580 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.114126  3580 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.116499  3580 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:14.132653  3639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61a6c336f56b4c248aac90993fb477cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:14.137562  1554 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:14.138007  1559 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:14.138885  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:14.139456  1552 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.090948  3557 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.142077  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:14.155597  3532 raft_consensus.cc:359] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:14.156682  3532 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
W20260709 13:00:14.157706  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 54970us
W20260709 13:00:14.162643  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 94460.3us
I20260709 13:00:14.163431  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:14.161552  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:14.166009  3538 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:14.169104  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:14.173763  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:14.174130  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:14.093542  3531 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:00:14.179471  1459 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:14.190730  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:14.191077  3646 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:14.192255  3646 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.192677  3646 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:14.146581  3552 tablet_bootstrap.cc:492] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:14.204403  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:14.204845  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:14.206190  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:14.140358  3677 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.147049  3557 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:14.145884  3652 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:14.144392  3643 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:14.206722  3649 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.199579  3597 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:14.212131  3678 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:00:14.235981  3646 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.236454  3558 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:14.233172  1633 raft_consensus.cc:2368] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3.
I20260709 13:00:14.242822  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:14.250094  1553 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:14.232051  3597 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:14.210072  3613 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:14.238870  3552 tablet_bootstrap.cc:492] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 13:00:14.226887  3504 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.255780  3597 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.238322  3646 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:14.231349  3649 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:14.294878  3544 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:14.202641  3686 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:00:14.258395  3613 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.168717  3532 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:14.336122  3544 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:14.454135  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:14.456766  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:14.457710  1635 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:14.485070  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:14.469780  3552 ts_tablet_manager.cc:1403] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 1.546s	user 0.049s	sys 0.060s
I20260709 13:00:14.490042  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:14.512962  3532 ts_tablet_manager.cc:1434] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.444s	user 0.007s	sys 0.003s
I20260709 13:00:14.515507  1635 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:14.537302   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:14.514950  3504 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:14.537835  3613 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:14.634997  1634 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.625133  3544 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:14.680796  3695 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:00:14.699615  3698 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:14.700239  3571 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.725399  3571 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:14.705407  3647 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.715s	user 0.000s	sys 0.014s
I20260709 13:00:14.759671  3692 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:14.702175  3698 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.778411  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:14.760389  3692 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:14.782675  3677 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:14.786226  3677 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:14.789247  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:14.796463  3652 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.798130  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:14.801348  3697 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.802063  3697 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.806459  3551 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:14.807087  3551 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.810421  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:14.790922  3571 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:14.813175  1706 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:14.813659  1710 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:14.747547  3647 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:14.813769  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:14.815968  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:14.766914  3552 raft_consensus.cc:359] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:14.788097  3677 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.025583  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:14.816515  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:14.946806  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 73932.5us
I20260709 13:00:14.841054  1674 leader_election.cc:400] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3.
I20260709 13:00:15.044574  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:15.048273  1674 leader_election.cc:403] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:14.808219  3669 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.741s	user 0.012s	sys 0.001s
W20260709 13:00:15.051465  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 210553us
I20260709 13:00:14.816926  1710 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 13:00:14.865147  1555 raft_consensus.cc:2368] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3.
I20260709 13:00:14.853706  3697 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:15.063274  1630 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:15.065156  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:15.065284  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:15.068248  1635 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:15.060571  3631 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.072335  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:15.073534  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:15.074270  1630 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:15.074343  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:15.025869  3698 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:15.031615  3551 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:14.945742  3571 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:15.033439  3615 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:14.840596  3699 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:14.852295  3628 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:15.085783  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:15.085984  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:15.086555  1702 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:15.049419  3552 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 13:00:15.071833  3580 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:15.082369  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:15.087086  1701 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:00:15.062866  3669 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.083832  3690 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:15.093811  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:15.078564  3698 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.101871  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:15.076607  3647 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:15.103905  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:15.116322  1705 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:15.128268  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.087332  3692 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:15.130702  3528 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:15.151515  3528 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.151958  3528 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.153067  3576 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:15.153760  3576 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.097566  3580 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:15.109658  3628 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.155982  3576 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:15.157016  3580 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:15.090751  3571 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.090452  3615 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.109035  3531 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:00:15.176442  3531 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:15.176899  3531 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:15.177803  3531 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.179134  3531 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:15.148160  3574 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:15.115355  3698 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.186827  3615 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:15.199679  1602 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e -> Peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Couldn't send request to peer 95fc6a4cf5874b7eb91280dffcd3b3fb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 316c6fe30c8f416c942b4338f4e6eab9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:15.208127  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:15.210027  2348 raft_consensus.cc:1217] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:15.209029  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:15.109254  3699 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.157034  3628 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:15.118947  3552 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:15.207700  3686 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.201746  3576 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.134209  3690 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:15.270028  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 194477us
I20260709 13:00:15.275123  1556 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:00:15.461115  3639 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.690s	user 0.009s	sys 0.009s
I20260709 13:00:15.644995  3639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59274 (request call id 311) took 2949 ms. Trace:
I20260709 13:00:15.645083  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:15.646288  1556 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:15.279920  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:15.506672  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 446895us
I20260709 13:00:15.312681  3669 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:15.274125  3701 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:15.207432  3672 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:15.589393  3631 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.520s	user 0.013s	sys 0.007s
I20260709 13:00:15.595671  3686 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.525086  3528 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.651192  3672 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.277051  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:15.314365  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 1900) took 1180 ms. Trace:
I20260709 13:00:15.669664  3669 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:15.670034  3528 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:15.669569  1555 rpcz_store.cc:276] 0709 13:00:14.134249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.140494 (+  6245us) service_pool.cc:224] Handling call
0709 13:00:15.314080 (+1173586us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b54003205d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:15.314281 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616684288}
I20260709 13:00:15.670248  3672 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:15.489938  1705 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:15.375241  3698 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:15.300427  3574 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.602458  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:15.276955  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:15.672288  1559 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:15.579133  3644 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:15.604689  3690 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:15.501768  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 101277us
I20260709 13:00:15.404524  3704 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:15.312129  3699 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:15.607967  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 330461us
I20260709 13:00:15.596932  1458 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 3 to 4, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:15.680531  1458 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 5, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 4 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:15.610967  3649 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:15.612123  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:15.613063  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:15.613178  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:15.614980  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:15.615980  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:15.615918  3576 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.618937  3599 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:15.743067  1558 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:15.619674  3522 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:15.401558  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:15.624069  3693 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:15.749711  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:15.626824  3707 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:15.627393  3615 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:15.627638  3552 ts_tablet_manager.cc:1434] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 1.135s	user 0.020s	sys 0.028s
I20260709 13:00:15.628731  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:15.586759  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:15.630167  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:15.632042  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.637934  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:15.637979  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:15.645510  3639 rpcz_store.cc:276] 0709 13:00:12.695939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.706117 (+ 10178us) service_pool.cc:224] Handling call
0709 13:00:13.026014 (+319897us) ts_tablet_manager.cc:934] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
0709 13:00:14.036737 (+1010723us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:00:14.764584 (+727847us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:14.770233 (+  5649us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:00:15.643997 (+873764us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:00:15.644493 (+   496us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:15.644864 (+   371us) ts_tablet_manager.cc:989] Replica registered.
0709 13:00:15.644925 (+    61us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":39638,"tablet-copy.queue_time_us":269172,"thread_start_us":356742,"threads_started":2}
I20260709 13:00:15.581116  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:15.676977  3616 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:15.656325  3631 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3.
I20260709 13:00:15.657032  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:15.666327  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:15.676416  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:15.676537  1560 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.682803  3684 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:15.699998  1458 catalog_manager.cc:5697] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 1 to 3, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.799503  1458 catalog_manager.cc:5697] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 3 to 4, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.707551  1631 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:15.670523  3669 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.619607  3659 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:15.813021  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:15.766268  3552 ts_tablet_manager.cc:1467] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Tablet startup took 1277ms
I20260709 13:00:15.749524  3686 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:15.662216  3596 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:15.583531  3571 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:15.672225  3558 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:15.683043  3698 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.757644  1632 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:15.758490  3705 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:15.758785  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:15.808616  3639 tablet_copy_client.cc:806] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:15.761495  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:15.682830  3528 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:15.762813  3599 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.772374  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:15.684221  3692 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:15.785878  3685 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:15.850698  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:15.814188  3626 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:15.792439  1703 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:15.793669  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:15.795368  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 157486us
I20260709 13:00:15.801308  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:15.755864  3605 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:15.760499  3652 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:15.761006  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:15.810851  1634 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:00:15.678176  3574 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.746965  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:15.760135  3704 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.817423  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:15.685518  3644 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.751502  3649 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.827605  1560 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:15.746948  1633 raft_consensus.cc:2368] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3.
I20260709 13:00:15.649603  3701 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.835964  1675 leader_election.cc:400] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3.
I20260709 13:00:15.853648  3639 tablet_copy_client.cc:670] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:15.805927  3526 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:15.842273  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 80875.6us
I20260709 13:00:15.872916  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:15.811277  3531 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:15.828855  3596 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.789868  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:15.851847  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 79581.7us
I20260709 13:00:15.855535  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:15.763417  3693 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.765657  3707 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.790362  3615 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:15.760833  3522 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.857393  3597 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:15.806241  1719 tablet_copy_service.cc:342] P a2ba7fc60f9641cdac25f1e016ef91d8: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-637e660a66be4924866287fa6150aaea received from {username='slave'} at 127.0.0.1:52646
I20260709 13:00:15.874258  1704 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:15.840615  3672 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:00:15.851423  1632 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.829339  3698 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.877985  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:15.844739  3528 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.801426  3616 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.868772  3651 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:15.887992  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:15.888489  3539 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:15.837450  3558 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:15.853075  3692 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:15.910334  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:15.849781  3679 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:00:15.861037  3626 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.820355  3659 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.906670  1674 leader_election.cc:400] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3.
I20260709 13:00:15.946460  1674 leader_election.cc:403] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:00:15.919359  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:15.889626  3671 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:15.864919  3652 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.893029  3715 ts_tablet_manager.cc:933] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:15.862097  3605 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.868623  3681 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.107s
I20260709 13:00:15.911567  1675 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:15.912201  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:15.900236  1719 tablet_copy_service.cc:434] P a2ba7fc60f9641cdac25f1e016ef91d8: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-637e660a66be4924866287fa6150aaea on tablet 637e660a66be4924866287fa6150aaea with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:15.859395  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:15.945623  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:15.882329  3686 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:00:15.877624  3526 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:15.954288  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47777.4us
I20260709 13:00:15.851207  3705 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.866499  3685 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:15.883067  3531 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:15.817427  3684 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:15.955497  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45268.1us
W20260709 13:00:16.006469  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 60971.4us
I20260709 13:00:16.103849  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:16.007757  1558 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:00:16.008946  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:16.010550  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 98403.2us
W20260709 13:00:16.011190  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 99706.6us
W20260709 13:00:16.094326  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 139506us
I20260709 13:00:15.889981  3597 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:16.102345  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:16.000389  3390 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3.
I20260709 13:00:16.010488  3480 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.978494  3577 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:16.112620  1675 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:15.915096  3539 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.953639  3584 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:16.119874  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 146407us
I20260709 13:00:15.953909  3646 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:16.018954  3667 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:16.138866  3676 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:16.001200  3647 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 13:00:16.153812  3705 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:15.917068  3673 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.223s
I20260709 13:00:16.021090  3700 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:16.113248  3693 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.111008  3576 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.120375  3669 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.132053  3597 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.119181  3599 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:16.178108  3703 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:00:16.199429  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:16.202509  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:16.204306  3704 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.205037  3709 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:16.205132  3715 tablet_copy_client.cc:323] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:37097
I20260709 13:00:16.195073  1527 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:16.223945  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:15.994949  3716 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.240s
I20260709 13:00:16.187860  3584 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:16.230738  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:16.248178  1632 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:16.008742  3671 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:16.168200  3646 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:16.264943  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 70013.4us
I20260709 13:00:16.300738  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:16.228345  3718 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
I20260709 13:00:16.272679  3711 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:16.279062  3707 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.244415  3584 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.257895  1706 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:16.282903  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:16.259172  3565 raft_consensus.cc:2764] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:16.306582  3596 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.244923  3574 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:16.332845  1631 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:16.225129  3717 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.436s
I20260709 13:00:16.303128  3685 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.309883  3698 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:16.268860  3522 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.326323  3626 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.326009  3692 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.303911  3528 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:16.330632  3652 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.370301  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:16.380107  3646 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.382364  3725 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.383126  3725 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:16.393129  3605 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.398279  3727 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:16.398835  3721 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:16.405623  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 293084us
W20260709 13:00:16.415825  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.006s	sys 0.023s
W20260709 13:00:16.416235  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.006s	sys 0.023s
W20260709 13:00:16.418409  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 556657us
I20260709 13:00:16.421300  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:16.421300  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:15.818681  3552 ts_tablet_manager.cc:1469] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Trace:
0709 13:00:09.819083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.819316 (+   233us) service_pool.cc:224] Handling call
0709 13:00:09.883592 (+ 64276us) ts_tablet_manager.cc:934] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
0709 13:00:11.493028 (+1609436us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:00:11.846147 (+353119us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:11.848561 (+  2414us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:00:12.110525 (+261964us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:00:12.110857 (+   332us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:12.111191 (+   334us) ts_tablet_manager.cc:989] Replica registered.
0709 13:00:12.111265 (+    74us) inbound_call.cc:177] Queueing success response
0709 13:00:12.923101 (+811836us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:00:14.499399 (+1576298us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:00:14.669178 (+169779us) tablet_replica.cc:261] Starting instrumentation
0709 13:00:14.670107 (+   929us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":33584,"raft.queue_time_us":29875,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":142,"tablet-copy.queue_time_us":62696,"tablet-download-pool.queue_time_us":333,"tablet-download-pool.run_cpu_time_us":1984,"tablet-download-pool.run_wall_time_us":23499,"thread_start_us":119414,"threads_started":2}
I20260709 13:00:16.430415  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:16.440601  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:16.443508  3725 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.420379  3605 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:00:16.411500  3659 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.446350  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:16.428864  3684 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.460224  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:16.432655  3711 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:16.438781  3721 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:16.480719  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 348766us
I20260709 13:00:16.461638  3605 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:16.448767  3528 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:16.497653  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:16.498682  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:16.501008  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:16.503202  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:16.502708  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:16.504765  1555 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
W20260709 13:00:16.491039  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 350370us
I20260709 13:00:16.448351  3679 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:16.444047  3727 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:16.510852  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:16.494694  3605 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:16.518924  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.522586  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:16.523003  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.524715  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:16.525446  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:16.528743  3501 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:16.511464  3721 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:16.539209  3729 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:16.539588  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1456) took 1263 ms. Trace:
I20260709 13:00:16.539817  3729 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:16.534533  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:16.540015  1704 rpcz_store.cc:276] 0709 13:00:15.276326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.276685 (+   359us) service_pool.cc:224] Handling call
0709 13:00:16.539412 (+1262727us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400224f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:16.539521 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463032576}
I20260709 13:00:16.542356  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:16.536221  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:16.543764  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:16.508410  3671 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.532632  3732 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:16.541153  3501 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:16.552731  3501 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:16.553588  3723 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:16.555824  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:16.534423  3652 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:16.563336  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.000s	sys 0.031s
W20260709 13:00:16.563759  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.000s	sys 0.031s
I20260709 13:00:16.568418  1718 tablet_copy_service.cc:342] P a2ba7fc60f9641cdac25f1e016ef91d8: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-8785cfce0d0042d3855cf9a7edaf1f7d received from {username='slave'} at 127.0.0.1:52646
I20260709 13:00:16.569027  1718 tablet_copy_service.cc:434] P a2ba7fc60f9641cdac25f1e016ef91d8: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-8785cfce0d0042d3855cf9a7edaf1f7d on tablet 8785cfce0d0042d3855cf9a7edaf1f7d with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:16.571873  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.572799  1704 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 1.
I20260709 13:00:16.577178  1702 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:16.563508  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:16.578737  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.579474  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:16.580724  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:16.579381  1702 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:16.554481  3723 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:16.585235  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:16.585870  3731 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:16.586474  3731 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:16.588733  3731 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.589074  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.590932  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:16.591867  3731 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:16.553727  3732 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:16.596449  3528 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.596930  3528 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.597317  3528 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.594903  1646 tablet_copy_service.cc:140] P ba0548b7d2a94e8598475a46c279fe0e: Received BeginTabletCopySession request for tablet 316c6fe30c8f416c942b4338f4e6eab9 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb ({username='slave'} at 127.0.0.1:48246)
I20260709 13:00:16.595018  3580 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:16.598053  1646 tablet_copy_service.cc:161] P ba0548b7d2a94e8598475a46c279fe0e: Beginning new tablet copy session on tablet 316c6fe30c8f416c942b4338f4e6eab9 from peer 95fc6a4cf5874b7eb91280dffcd3b3fb at {username='slave'} at 127.0.0.1:48246: session id = 95fc6a4cf5874b7eb91280dffcd3b3fb-316c6fe30c8f416c942b4338f4e6eab9
I20260709 13:00:16.571736  3659 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.606237  3734 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:16.598107  3580 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:16.607415  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.613932  1701 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:16.595470  3723 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:16.610291  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:16.608702  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:16.621608  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:16.564385  3652 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:16.627334  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:16.623165  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 97533.6us
I20260709 13:00:16.629706  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:16.629968  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:16.633157  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:16.639621  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:16.615721  3580 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:16.576568  3605 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:16.662235  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.662516  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.664213  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.669217  1705 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:16.669505  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.670092  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.670595  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.625777  3501 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:16.667143  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.668169  1709 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:00:16.673167  1709 pending_rounds.cc:85] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 5
I20260709 13:00:16.673506  1709 pending_rounds.cc:107] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:00:16.673816  1709 raft_consensus.cc:2889] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:16.675783  3662 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:00:16.595642  3729 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:16.678810  3672 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:16.679415  3672 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:16.679937  3672 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:16.684181  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 580) took 1049 ms. Trace:
I20260709 13:00:16.613659  3659 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:16.659166  3676 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:16.684815  2349 rpcz_store.cc:276] 0709 13:00:15.635027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.635153 (+   126us) service_pool.cc:224] Handling call
0709 13:00:15.635470 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:15.878572 (+243102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.878592 (+    20us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:15.878622 (+    30us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:15.878641 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.683674 (+805033us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5400541ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:16.683741 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.683903 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558605056}
I20260709 13:00:16.694993  3732 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.695371  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.667794  3693 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:16.651391  3734 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:16.674899  3669 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.667848  3626 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:16.714737  3616 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.714974  3734 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:16.722309  3730 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.722884  3730 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:16.723263  3730 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.704973  3693 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:16.708824  3669 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:16.710373  3626 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:16.864243  1459 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 183) took 2794 ms. Trace:
I20260709 13:00:16.839074  3480 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:16.865882  3693 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:16.738085  3683 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:16.866668  3480 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:16.813987  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.915452  3480 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.922331   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:16.758690  3649 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.852906  3549 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:16.740837  3744 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:00:16.768563  3644 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:16.762718  3639 tablet_copy_client.cc:538] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:16.854236  3701 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:16.933809  3702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.040s	sys 0.125s
I20260709 13:00:16.821177  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:16.956560  3549 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:16.973397  3702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.041s	sys 0.127s
I20260709 13:00:16.750578  3652 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:16.754616  3659 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.727063  3558 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:16.990440  3626 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:16.963599  3549 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.029771  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.038646  3672 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:16.865217  1459 rpcz_store.cc:276] 0709 13:00:14.077879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.078226 (+   347us) service_pool.cc:224] Handling call
0709 13:00:16.863783 (+2785557us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:14.513170 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:14.513901 (+   731us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:14.513937 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:00:14.513956 (+    19us) tablet.cc:662] Decoding operations
0709 13:00:14.536608 (+ 22652us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:14.539547 (+  2939us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:14.607409 (+ 67862us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:14.624061 (+ 16652us) tablet.cc:701] Row locks acquired
0709 13:00:14.625589 (+  1528us) write_op.cc:260] PREPARE: finished
0709 13:00:14.657614 (+ 32025us) write_op.cc:270] Start()
0709 13:00:14.671110 (+ 13496us) write_op.cc:276] Timestamp: P: 1783602014657515 usec, L: 0
0709 13:00:14.695937 (+ 24827us) op_driver.cc:348] REPLICATION: starting
0709 13:00:14.766547 (+ 70610us) log.cc:844] Serialized 536 byte log entry
0709 13:00:15.585764 (+819217us) op_driver.cc:464] REPLICATION: finished
0709 13:00:15.866943 (+281179us) write_op.cc:301] APPLY: starting
0709 13:00:15.872805 (+  5862us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:15.879603 (+  6798us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:15.916627 (+ 37024us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:15.956243 (+ 39616us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:16.022780 (+ 66537us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:16.074122 (+ 51342us) write_op.cc:312] APPLY: finished
0709 13:00:16.109329 (+ 35207us) log.cc:844] Serialized 23 byte log entry
0709 13:00:16.109620 (+   291us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:16.109726 (+   106us) write_op.cc:454] Released schema lock
0709 13:00:16.109778 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:16.110252 (+   474us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:16.110295 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":272017,"apply.run_cpu_time_us":3348,"apply.run_wall_time_us":250159,"num_ops":1,"prepare.queue_time_us":329339,"prepare.run_cpu_time_us":24511,"prepare.run_wall_time_us":431336,"raft.queue_time_us":432560,"raft.run_cpu_time_us":54914,"raft.run_wall_time_us":176956,"replication_time_us":889613,"thread_start_us":936891,"threads_started":5,"wal-append.queue_time_us":339110}]]}
I20260709 13:00:16.725533  3528 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.062564  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 307) took 1152 ms. Trace:
I20260709 13:00:17.073861  3672 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.080180  3725 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:17.081286  3725 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.083376  3526 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.084710  3725 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.070889  1710 rpcz_store.cc:276] 0709 13:00:15.909636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.909770 (+   134us) service_pool.cc:224] Handling call
0709 13:00:15.910065 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.013764 (+103699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.013798 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:16.013846 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:16.013881 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.056259 (+1042378us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400250dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:17.056346 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.062176 (+  5830us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238000384}
I20260709 13:00:17.087883  3531 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.096297  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1915) took 1334 ms. Trace:
I20260709 13:00:17.110131  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 211) took 1899 ms. Trace:
I20260709 13:00:17.110922  1556 rpcz_store.cc:276] 0709 13:00:15.210828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.737881 (+527053us) service_pool.cc:224] Handling call
0709 13:00:15.738178 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:15.902372 (+164194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.902394 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:15.902424 (+    30us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:15.902451 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.109463 (+1207012us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540038a9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:17.109558 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.109912 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376898432}
W20260709 13:00:17.109965  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 59852.8us
I20260709 13:00:16.749763  3723 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.116016  3528 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:17.119930  3727 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.122321  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:17.127257  3627 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:17.128756  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.128464  3627 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.131489  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.133997  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.135866  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.136464  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.137873  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.140239  3627 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.142017  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1491) took 1381 ms. Trace:
I20260709 13:00:17.142593  1558 rpcz_store.cc:276] 0709 13:00:15.759946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.836638 (+ 76692us) service_pool.cc:224] Handling call
0709 13:00:15.906354 (+ 69716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.105670 (+199316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.141400 (+1035730us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54003309d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:17.141736 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232235264}
I20260709 13:00:17.143425  1705 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:17.145066  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.146759  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 308) took 1236 ms. Trace:
I20260709 13:00:17.147118  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.147186  1703 rpcz_store.cc:276] 0709 13:00:15.910481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.913575 (+  3094us) service_pool.cc:224] Handling call
0709 13:00:15.913889 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.010869 (+ 96980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.010903 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:16.010951 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:16.010984 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.146284 (+1135300us) spinlock_profiling.cc:243] Waited 96.7 ms on lock 0x7b540036e2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:17.146350 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.146561 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222503680}
I20260709 13:00:17.148102  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.151117  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.153578  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:17.153805  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 431410us
I20260709 13:00:17.154386  1646 tablet_copy_source_session.cc:215] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:17.097123  1557 rpcz_store.cc:276] 0709 13:00:15.621025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.789358 (+168333us) service_pool.cc:224] Handling call
0709 13:00:15.903763 (+114405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:15.983505 (+ 79742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.983561 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:15.983644 (+    83us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:15.983713 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.917082 (+933369us) spinlock_profiling.cc:243] Waited 79.4 ms on lock 0x7b5400311854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:16.917267 (+   185us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.955636 (+ 38369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182583552}
I20260709 13:00:17.166477  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.168488  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.118897  3723 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.181236  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.237957  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:17.249317  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:17.193339  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:17.234189  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 58980.5us
I20260709 13:00:17.193362  3480 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:17.255966  3480 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:17.259101  3480 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.185627  3577 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:17.259999  3577 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.262831  3577 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:17.240577  3531 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:17.265417  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.270308  3652 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.270505  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.271381  1704 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:17.272957  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:17.275599  3687 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:17.276818  3597 consensus_meta.cc:287] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.049s	user 0.012s	sys 0.008s
I20260709 13:00:17.276558  3687 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.271113  3652 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.281497  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.263926  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.267580  3531 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.294332  3531 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.285043  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.239140  3576 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.310725  3577 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.323253  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1448) took 2238 ms. Trace:
I20260709 13:00:17.323807  1706 rpcz_store.cc:276] 0709 13:00:15.084572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.085536 (+   964us) service_pool.cc:224] Handling call
0709 13:00:17.322671 (+2237135us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b540014c8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:17.322987 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1666201856}
I20260709 13:00:17.324299  3390 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.324877  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.325147  3390 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.323348  3577 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.326462  3644 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:17.327198  3644 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.328662  3577 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.320839  3576 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.296491  3597 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.331415  3576 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:17.325462  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:17.331694  3652 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.333009  2345 raft_consensus.cc:1217] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:17.334594  3627 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:17.334754  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:17.335126  3627 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.338138  2347 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 13:00:17.184600  3693 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.335598  3627 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:17.349385  1630 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:17.353861  3390 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:17.345198  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47654.8us
I20260709 13:00:17.383692  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 214) took 1620 ms. Trace:
I20260709 13:00:17.384240  1560 rpcz_store.cc:276] 0709 13:00:15.762711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.884316 (+121605us) service_pool.cc:224] Handling call
0709 13:00:15.884775 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.013411 (+128636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.013452 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:16.013575 (+   123us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:16.013613 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.383168 (+1369555us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400313654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:17.383248 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.383486 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295368960}
I20260709 13:00:17.381211  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1922) took 1757 ms. Trace:
I20260709 13:00:17.412147  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.395085  1552 rpcz_store.cc:276] 0709 13:00:15.623198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.803089 (+179891us) service_pool.cc:224] Handling call
0709 13:00:15.803372 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:15.894353 (+ 90981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.894390 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:15.894441 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:15.894472 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.380554 (+1486082us) spinlock_profiling.cc:243] Waited 90.3 ms on lock 0x7b5400206f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:17.380652 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.380963 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207639296}
W20260709 13:00:17.508427  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 193240us
I20260709 13:00:17.497373  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:17.530821  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 211105us
W20260709 13:00:17.470044  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:17.474300  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 79160.2us
I20260709 13:00:17.534734  1602 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:17.541052  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:17.500178  3730 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.776s	user 0.015s	sys 0.004s
I20260709 13:00:17.545838  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.555843  3748 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.556602  3748 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.559146  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:17.551285  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:17.562315  3742 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:00:17.562822  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 162682us
I20260709 13:00:17.563472  3531 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:17.565310  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.566313  2346 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 13:00:17.567291  3605 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:17.550889  3639 tablet_bootstrap.cc:492] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 13:00:17.462872  3501 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:17.553375  3644 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.553531  3687 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:17.516379  3746 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.206s	user 0.000s	sys 0.002s
I20260709 13:00:17.562692  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.414397  3577 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:17.521667  3580 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.834s	user 0.017s	sys 0.001s
I20260709 13:00:17.420893  3480 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:17.599797  3480 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.599634  3580 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:17.601958  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.602463  3480 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.603430  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1497) took 1045 ms. Trace:
I20260709 13:00:17.506167  3754 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:17.603855  1559 rpcz_store.cc:276] 0709 13:00:16.558163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.558356 (+   193us) service_pool.cc:224] Handling call
0709 13:00:16.584086 (+ 25730us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.584439 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.584466 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.599033 (+ 14567us) log.cc:844] Serialized 26 byte log entry
0709 13:00:17.039683 (+440650us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:17.039822 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.039971 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:17.603011 (+563040us) raft_consensus.cc:1685] finished
0709 13:00:17.603031 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.603209 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10787,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":383585,"threads_started":1,"wal-append.queue_time_us":696052}
I20260709 13:00:17.604998  3727 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:17.605333  3754 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.559463  3748 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:17.577291  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.499511  3697 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:17.496923  3752 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.589213  3750 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.591527  3577 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.593281  3623 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:17.594826  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.595593  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.598016  3311 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:17.605738  3727 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.606379  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.515182  3693 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.610555  3526 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:17.611805  1635 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:17.625789  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.618666  3615 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:17.622992  2345 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 13:00:17.650856  3623 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.651444  1705 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:17.623205  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:17.643630  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:17.583238  3730 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:17.661161  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 46058.9us
I20260709 13:00:17.661741  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:17.662003  1630 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:17.663282  1557 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:17.663865  3311 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.665167  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.666711  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.667797  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.669431  3577 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:17.670440  3758 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:00:17.674039  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.690248  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:17.674847  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 140259us
I20260709 13:00:17.691707  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:17.622200  3687 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:17.705869  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.708809  1556 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:17.712632  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:17.716972  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.661878  3528 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:17.719897  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:17.710987  1601 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 3.324s (SENT)
I20260709 13:00:17.625919  3750 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.653537  3752 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.718427  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.706970  3501 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:17.714051  3644 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:17.731956  3644 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.732594  3644 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:17.734774  1705 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 1.
I20260709 13:00:17.735316  1706 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:17.736202  3626 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:17.736673  3626 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.737088  3626 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:17.738034  1636 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:17.738237  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.739745  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.740581  1556 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:17.731999  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.742024  1557 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:17.743099  3311 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.745406  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:17.726684  1709 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:17.729516  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.756325  3627 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.757542  3729 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.132s
I20260709 13:00:17.722724  3528 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.764613  1527 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:17.685449  3697 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.775246  3683 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:17.777180  3597 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.778554  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.778663  3623 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:17.779335  3580 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.779688  3599 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:17.780512  3599 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.769316  3528 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:17.783649  3599 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.784920  1458 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 270) took 3861 ms. Trace:
I20260709 13:00:17.762143  3501 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:00:17.791366  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 176450us
I20260709 13:00:17.729719  3687 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.808980  3750 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.810405  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 221) took 1251 ms. Trace:
I20260709 13:00:17.810868  1554 rpcz_store.cc:276] 0709 13:00:16.559146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.560997 (+  1851us) service_pool.cc:224] Handling call
0709 13:00:16.561425 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.810019 (+248594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.810136 (+   117us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:16.810263 (+   127us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:16.810346 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.809979 (+999633us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54003200d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:17.810029 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.810210 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":569510784}
I20260709 13:00:17.811767  3715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 316c6fe30c8f416c942b4338f4e6eab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.790899  3501 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.815030  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:17.800900  1459 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 3 to 4, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:17.799109  3730 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.803149  3599 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.817723  3615 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:17.785341  1458 rpcz_store.cc:276] 0709 13:00:13.923715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.923891 (+   176us) service_pool.cc:224] Handling call
0709 13:00:17.784695 (+3860804us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:16.219096 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:16.272924 (+ 53828us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:16.272975 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:00:16.272999 (+    24us) tablet.cc:662] Decoding operations
0709 13:00:16.373696 (+100697us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:16.379252 (+  5556us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:16.386511 (+  7259us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:00:16.398896 (+ 12385us) tablet.cc:701] Row locks acquired
0709 13:00:16.398920 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:00:16.399203 (+   283us) write_op.cc:270] Start()
0709 13:00:16.399342 (+   139us) write_op.cc:276] Timestamp: P: 1783602016399053 usec, L: 0
0709 13:00:16.399363 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:00:16.409935 (+ 10572us) log.cc:844] Serialized 1764 byte log entry
0709 13:00:17.090641 (+680706us) op_driver.cc:464] REPLICATION: finished
0709 13:00:17.353321 (+262680us) write_op.cc:301] APPLY: starting
0709 13:00:17.353449 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:17.380654 (+ 27205us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:17.380842 (+   188us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:17.382044 (+  1202us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:17.382416 (+   372us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:00:17.382468 (+    52us) write_op.cc:312] APPLY: finished
0709 13:00:17.382931 (+   463us) log.cc:844] Serialized 41 byte log entry
0709 13:00:17.383169 (+   238us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:17.383393 (+   224us) write_op.cc:454] Released schema lock
0709 13:00:17.383462 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:17.383978 (+   516us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:17.384009 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":262205,"apply.run_cpu_time_us":3811,"apply.run_wall_time_us":30668,"num_ops":4,"prepare.queue_time_us":355044,"prepare.run_cpu_time_us":6679,"prepare.run_wall_time_us":219218,"replication_time_us":689631,"thread_start_us":386780,"threads_started":2}]]}
I20260709 13:00:17.842430  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:17.838219  3697 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.820125  3501 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:17.845762  3669 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:17.846230  3584 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.476s	user 0.009s	sys 0.014s
I20260709 13:00:17.847695  3599 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:17.850695  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 89125.5us
W20260709 13:00:17.850853  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 142911us
I20260709 13:00:17.858319  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.838241  3504 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.862071  3584 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.862556  3504 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.863566  2351 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:17.858330  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.870999  3752 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.822302  3580 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:17.873584  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.873885  3580 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.876272  3580 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.876549  3687 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.876751  3727 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:17.878216  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 167275us
I20260709 13:00:17.882934  3754 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.883471  3652 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:17.884263  3652 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:17.857308  3605 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:17.887228  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.888060  1628 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:17.864419  3390 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:17.898301  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.898512  3390 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:17.900288  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.900537  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.901139  3390 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:17.897128  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.897351  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.896924  3754 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:17.902263  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.894691  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.940461  3528 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.941087  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.959033  3584 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.974180  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1929) took 1477 ms. Trace:
I20260709 13:00:17.974728  1561 rpcz_store.cc:276] 0709 13:00:16.496463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.496629 (+   166us) service_pool.cc:224] Handling call
0709 13:00:16.498051 (+  1422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.530647 (+ 32596us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:16.533598 (+  2951us) log.cc:844] Serialized 14 byte log entry
0709 13:00:17.260460 (+726862us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b180008eba0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:00:17.543797 (+283337us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:17.580518 (+ 36721us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:17.586222 (+  5704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.973774 (+387552us) spinlock_profiling.cc:243] Waited 32.3 ms on lock 0x7b5400230d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:17.973823 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.973981 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57061,"prepare.queue_time_us":155,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":64149,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":86055680,"thread_start_us":207917,"threads_started":1,"wal-append.queue_time_us":1007381}
I20260709 13:00:17.978066  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.979031  3531 raft_consensus.cc:1064] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:00:17.980280  3627 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.981618  3501 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.983326  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:17.901258  1633 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:17.899650  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.912432  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.991201  1628 raft_consensus.cc:2410] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:00:17.901561  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:17.906773  3693 raft_consensus.cc:2955] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:17.902818  3599 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:17.905826  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:17.995620  1631 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 1.
I20260709 13:00:17.904862  3504 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:17.919737  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 123491us
I20260709 13:00:17.850467  3626 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:17.933238  3754 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:18.000605  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:17.984004  3501 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:17.986469  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:17.907817  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:17.985940  3531 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:18.007913  1634 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 13:00:18.005910  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:18.006292  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:18.006723  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:18.011066  1705 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:18.014158  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.014412  1633 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:18.016168  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:18.016906  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.017021  1633 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:18.025578  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:18.025839  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:18.017831  3539 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.018225  1710 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:18.027135  1634 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:18.028326  1561 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:00:18.028741  1561 pending_rounds.cc:85] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 3
I20260709 13:00:18.028990  1561 pending_rounds.cc:107] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:00:18.029289  1561 raft_consensus.cc:2889] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:18.031862  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.019065  3669 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.034046  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.034384  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.035064  1555 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:18.029297  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.035943  1558 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:18.016317  3754 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149)
I20260709 13:00:18.038198  3584 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:18.038848  3584 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:18.017938  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:17.993180  3652 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:18.046358  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:18.042409  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.051867  1710 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:18.053220  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.047004  1705 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:18.047684  3584 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:18.060935  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:18.061690  3727 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.066461  3644 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:18.055758  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:18.056535  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:18.072237  1526 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:18.073151  3615 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.074020  3615 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:18.075855  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:18.077014  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:18.077247  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:18.077538  3605 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.077876  2348 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:18.078794  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:18.079653  3605 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.057358  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:18.057682  3652 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:18.057977  3626 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:18.007416  3390 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:00:18.082319  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:18.082871  3390 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:18.083276  3390 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:18.083335  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.084276  3390 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:18.059777  3539 raft_consensus.cc:1064] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:00:18.060015  3528 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:18.060129  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.060192  3576 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:18.049660  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:18.087175  1631 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:18.087105  3576 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:18.057083  1674 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.076442  3615 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:18.084708  2345 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:18.059645  1675 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.089052  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:18.092020  1636 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:18.097069  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:18.097218  3528 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:18.094463  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:18.097867  3576 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:18.100955  3528 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.102025  3528 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:18.103160  3763 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:18.103273  2348 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:18.103609  3763 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:18.104730  3763 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:18.106662  3763 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.094669  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:18.097733  3390 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.097759  3626 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:18.098421  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:18.094296  1629 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:18.107928  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:18.094650  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:18.109148  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:18.110839  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:18.111227  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:18.095813  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:18.116707  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.114776  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.118808  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.124202  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:18.124941  3645 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:18.125864  3584 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.126569  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:18.127458  3528 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:18.127612  3687 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.127959  3528 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:18.128484  3528 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:18.128284  3687 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:18.125010  2351 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
W20260709 13:00:18.131889  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 49637.7us
I20260709 13:00:18.132205  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.124265  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.127975  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:18.125772  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:18.141981  1557 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:18.144344  1526 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:18.149137  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:18.149201  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:18.154232  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:18.155555  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:18.156750  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:18.146991  3645 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:18.158253  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:18.149510  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:18.158885  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.158694  3645 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:18.146790  3615 consensus_queue.cc:1059] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:00:18.160393  3623 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:18.160638  3626 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:18.163271  3683 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:18.163823  3683 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:18.163242  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.164845  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:18.165436  2317 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.163188  3687 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:18.166616  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 49998.9us
I20260709 13:00:18.118862  3727 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:18.160667  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:18.166821  2317 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.164237  3683 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.171105  1710 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:18.170732  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.088492  3539 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:18.143834  3669 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:18.175457  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.179936  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.182775  1602 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:18.173466  3576 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:18.190507  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:18.172055  3501 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.190704  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.177964  3584 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:18.198266  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:18.189152  3623 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.191527  3599 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:18.222939  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.192181  3750 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:18.190722  3576 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:18.204466  3584 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:18.228160  3727 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:18.243693  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 75666.5us
I20260709 13:00:18.231146  3729 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:18.244979  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 128958us
W20260709 13:00:18.239786  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 65376.7us
I20260709 13:00:18.228595  3599 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:18.249662  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:18.251369  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:18.233506  3576 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:18.202811  3754 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
W20260709 13:00:18.266435  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 82465.7us
I20260709 13:00:18.267568  1674 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:18.269109  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:18.239609  3504 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:18.274935  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:18.245178  3584 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:18.263929  3599 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:18.279901  3752 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:00:18.274206  3669 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:18.292196  3681 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.293339  3697 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:18.307595  3697 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:18.308130  3697 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:18.309691  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 58458.9us
I20260709 13:00:18.311002  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:18.312419  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:18.293733  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:18.314637  3727 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:18.314877  3775 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:18.315114  3727 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:18.315623  3727 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:18.316992  3723 raft_consensus.cc:3055] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:18.318186  3770 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:00:18.298902  3779 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:18.304142  3707 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:18.304912  3669 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:18.320464  1525 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:18.272694  3693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:48236 (request call id 12) took 4704 ms. Trace:
I20260709 13:00:18.279029  3311 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.322449  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:18.319335  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 132617us
W20260709 13:00:18.339773  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 70739.8us
I20260709 13:00:18.345844  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:18.317423  3723 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:18.357007  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:18.341951  3752 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.367257  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:00:18.369825  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 186443us
I20260709 13:00:18.377416  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.380852  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:18.387954  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 141997us
I20260709 13:00:18.361347  3774 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:18.347568  3779 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:18.423811  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.350272  3707 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:18.363442  3528 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:18.380203  3752 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.387530  3748 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:18.432243  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.386246  3783 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:18.431164  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:18.427263  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 70361.3us
I20260709 13:00:18.436000  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:18.445750  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:18.454430  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 134072us
I20260709 13:00:18.464732  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:18.442499  3504 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:18.479779  3790 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:18.341030  3693 rpcz_store.cc:276] 0709 13:00:13.541870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.542002 (+   132us) service_pool.cc:224] Handling call
0709 13:00:13.924430 (+382428us) log.cc:844] Serialized 506 byte log entry
0709 13:00:14.513701 (+589271us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b1800398740. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda2765ab26 00007fda2765449a 00007fda27653714
0709 13:00:14.521739 (+  8038us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b54003401a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda2765ab26 00007fda2765449a 00007fda27653714 00007fda27633eaa 00007fda2763ad53 00007fda2763acd9 00007fda2763ac69 00007fda2763ac31 00007fda27639dfd 000000000038f354
0709 13:00:14.944645 (+422906us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b1800398740. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27653142 00007fda27633eaa 00007fda2763ad53
0709 13:00:15.129537 (+184892us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b180021ffc0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27716d01 00007fda27653691 00007fda27633eaa 00007fda2763ad53 00007fda2763acd9 00007fda2763ac69 00007fda2763ac31 00007fda27639dfd 000000000038f354 000000000038f199
0709 13:00:15.598044 (+468507us) spinlock_profiling.cc:243] Waited 9.19 ms on lock 0x7b54003401a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27653142 00007fda27633eaa 00007fda2763ad53 00007fda2763acd9 00007fda2763ac69 00007fda2763ac31 00007fda27639dfd 000000000038f354 000000000038f199 00007fda245ed475
0709 13:00:18.233287 (+2635243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60330,"raft.queue_time_us":2494,"raft.run_cpu_time_us":100070,"raft.run_wall_time_us":3049875,"spinlock_wait_cycles":44457856,"thread_start_us":92724,"threads_started":1,"wal-append.queue_time_us":190235,"wal-append.run_cpu_time_us":2259,"wal-append.run_wall_time_us":1103791}
I20260709 13:00:18.481058  3790 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:18.443774  3687 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:00:18.482295  3771 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:18.482777  3771 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:18.483162  3771 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.471040  3644 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:18.484607  3786 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:18.485896  3776 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:00:18.487401  3784 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:18.489108  3782 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:18.489567  3782 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.490001  3782 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.490283  3734 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:18.510512  3565 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:18.512363  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:18.515621  3627 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:18.516264  3627 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:18.517555  3627 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:18.470800  3504 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:18.530511  2345 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:18.531706  3772 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:18.534359  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 69759us
I20260709 13:00:18.535439  1675 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.533643  3504 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:18.490856  3734 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:18.563772  3734 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:18.563884  3777 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:00:18.466892  3773 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:18.568811  3565 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:18.569067  3652 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:18.569427  3780 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:00:18.570299  3768 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 13:00:18.434782  3723 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:18.596155  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.613657  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.502470  3790 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:18.619796  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:18.597388  3626 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:18.633049  1675 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.597935  3627 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:18.633536  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 202451us
I20260709 13:00:18.657166  1710 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:18.670066  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 118751us
I20260709 13:00:18.682480  2349 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:18.686806  1708 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:18.687022  3779 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:18.688241  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:18.689077  1708 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:00:18.689790  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:18.688145  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:18.691327  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.694800  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:18.695931  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:18.690253  3597 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.697883  3565 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:18.698055  3597 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:18.699189  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:18.700541  3597 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:18.690738  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:18.719241  3683 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.534s	user 0.020s	sys 0.008s
I20260709 13:00:18.719413  3707 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:18.719919  3683 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:18.638504  3626 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:18.732818  1556 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:18.728309  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:18.726275  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:18.720970  2344 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:18.745435  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:18.746364  1552 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:18.674925  3644 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:18.754837  1675 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:18.681257  3723 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:18.680146  3528 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:18.769512  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 115886us
W20260709 13:00:18.773916  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 63991us
I20260709 13:00:18.781569  1704 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:18.703238  3501 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:18.762204  3390 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.638s	user 0.016s	sys 0.015s
I20260709 13:00:18.784154  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:18.758065  3626 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.732131  3683 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:18.770205  3792 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:00:18.814263  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.730342  3752 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
W20260709 13:00:18.797338  3763 consensus_meta.cc:287] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.690s	user 0.013s	sys 0.006s
I20260709 13:00:18.728781  3599 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:18.848727  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 84252.7us
I20260709 13:00:18.823383  1560 raft_consensus.cc:2955] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:18.874668  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:18.878475  3754 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:18.879176  3754 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:18.879680  3754 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:18.878616  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 150399us
I20260709 13:00:18.881320  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:18.804605  3501 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:18.895450  3752 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:18.930212  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 75192.4us
I20260709 13:00:18.831775  3707 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 13:00:18.796002  3802 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:00:18.961416  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:18.813011  3644 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
W20260709 13:00:18.950698  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 76121.4us
I20260709 13:00:18.971948  1633 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:00:18.972457  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 91229.5us
I20260709 13:00:18.978443  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:00:18.993973  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 209047us
I20260709 13:00:18.997751  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:18.999948  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:19.000340  1629 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:19.011358  2345 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:18.943482  3752 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:18.994745  3697 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.686s	user 0.012s	sys 0.010s
W20260709 13:00:18.995428  3584 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.011s
W20260709 13:00:19.019775  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 50924.1us
W20260709 13:00:19.024096  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46244.6us
I20260709 13:00:19.002337  3597 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:19.015363  3669 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
W20260709 13:00:19.075788  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 49065.9us
W20260709 13:00:19.077029  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 299410us
I20260709 13:00:19.034351  3599 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:19.043010  3782 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.003s
I20260709 13:00:18.860986  1554 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:00:19.082082  1554 raft_consensus.cc:606] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:19.075441  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 57618.4us
I20260709 13:00:19.013645  3693 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:18.902782  3797 consensus_queue.cc:1059] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.125s
I20260709 13:00:19.077888  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:19.082494  1554 raft_consensus.cc:614] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:19.086802  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:19.024462  3565 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:19.078207  3504 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.516s	user 0.017s	sys 0.002s
I20260709 13:00:19.024256  3652 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:19.090565  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 63811.7us
I20260709 13:00:19.096606  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:19.076623  3597 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:19.086369  3693 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:19.130467  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:19.087110  3782 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.137714  1704 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:19.139240  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:19.028929  3697 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.084290  3779 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:00:19.102294  3597 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.097751  3599 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:19.132361  3803 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:00:19.028647  3584 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:19.148021  3727 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.832s	user 0.017s	sys 0.007s
I20260709 13:00:19.148483  2345 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:19.145640  3627 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.191257  1708 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:19.149222  3804 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:00:19.191316  3727 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:19.193889  3615 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.187101  3526 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.194715  3727 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:19.152096  3311 consensus_meta.cc:287] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.799s	user 0.005s	sys 0.014s
I20260709 13:00:19.195672  3311 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.152994  3707 consensus_queue.cc:1243] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer 95fc6a4cf5874b7eb91280dffcd3b3fb log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:00:19.154616  3802 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:19.196538  3802 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:19.170657  3734 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.606s	user 0.014s	sys 0.007s
I20260709 13:00:19.197418  3707 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:19.197574  3802 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.198659  3707 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:19.199056  3707 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:19.202616  3782 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:19.203670  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:19.203557  3683 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:19.204252  1701 raft_consensus.cc:2368] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4.
I20260709 13:00:19.133759  3565 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.132256  3752 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.163408  3797 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:19.205690  3797 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:19.197377  3734 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.206398  3565 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:19.206750  3797 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.208021  3734 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:19.170550  3789 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 13:00:19.133623  3693 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:19.174532  3715 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.981s	user 0.012s	sys 0.001s
I20260709 13:00:19.130895  3681 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.219336  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:19.186519  1706 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:19.209306  1702 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:19.204298  1705 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:19.212178  1707 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:19.198992  3802 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:19.097523  3504 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.217723  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:19.227844  1635 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:19.214598  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 136804us
I20260709 13:00:19.221800  1704 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:19.218817  3808 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:19.187429  3773 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:19.276305  3773 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.218596  1558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:00:19.278595  3504 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:19.279003  1558 raft_consensus.cc:606] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:19.278671  3773 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:19.281302  1705 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:19.281457  3652 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:19.282141  3652 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.208148  3797 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:19.255764  3801 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:19.235651  3790 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.288627  1526 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:19.236158  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:19.207480  3707 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.275379  1635 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:19.283529  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:19.284652  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 68222.1us
I20260709 13:00:19.287410  1554 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:19.289500  3772 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.757s	user 0.020s	sys 0.001s
I20260709 13:00:19.291154  2346 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:19.296177  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:19.301059  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:19.301198  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:19.302304  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:19.297227  1554 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:19.309070  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:19.309129  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:19.309321  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:19.304066  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:19.286595  3734 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.313573  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:19.314126  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:19.314715  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:19.315462  3799 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:19.316082  3799 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.319340  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:19.321321  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:19.321831  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:19.327627  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:19.320497  3659 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.325402  3528 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:19.291905  3727 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:00:19.335453  3528 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:19.306241  3772 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.312906  3605 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:19.312705  3785 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.346899  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:19.318776  3763 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.320782  3599 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:19.318485  3531 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:19.325737  3729 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.351949  3809 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:00:19.312244  3808 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:00:19.336609  3790 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:19.335590  3801 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.352483  3528 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.355093  1554 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.364861  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:19.352859  3599 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:19.324442  3734 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.355738  3311 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.353360  3799 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:19.355965  3605 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:19.361339  3707 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:19.373926  3778 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.376729  3599 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:19.388545  1704 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:19.396585  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:19.400276  3639 tablet_bootstrap.cc:492] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:19.402491  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1485) took 1294 ms. Trace:
I20260709 13:00:19.403532  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:19.404214  1559 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:19.406281  3576 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.105s	user 0.010s	sys 0.014s
I20260709 13:00:19.406757  3576 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.408016  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:19.402973  1706 rpcz_store.cc:276] 0709 13:00:18.108251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.108485 (+   234us) service_pool.cc:224] Handling call
0709 13:00:19.402269 (+1293784us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b5400223654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:19.402391 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1650344832}
I20260709 13:00:19.413501  2349 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:19.414724  3576 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:19.415659  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:19.418584  3771 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.935s	user 0.018s	sys 0.005s
I20260709 13:00:19.419155  3771 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.419716  3584 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:19.419819  1459 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 184) took 1994 ms. Trace:
I20260709 13:00:19.420482  1703 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:19.422200  3813 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:00:19.421952  3624 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:19.380741  3715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59270 (request call id 581) took 3721 ms. Trace:
I20260709 13:00:19.420480  1459 rpcz_store.cc:276] 0709 13:00:17.492475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.492712 (+   237us) service_pool.cc:224] Handling call
0709 13:00:19.409039 (+1916327us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:18.238748 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:18.360769 (+122021us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:18.369312 (+  8543us) write_op.cc:435] Acquired schema lock
0709 13:00:18.370547 (+  1235us) tablet.cc:662] Decoding operations
0709 13:00:18.466825 (+ 96278us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:18.469122 (+  2297us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:18.475510 (+  6388us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:18.481765 (+  6255us) tablet.cc:701] Row locks acquired
0709 13:00:18.481806 (+    41us) write_op.cc:260] PREPARE: finished
0709 13:00:18.482012 (+   206us) write_op.cc:270] Start()
0709 13:00:18.482185 (+   173us) write_op.cc:276] Timestamp: P: 1783602018481947 usec, L: 0
0709 13:00:18.482208 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:00:18.482977 (+   769us) log.cc:844] Serialized 536 byte log entry
0709 13:00:19.189252 (+706275us) op_driver.cc:464] REPLICATION: finished
0709 13:00:19.316038 (+126786us) write_op.cc:301] APPLY: starting
0709 13:00:19.316129 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:19.316244 (+   115us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:19.316357 (+   113us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:19.316628 (+   271us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:19.316801 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:19.316843 (+    42us) write_op.cc:312] APPLY: finished
0709 13:00:19.317041 (+   198us) log.cc:844] Serialized 23 byte log entry
0709 13:00:19.317263 (+   222us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:19.317347 (+    84us) write_op.cc:454] Released schema lock
0709 13:00:19.317385 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:19.317779 (+   394us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:19.317811 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":284889,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":126508,"apply.run_cpu_time_us":1715,"apply.run_wall_time_us":1716,"num_ops":1,"prepare.queue_time_us":382665,"prepare.run_cpu_time_us":4789,"prepare.run_wall_time_us":246738,"replication_time_us":706871,"thread_start_us":367512,"threads_started":2}]]}
W20260709 13:00:19.393759  3626 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.568s	user 0.012s	sys 0.000s
I20260709 13:00:19.410844  3639 tablet_bootstrap.cc:492] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 13:00:19.393594  3809 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:19.425004  3715 rpcz_store.cc:276] 0709 13:00:15.636025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.636334 (+   309us) service_pool.cc:224] Handling call
0709 13:00:15.956191 (+319857us) ts_tablet_manager.cc:934] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
0709 13:00:17.809968 (+1853777us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:00:18.142256 (+332288us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:18.153394 (+ 11138us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:00:19.226152 (+1072758us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:00:19.287995 (+ 61843us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:19.337118 (+ 49123us) ts_tablet_manager.cc:989] Replica registered.
0709 13:00:19.357066 (+ 19948us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":548,"tablet-copy.queue_time_us":226344,"thread_start_us":266400,"threads_started":2}
I20260709 13:00:19.457374  3715 tablet_copy_client.cc:806] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:19.458158  3715 tablet_copy_client.cc:670] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Starting download of 1 WAL segments...
W20260709 13:00:19.458542  3754 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.578s	user 0.014s	sys 0.000s
W20260709 13:00:19.460341  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 62378.1us
I20260709 13:00:19.462054  3669 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.465674  1601 leader_election.cc:400] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4.
I20260709 13:00:19.466140  1601 leader_election.cc:403] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:19.473632  3817 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:19.474433  3817 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.477159  3817 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:19.432093  3623 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:00:19.435137  3526 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.572161  3809 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.462769  3669 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.578701  1559 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:19.582355  3799 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:19.583184  3799 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.704526  3820 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.226s
I20260709 13:00:19.705543  3565 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4.
I20260709 13:00:19.705937  3820 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:19.706378  3820 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:19.704947  3822 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:00:19.713975  1675 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:19.459285  3754 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.721128  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:19.742172  1709 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:19.720333  3605 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.745520  3784 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:19.745860  3771 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:19.746488  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.708930  3639 ts_tablet_manager.cc:1403] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 2.159s	user 0.037s	sys 0.032s
W20260709 13:00:19.769861  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.618919  3669 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:19.779492  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:19.779903  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.393786  3778 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.438287  3626 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:19.783041  1636 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:19.780319  1559 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:19.778113  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:19.793404  3729 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:19.794277  3729 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.797156  3729 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:19.800294  3626 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:19.801240  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:19.802541  1710 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:19.804060  3799 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:19.814059  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 352887us
I20260709 13:00:19.808437  3819 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.831190  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:19.832265  1706 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:00:19.809790  3693 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.597s	user 0.005s	sys 0.006s
I20260709 13:00:19.840425  1628 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:19.843048  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:19.843528  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:19.843989  1709 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:19.844373  1635 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:19.834770  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:19.804890  3820 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.774024  3652 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.813264  3778 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:19.841601  3693 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:19.830036  3802 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.000s
I20260709 13:00:19.829272  3809 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:19.873389  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:19.881837  1629 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:19.885073  1555 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:19.889407  1706 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:19.892489  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:19.828686  3819 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:19.895789  3819 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:19.896759  3754 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:19.898301  3599 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:19.899704  3652 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:19.881155  3659 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:19.903756  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:19.907474  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.907732  1633 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:19.914225  1703 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:19.914716  1630 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:19.904462  3797 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.608s	user 0.012s	sys 0.000s
I20260709 13:00:19.932994  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:19.923765  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:19.938369  1630 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:19.937186  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:19.916565  3597 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.741s	user 0.011s	sys 0.002s
I20260709 13:00:19.950217  1635 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:20.049880  1629 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:20.050822  1633 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:20.048906  3797 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:20.037801  3778 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.037374  3639 raft_consensus.cc:359] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:19.901387  3659 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.042199  1634 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:20.042822  1553 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:19.943224  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:19.943562  3599 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.046802  3693 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.051107  1635 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:20.036759  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:20.052755  1709 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:20.052776  3504 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.053432  3623 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:20.068586  3802 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:20.103978  3639 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
W20260709 13:00:20.161521  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.164674  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:20.166457  3820 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:20.166553  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.167263  2348 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:20.167667  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.167838  2349 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:20.168596  2344 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:20.116627  3786 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:20.175894  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.174153  1527 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:20.181533  1709 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:20.181885  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:20.180724  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 399538us
W20260709 13:00:20.188557  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 325368us
I20260709 13:00:20.105269  3778 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.103384  3597 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:20.188988  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:20.200249  1557 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:20.175765  3786 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.180368  3727 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:20.127880  3797 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.132879  3504 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.205273  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:20.188586  3828 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:20.142771  3623 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.222265  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:20.149941  3802 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.201690  3752 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.201103  3390 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:20.195410  3796 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:20.238118  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 316986us
I20260709 13:00:20.201457  3783 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:00:20.225204  3828 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:20.230185  3707 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.845s	user 0.014s	sys 0.000s
I20260709 13:00:20.232070  3390 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.228335  3727 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.255710  3659 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.256081  3727 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.244686  3729 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:00:20.257234  3531 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:20.241346  3796 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.257962  3830 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:20.260424  3830 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.261385  3796 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.257822  3793 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:20.262228  3763 raft_consensus.cc:3055] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.262670  3763 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:20.262562  3793 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.226969  3808 consensus_queue.cc:1059] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:00:20.265686  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:20.242276  3775 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:20.246332  3828 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.272303  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:20.273028  1635 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:20.273676  1630 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:20.275941  1602 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:20.275996  3772 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:20.276531  3836 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:20.277115  3833 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:20.223657  3639 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:20.269743  3773 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:20.279678  3727 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:00:20.284008  3778 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.284175  3773 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.285130  3727 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.285547  3727 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:20.286494  3727 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.286729  3773 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.287775  3727 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.288087  3623 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:20.286803  3786 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.290208  3801 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:20.259191  3390 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.279830  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:20.299055  1635 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:20.299459  1635 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:20.301043  3825 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:20.302356  3597 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.303069  3793 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.263722  3763 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.304042  3501 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:20.304205  3763 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.304410  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:20.306492  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1501) took 1008 ms. Trace:
I20260709 13:00:20.306902  1704 rpcz_store.cc:276] 0709 13:00:19.297733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.297864 (+   131us) service_pool.cc:224] Handling call
0709 13:00:19.298181 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.410534 (+112353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.306221 (+895687us) spinlock_profiling.cc:243] Waited 90.1 ms on lock 0x7b5400247054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:20.306428 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207305984}
I20260709 13:00:20.309163  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.259500  3531 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.311719  3531 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.311937  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.298391  3707 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.313148  1602 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:20.314323  3752 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.317317  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:20.320240  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:20.322297  1705 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:20.323408  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:20.291383  3801 raft_consensus.cc:3055] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.322801  3799 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:20.277957  3836 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.292393  3775 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.323375  3808 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:20.319116  3797 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.323609  3826 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.324465  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:20.325395  3832 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:00:20.325587  3825 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:20.334785  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.334940  3825 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.335428  1705 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 1.
I20260709 13:00:20.337652  3825 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.338052  1708 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:20.327011  3707 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:20.327313  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2080) took 1163 ms. Trace:
W20260709 13:00:20.335556  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.340453  1636 rpcz_store.cc:276] 0709 13:00:19.163722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.163898 (+   176us) service_pool.cc:224] Handling call
0709 13:00:19.164248 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.789029 (+624781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.327059 (+538030us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b5400344254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:20.327232 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422489472}
I20260709 13:00:20.325722  3801 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:20.342675  3707 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:20.342463  3801 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.343349  3707 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.318493  1635 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.328890  1554 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.331985  1707 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:20.336403  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.325682  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.347679  3801 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.339895  3799 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:20.336406  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.331700  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:20.330118  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:20.330751  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:20.351341  1561 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:20.330483  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:20.329828  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:20.331372  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:20.352896  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:20.353863  1701 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.361851  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:20.363343  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:20.330372  3808 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.364614  1526 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:20.365705  3808 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:20.366029  3816 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:20.366514  3816 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:20.366580  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:20.366858  3825 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:20.365391  3390 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.348752  3799 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:20.323726  3790 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.948s	user 0.010s	sys 0.005s
I20260709 13:00:20.382964  3828 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.331172  3802 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.388002  3390 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.391440  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.392931  1703 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:20.329787  3775 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.393733  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.392944  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:20.384234  1554 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:20.397284  3605 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.383661  3826 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:20.397971  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 244) took 1240 ms. Trace:
I20260709 13:00:20.384986  3783 raft_consensus.cc:3055] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.402490  1554 rpcz_store.cc:276] 0709 13:00:19.158095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.187292 (+ 29197us) service_pool.cc:224] Handling call
0709 13:00:19.190978 (+  3686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.287295 (+ 96317us) spinlock_profiling.cc:243] Waited 5.58 ms on lock 0x7b54002371a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27651dc0 00007fda276dc1c0 00007fda276dd231 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c
0709 13:00:20.328385 (+1041090us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b58001cf1d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276360aa 00007fda276b8447 00007fda276d0fe7 00007fda276d27de 00007fda276dcb49 00007fda276dd416 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4
0709 13:00:20.396883 (+ 68498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.397909 (+  1026us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1118241408}
I20260709 13:00:20.403733  1557 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:20.384338  3838 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:20.367573  3825 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 13:00:20.401891  3800 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:00:20.385325  1455 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 5 to 6, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:20.367549  3816 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.411126  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41400 (request call id 14) took 1255 ms. Trace:
I20260709 13:00:20.404949  3605 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.412689  1558 rpcz_store.cc:276] 0709 13:00:19.153777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.186390 (+ 32613us) service_pool.cc:224] Handling call
0709 13:00:20.398438 (+1212048us) spinlock_profiling.cc:243] Waited 47.5 ms on lock 0x7b5400250420. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27657520 00007fda276d4652 00007fda2c6a260a 00007fda25c287d4 00007fda25c28762 00007fda25c28691 00007fda25c2860c 00007fda25c278c2 00007fda25a1548c 00007fda25a148c6
0709 13:00:20.409150 (+ 10712us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109298560}
I20260709 13:00:20.402236  3826 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:20.406313  3836 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:20.403420  3783 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:20.426406  3775 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:20.406716  3838 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.420842  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.406729  3501 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:00:20.431350  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.424552  3624 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:20.425630  3816 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.432283  3624 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:20.433355  3624 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.433753  3772 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:20.434463  3624 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.435209  3772 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:20.426656  3820 raft_consensus.cc:993] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: : Instructing follower a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:20.436367  3820 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.436880  3820 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:20.437098  3838 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.437363  3501 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:20.438014  3501 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.438797  3799 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.439443  3820 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.440230  3501 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.420470  3826 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.440789  3820 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.441464  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:00:20.437316  1556 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:20.444782  3504 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.440256  3799 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:20.447795  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.448506  2345 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:20.450299  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:20.436414  3772 raft_consensus.cc:1081] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Signalling peer a2ba7fc60f9641cdac25f1e016ef91d8 to start an election
I20260709 13:00:20.427537  3783 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.451289  3504 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:20.451679  3504 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.452154  3504 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:20.458328  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:20.458968  3615 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:20.459151  1554 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:20.459444  3615 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:20.461580  1707 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:20.407971  3790 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:00:20.467042  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 618) took 1141 ms. Trace:
I20260709 13:00:20.460654  3615 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.467547  2347 rpcz_store.cc:276] 0709 13:00:19.325758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.327736 (+  1978us) service_pool.cc:224] Handling call
0709 13:00:20.466164 (+1138428us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b5800003c98. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:00:20.466370 (+   206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:20.466563 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:20.466592 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:20.466665 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:20.466722 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.466796 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:20.466938 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6923648}
I20260709 13:00:20.476473  3786 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:20.482918  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:20.481045  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 184010us
I20260709 13:00:20.446933  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.488130  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.489115  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:20.487175  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:20.480055  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:20.488791  2350 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:20.497529  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:20.498503  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:20.499184  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.498503  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:20.500288  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:20.502422  3390 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:20.506873  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:20.512672  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:20.513883  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:20.516551  3390 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.518754  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:20.521399  1702 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.486084  3783 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.477321  3786 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.523419  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:20.476186  3501 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:20.497185  3772 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:20.565964  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.566123  3772 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:20.566834  3772 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.570405  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1492) took 1887 ms. Trace:
I20260709 13:00:20.571015  1710 rpcz_store.cc:276] 0709 13:00:18.682855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.697090 (+ 14235us) service_pool.cc:224] Handling call
0709 13:00:18.697430 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.802976 (+1105546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.570098 (+767122us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540016c454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:20.570318 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2541011200}
I20260709 13:00:20.574283  3797 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:20.574797  3797 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:20.575187  3786 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.575693  3797 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.577039  3797 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:20.531630  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.495803  3627 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:20.542012  3639 ts_tablet_manager.cc:1434] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.759s	user 0.021s	sys 0.016s
I20260709 13:00:20.517098  1704 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:46210
I20260709 13:00:20.542238  3390 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:20.541941  3504 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.542912  1559 raft_consensus.cc:606] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:20.543675  3774 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.495745  3836 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:20.526012  3782 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:00:20.522951  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:20.555517  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 745) took 2029 ms. Trace:
I20260709 13:00:20.552951  1702 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:20.578254  3817 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:20.523506  3833 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.558619  2350 raft_consensus.cc:1217] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEARNER]: Deduplicated request from leader. Original: 4.4->[4.5-4.5]   Dedup: 4.5->[]
I20260709 13:00:20.560539  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.560902  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:20.526595  3623 consensus_queue.cc:1059] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:00:20.608695  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 88194.1us
I20260709 13:00:20.522768  3615 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:20.621615  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.516886  3752 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:20.625833  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:20.627022  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:20.627588  1556 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:20.552899  3697 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.629848  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:20.630718  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:20.631356  1557 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.632442  3504 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.582089  3501 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.634774  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:20.581979  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:20.514623  3681 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:20.579378  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:20.586680  1704 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.641667  1704 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.594281  1559 raft_consensus.cc:614] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:20.554347  3599 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:20.556357  3828 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.610271  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:20.615696  2348 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.607261  3783 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:20.648681  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 69389.2us
I20260709 13:00:20.651806  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:20.586413  3639 ts_tablet_manager.cc:1467] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Tablet startup took 835ms
W20260709 13:00:20.655085  3827 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.532s	user 0.004s	sys 0.000s
I20260709 13:00:20.657678  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2071) took 1697 ms. Trace:
I20260709 13:00:20.660365  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:20.657768  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:20.652760  3639 ts_tablet_manager.cc:1469] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Trace:
0709 13:00:12.695939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.706117 (+ 10178us) service_pool.cc:224] Handling call
0709 13:00:13.026014 (+319897us) ts_tablet_manager.cc:934] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
0709 13:00:14.036737 (+1010723us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:00:14.764584 (+727847us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:14.770233 (+  5649us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:00:15.643997 (+873764us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:00:15.644493 (+   496us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:15.644864 (+   371us) ts_tablet_manager.cc:989] Replica registered.
0709 13:00:15.644925 (+    61us) inbound_call.cc:177] Queueing success response
0709 13:00:16.761899 (+1116974us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b4c00384d00. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda259a6a20 00007fda259b7a29 00007fda259b813f 00007fda2597c2ab 00007fda259a9f3c 00007fda259aa461 00007fda259aa768 00007fda2b73066b 00007fda2c66d15b 00007fda2c66cf98 00007fda2c65e322
0709 13:00:17.550117 (+788218us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:00:19.783049 (+2232932us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:00:19.859716 (+ 76667us) tablet_replica.cc:261] Starting instrumentation
0709 13:00:19.860467 (+   751us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":39638,"raft.queue_time_us":2242,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":92,"spinlock_wait_cycles":82431744,"tablet-copy.queue_time_us":269172,"tablet-download-pool.queue_time_us":5560,"tablet-download-pool.run_cpu_time_us":13330,"tablet-download-pool.run_wall_time_us":883288,"thread_start_us":465331,"threads_started":3}
W20260709 13:00:20.664515  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.658313  1631 rpcz_store.cc:276] 0709 13:00:18.960275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.960408 (+   133us) service_pool.cc:224] Handling call
0709 13:00:19.000001 (+ 39593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:19.022681 (+ 22680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:19.022724 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:19.023345 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 13:00:19.788781 (+765436us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b1800280c00. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:00:20.104678 (+315897us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:20.104738 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.104850 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:20.654404 (+549554us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:20.657461 (+  3057us) raft_consensus.cc:1685] finished
0709 13:00:20.657499 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:20.657619 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13598,"raft.queue_time_us":18042,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"spinlock_wait_cycles":29451136,"thread_start_us":314566,"threads_started":1,"wal-append.queue_time_us":1093144}
I20260709 13:00:20.665637  3825 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:20.673759  3825 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:20.676192  3825 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.666484  3715 tablet_copy_client.cc:538] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:20.657636  3788 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:20.672817  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 90954.7us
I20260709 13:00:20.672861  3697 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.634788  3501 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:20.687593  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.689467  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:20.691184  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:20.701519  1560 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:20.636479  3817 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:20.701575  1710 raft_consensus.cc:2376] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3. Re-sending same reply.
I20260709 13:00:20.720476  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.694005  2317 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:20.715468  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.718569  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:20.723547  1631 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:20.723822  3820 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:20.723974  3817 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.701807  3788 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.724354  3820 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.659859  3816 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:20.612934  3836 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:20.733191  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.608876  3774 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.726590  3820 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.743997  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:20.750167  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:20.754221  1631 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:20.702469  3786 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:20.608726  2346 rpcz_store.cc:276] 0709 13:00:18.526589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.532294 (+  5705us) service_pool.cc:224] Handling call
0709 13:00:18.572307 (+ 40013us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.309912 (+737605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.555178 (+1245266us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b540069e6d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:20.555436 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1509138304}
I20260709 13:00:20.734097  3501 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:20.669675  3783 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.726605  3788 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.703747  3729 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:00:20.778617  3501 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:20.752432  3816 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:20.762220  3786 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:20.767892  3831 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.797156  1560 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:00:20.578529  3659 consensus_queue.cc:1175] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:20.769747  3852 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:00:20.826828  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 163926us
I20260709 13:00:20.822266  1704 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:20.831976  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51978.1us
I20260709 13:00:20.834630  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:20.835826  3754 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:20.798004  3786 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:20.795055  3788 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.858225  1704 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.862948  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1509) took 1014 ms. Trace:
I20260709 13:00:20.863459  1706 rpcz_store.cc:276] 0709 13:00:19.848512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.848689 (+   177us) service_pool.cc:224] Handling call
0709 13:00:19.872809 (+ 24120us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.890081 (+ 17272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.862686 (+972605us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b54001e9454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:20.862881 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37480320}
I20260709 13:00:20.826550  3782 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:20.857255  3801 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.005s
I20260709 13:00:20.869755  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:20.873548  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:20.876008  2352 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.876070  3754 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:20.876838  3754 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:20.875399  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 144766us
I20260709 13:00:20.883041  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:20.856065  3852 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:20.890487  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.884248  2348 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:20.888350  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.893837  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:20.903672  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.904075  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.907119  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:20.907934  2349 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.865072  3788 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.887207  1719 tablet_copy_service.cc:342] P a2ba7fc60f9641cdac25f1e016ef91d8: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-61a6c336f56b4c248aac90993fb477cf received from {username='slave'} at 127.0.0.1:52646
I20260709 13:00:20.903487  3652 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:20.923540  1719 tablet_copy_service.cc:434] P a2ba7fc60f9641cdac25f1e016ef91d8: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-61a6c336f56b4c248aac90993fb477cf on tablet 61a6c336f56b4c248aac90993fb477cf with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:20.923815  3652 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:20.925061  3774 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:20.924916  3652 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.917593  3783 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:20.917829  3845 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:00:20.918630  3849 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:00:20.934624  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:20.922900  3836 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.916904  1557 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:20.885859  3782 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.909317  3825 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:20.903857  3801 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:20.929509  3707 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.578s	user 0.007s	sys 0.004s
I20260709 13:00:20.951514  1701 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 1.
I20260709 13:00:20.899464  3826 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.885320  3816 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.911451  3615 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:20.940104  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.884248  3501 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.961216  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:20.966105  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:20.967046  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:20.953063  3778 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.978440  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:20.979383  2352 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:20.981266  1676 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:20.956306  3833 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.980827  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:20.903041  3754 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.986685  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:20.989271  2351 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:20.984148  3856 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:20.992970  3856 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.993676  3772 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.986451  3820 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:21.000572  3715 tablet_bootstrap.cc:492] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap starting.
I20260709 13:00:20.987033  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:20.986775  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.008955  1631 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 1.
W20260709 13:00:20.991307  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.021832  3624 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:21.022598  3624 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:20.986603  3828 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.989079  3837 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:21.027895  3624 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:20.989231  3858 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:20.983089  3790 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:21.029379  3858 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.032045  3858 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:20.990868  3783 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:21.033331  3783 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:21.034582  3783 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.007865  3729 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:21.038828  3805 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:21.039216  3729 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:21.040463  3729 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.041719  3826 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.028195  3837 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:20.959408  3801 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.050139  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:20.960521  3707 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.048995  1557 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:21.045958  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:21.075078  3528 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.075809  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.064899  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.077023  3788 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.089859  1702 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:21.087306  1703 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:21.090045  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:20.940003  3852 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.075341  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.080756  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.095429  1627 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:20.957358  3825 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.084219  2349 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:20.974711  3615 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.088896  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.088649  3528 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:21.095655  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.097802  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:21.029239  3624 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:21.102176  1556 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
W20260709 13:00:21.087831  3838 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.621s	user 0.020s	sys 0.000s
I20260709 13:00:21.105979  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.112890  1703 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
W20260709 13:00:21.104628  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 58746.1us
I20260709 13:00:21.111982  1552 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:21.148442  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 193377us
I20260709 13:00:21.092109  3763 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:21.158442  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 172402us
I20260709 13:00:21.039474  3805 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:21.138687  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.112809  3807 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:21.121105  3565 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.119446  3752 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:21.121397  3831 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.119096  3659 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:21.159741  3800 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:21.200064  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:21.201107  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:21.202924  2349 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 13:00:21.205235  2347 raft_consensus.cc:2368] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4.
I20260709 13:00:21.122025  3838 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.160004  3817 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:00:21.180186  3807 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:21.217813  2346 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:00:21.218510  3615 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.219040  3528 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:21.222980  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:21.223860  2349 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
W20260709 13:00:21.213703  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 48698.1us
I20260709 13:00:21.164446  3788 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:00:21.179039  3782 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.217792  3838 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.249732  1708 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:21.197255  3856 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.205005  3800 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.265738  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.272176  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:21.272612  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 67764.9us
I20260709 13:00:21.203944  3854 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:21.278828  3754 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:21.272212  3800 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:21.282241  3799 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.835s	user 0.009s	sys 0.013s
I20260709 13:00:21.285887  3754 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:21.220446  3805 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:21.198921  3565 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.178647  3763 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.204320  3752 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:21.293064  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:21.312369  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.294042  1676 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:21.298045  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:21.298812  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:21.297272  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:21.308179  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.330502  1710 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 13:00:21.200711  3828 consensus_queue.cc:1059] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:00:21.332510  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:21.290051  3826 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:21.347357  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:21.291031  3858 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:21.350512  3858 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.211767  3831 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.355964  3831 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.292743  3605 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.342206  3805 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:21.327600  3838 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:00:21.369432  3858 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.327579  3854 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.362972  3659 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:21.510663  3763 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.429762  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:21.576717  3786 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.709s	user 0.006s	sys 0.005s
W20260709 13:00:21.411185  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.379662  3707 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.388808  3801 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.383791  3805 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:21.595865  3805 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:21.596473  3687 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.597184  3687 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.599992  3687 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:21.601202  3687 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.605660  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41400 (request call id 15) took 1164 ms. Trace:
I20260709 13:00:21.511518  3565 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:21.577214  3867 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
W20260709 13:00:21.513778  2345 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.879s	user 0.011s	sys 0.004s
I20260709 13:00:21.611768  1553 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:00:21.606204  1559 rpcz_store.cc:276] 0709 13:00:20.441153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.441294 (+   141us) service_pool.cc:224] Handling call
0709 13:00:21.605414 (+1164120us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400337cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d4853 00007fda2c6a260a 00007fda25c287d4 00007fda25c28762 00007fda25c28691 00007fda25c2860c 00007fda25c278c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:21.605592 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231767424}
I20260709 13:00:21.427953  3856 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:21.371768  3826 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:21.620683  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 275943us
I20260709 13:00:21.395812  3874 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:21.593392  3859 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:21.607146  3825 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.442337  3837 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.382473  3734 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:21.637406  1635 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:21.648978  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.634516  3870 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:21.602025  3687 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.655633  3870 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.656328  1559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:00:21.655304  3859 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.658185  3870 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.623030  3783 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.602226  3786 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.622380  2345 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
W20260709 13:00:21.676841  3797 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.099s	user 0.009s	sys 0.006s
I20260709 13:00:21.690208  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:21.634806  3697 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:21.677608  3833 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.681975  3659 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.683331  3826 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.683729  3854 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.685858  3870 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:21.685956  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:21.708264  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.708855  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.686813  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.686901  3858 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:21.686990  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.687340  3857 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:21.687803  3818 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.688194  1631 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:21.637321  3808 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:21.697594  3820 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.685786  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:21.685976  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.718353  1628 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:21.689325  1627 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
W20260709 13:00:21.694195  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 361767us
I20260709 13:00:21.698376  3825 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:21.698740  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:21.699314  1554 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:21.690954  3752 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.701710  3697 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.690405  3783 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.703392  1629 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:21.703526  3805 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:21.712324  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.714066  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.687814  3687 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:21.722754  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:21.734511  1635 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:21.725232  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:21.727101  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.730997  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:21.732810  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.735878  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 1884) took 1461 ms. Trace:
I20260709 13:00:21.737847  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.737059  1635 rpcz_store.cc:276] 0709 13:00:20.274741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.274849 (+   108us) service_pool.cc:224] Handling call
0709 13:00:20.275101 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:20.563197 (+288096us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b18002e7e80. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27715646 00007fda2764f58e 00007fda276d0f9e 00007fda276d27de 00007fda276dcb49 00007fda276dd416 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4
0709 13:00:21.633733 (+1070536us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b58000878d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276360aa 00007fda276b8447 00007fda276d0fe7 00007fda276d27de 00007fda276dcb49 00007fda276dd416 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4
0709 13:00:21.735219 (+101486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.735831 (+   612us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1607430272}
I20260709 13:00:21.741169  1709 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:21.741861  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:21.742607  1635 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:21.736076  1627 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:21.722218  3565 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.705190  3833 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.728505  3826 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:21.750976  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:21.751312  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:21.717622  3858 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.715396  3870 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.740551  3797 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:21.717316  3857 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.740767  3859 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.723866  3820 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.729065  3825 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.750599  3837 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:21.720196  3818 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.728961  3808 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.755175  3826 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.761395  3889 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:21.762416  3881 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:21.775595  3837 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.766781  3857 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.768178  3833 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.769642  1632 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:21.770256  3786 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.770831  3697 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.764626  1633 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:21.764163  3884 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:21.788033  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1547) took 1164 ms. Trace:
I20260709 13:00:21.788196  3884 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:21.788632  3884 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:21.788425  1554 rpcz_store.cc:276] 0709 13:00:20.623496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.635524 (+ 12028us) service_pool.cc:224] Handling call
0709 13:00:21.787872 (+1152348us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54003377d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:21.787974 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2386555136}
I20260709 13:00:21.775787  3881 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.778684  3805 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.760308  3877 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.780068  3837 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.782107  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1545) took 1159 ms. Trace:
I20260709 13:00:21.776556  3826 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:21.754381  3565 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:21.761577  3797 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.785732  1707 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:21.786171  3820 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.786425  3825 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.790503  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.792035  1636 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:21.794157  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:21.795732  2347 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:21.759955  3885 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:21.799050  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.799768  1553 rpcz_store.cc:276] 0709 13:00:20.622921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.629585 (+  6664us) service_pool.cc:224] Handling call
0709 13:00:21.781838 (+1152253us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b540038a4d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:21.781997 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1523273088}
I20260709 13:00:21.809150  1634 raft_consensus.cc:2410] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:00:21.797894  3800 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.000s
I20260709 13:00:21.798915  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:21.801873  3877 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:21.856654  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.854893  1703 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:22.062059  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.851881  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:21.857116  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.934468  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:22.065662  1555 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:22.066234  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:21.906420  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:21.848140  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:21.857548  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:22.067888  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:21.874320  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:22.064878  1701 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:22.069126  1561 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
W20260709 13:00:22.069837  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 221795us
I20260709 13:00:22.060791  1710 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:22.070676  1674 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:22.067247  1558 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:22.073388  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:21.848861  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:22.075528  1674 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:22.073396  3800 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:21.865945  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:22.076767  1560 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:22.068502  1553 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:22.078275  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:22.059576  3890 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:00:22.080930  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1560) took 1026 ms. Trace:
W20260709 13:00:22.081118  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.084100  1556 rpcz_store.cc:276] 0709 13:00:21.046920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.049834 (+  2914us) service_pool.cc:224] Handling call
0709 13:00:22.072757 (+1022923us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b5400208354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:22.072852 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97256704}
I20260709 13:00:22.084712  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:22.084566  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:22.085654  1706 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:22.087493  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:22.057435  3627 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:21.873400  3782 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.101492  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:21.859061  3805 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:21.852195  3799 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:22.006647  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:21.808495  3797 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:22.091845  1556 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:21.810952  3830 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.908592  3818 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.911391  3858 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.853657  3874 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:21.872040  3881 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:22.098482  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:22.128600  1706 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:22.093320  3627 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:22.109360  3825 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:21.918629  3808 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:21.910368  3870 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:22.004694  3801 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.203503  3627 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:22.205153  3627 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:22.140575  3825 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.906167  3526 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:22.066677  3783 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:22.072516  3539 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:21.920058  3652 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:22.218392  3526 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.218791  3652 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.220793  3652 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:22.003482  3828 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:22.221035  3526 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:21.980782  3772 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:21.918814  3754 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:22.250116  3772 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:22.250619  3772 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.252612  3772 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:22.208487  3816 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:22.254076  3816 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:21.793278  3889 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.255213  2349 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 13:00:21.841114  3687 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.293s
I20260709 13:00:22.114429  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:22.111580  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:22.095140  3697 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:22.077628  3707 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:00:22.101408  3833 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:21.935458  3820 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.080762  3734 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:00:21.978691  3785 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.129549  3874 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:22.137169  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:22.264474  3816 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:22.191263  1559 raft_consensus.cc:606] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:22.113355  3752 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:00:22.074781  3884 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.221454  3828 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.005439  3882 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:22.222263  1703 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
W20260709 13:00:22.225607  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.268764  3526 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:22.129951  3805 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.111862  3782 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:22.198004  3801 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:22.217933  3825 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.198617  3800 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:22.231739  3877 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:22.090349  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:22.248282  3826 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:22.108721  3896 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:00:22.265317  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 153823us
I20260709 13:00:22.287196  3833 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:22.305320  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:22.263199  3881 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:22.268600  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 262035us
I20260709 13:00:22.316108  1527 leader_election.cc:400] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4.
I20260709 13:00:22.316530  1527 leader_election.cc:403] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:22.274425  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:22.318034  1554 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 2: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:22.110288  3799 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:22.267035  3697 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:22.226894  3715 tablet_bootstrap.cc:700] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent opening tablet: real 0.115s	user 0.000s	sys 0.000s
W20260709 13:00:22.337541  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 228953us
I20260709 13:00:22.298055  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:22.304139  1707 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:22.309702  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:22.275154  3820 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.310678  3837 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:22.320737  3881 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.320128  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:22.296896  3526 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.288275  3785 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.342594  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:22.344784  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:22.347261  2347 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:22.309942  3896 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:22.317131  3888 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:22.352880  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:22.361932  1554 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 2: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:22.350301  1707 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:22.322721  3875 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:22.337518  3697 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.348735  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:22.350549  3877 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:22.356739  3801 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:22.360795  3900 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:22.341312  3707 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:22.358426  3752 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:22.358950  3884 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:22.358873  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:22.349797  3526 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:22.385833  3752 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:22.385957  3828 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:22.299376  3882 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.387490  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:22.387866  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:22.388298  1558 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:22.340142  3901 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.360368  3833 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:22.371366  3889 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:22.372932  3826 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:22.377785  1706 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:22.347755  3837 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.374068  3877 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.361030  3896 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:22.357597  3874 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:22.384634  3900 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:22.389639  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.391265  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:00:22.392205  3891 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.586s	user 0.004s	sys 0.005s
I20260709 13:00:22.364367  3888 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:22.386336  3752 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:22.357131  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:22.388998  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:23.705073  3828 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.705837  3828 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:23.708808  3828 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:22.392388  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:23.710676  3911 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:22.444846  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:23.867972  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 1.42304e+06us
W20260709 13:00:22.442713  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 79542us
I20260709 13:00:22.374081  2346 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:00:22.926806  3526 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.003s
I20260709 13:00:22.424578  3597 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:23.930853  3597 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:23.931524  3597 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:22.395782  3681 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:23.934762  3681 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:23.935353  3681 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:22.937307  3697 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.003s
I20260709 13:00:23.938594  3697 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:23.066913  3707 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.675s	user 0.008s	sys 0.004s
I20260709 13:00:23.943521  3707 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4.
I20260709 13:00:23.079911  3715 tablet_bootstrap.cc:492] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:00:22.451998  3734 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.966871  3734 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:22.951730  3752 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.002s
I20260709 13:00:23.970786  3752 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:22.441938  3797 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:22.469349  3805 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:22.455499  3782 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:23.026065  3816 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.732s	user 0.009s	sys 0.003s
I20260709 13:00:23.978067  1556 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:23.977988  3816 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:23.980458  3816 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:23.980609  3797 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:22.454649  3785 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:22.395047  3799 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:23.988190  3799 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:22.459614  3820 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:23.460067  3807 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:23.996501  3807 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:23.996799  3715 tablet_bootstrap.cc:492] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Bootstrap complete.
I20260709 13:00:23.997727  3715 ts_tablet_manager.cc:1403] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent bootstrapping tablet: real 2.997s	user 0.044s	sys 0.097s
W20260709 13:00:23.028169  3825 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.001s
I20260709 13:00:22.418337  3837 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:22.464143  3858 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.361s
I20260709 13:00:22.394971  3870 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:22.439635  3874 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:22.420229  3877 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.891582  3881 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:22.458369  3882 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:22.468218  3888 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:22.947994  3896 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.000s
I20260709 13:00:24.014212  3896 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.014739  2346 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
W20260709 13:00:24.015587  2346 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 775) took 1696 ms (client timeout 1541 ms). Trace:
W20260709 13:00:24.016036  2346 rpcz_store.cc:269] 0709 13:00:22.319190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.323094 (+  3904us) service_pool.cc:224] Handling call
0709 13:00:24.015498 (+1692404us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.452279  3899 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:00:24.018303  3899 consensus_queue.cc:1243] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:24.018858  3899 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:24.019186  3899 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:24.020131  3899 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:22.461309  3900 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:22.395357  3901 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:23.529649  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41400 (request call id 16) took 1873 ms. Trace:
W20260709 13:00:23.536227  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:24.023836  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.023509  1559 rpcz_store.cc:276] 0709 13:00:21.656023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.656194 (+   171us) service_pool.cc:224] Handling call
0709 13:00:23.529348 (+1873154us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b54000bf254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d4853 00007fda2c6a260a 00007fda25c287d4 00007fda25c28762 00007fda25c28691 00007fda25c2860c 00007fda25c278c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:23.529545 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1228878080}
I20260709 13:00:24.025086  3874 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.025127  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.026015  1559 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:24.025715  3874 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.027304  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:24.027568  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:23.544618  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 1.21939e+06us
I20260709 13:00:23.549234  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:23.552035  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2010) took 1234 ms. Trace:
W20260709 13:00:23.557252  1676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.531s (SENT) [suppressed 3 similar messages]
I20260709 13:00:23.564647  3539 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:23.593765  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:23.607785  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:23.624490  1707 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.259s	user 0.009s	sys 0.001s
W20260709 13:00:24.037302  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 480063us
I20260709 13:00:24.037696  1707 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
W20260709 13:00:24.038254  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2021) took 1699 ms (client timeout 1541 ms). Trace:
I20260709 13:00:23.632359  3599 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:23.635135  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2015) took 2001 ms (client timeout 1842 ms). Trace:
I20260709 13:00:23.639456  3754 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:23.656631  3873 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:23.657624  3857 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:23.666640  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.223s	user 0.015s	sys 0.011s
I20260709 13:00:23.680064  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1562) took 2632 ms. Trace:
I20260709 13:00:23.687041  3875 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.689880  3801 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.691017  3826 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:23.697422  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.060091  1701 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
W20260709 13:00:24.060633  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1541) took 1724 ms (client timeout 2110 ms). Trace:
I20260709 13:00:24.021414  3899 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:23.703558  1525 connection.cc:256] RPC call timeout handler was delayed by -1.319451006999941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
W20260709 13:00:24.067188  1525 connection.cc:256] RPC call timeout handler was delayed by -1.2604498189999731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
I20260709 13:00:24.068145  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:23.704152  3772 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:23.709416  3828 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:23.711426  1558 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:24.073644  1558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2004) took 1780 ms (client timeout 2032 ms). Trace:
W20260709 13:00:23.718168  2349 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 772) took 1924 ms (client timeout 1842 ms). Trace:
I20260709 13:00:23.711431  3911 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:23.731757  3854 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:23.736114  3898 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.739612  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:23.745772  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:23.750262  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:23.752136  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1532) took 1690 ms (client timeout 1810 ms). Trace:
I20260709 13:00:23.705858  3909 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:23.734645  3907 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.764931  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:23.767976  3852 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:23.807046  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.836658  3652 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:23.837003  3913 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:23.839391  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2009) took 2772 ms. Trace:
I20260709 13:00:23.993288  3681 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:23.847213  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.863685  3856 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:23.870343  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:23.873708  3774 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:23.876708  3808 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:23.879562  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.881279  3783 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:23.884423  3912 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:24.182816  3912 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.182871  3808 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:24.185348  3912 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:23.886381  1455 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 271) took 8434 ms. Trace:
I20260709 13:00:24.185727  3898 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.187783  3715 raft_consensus.cc:359] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.188588  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.188876  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.186036  1455 rpcz_store.cc:276] 0709 13:00:15.452437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.452701 (+   264us) service_pool.cc:224] Handling call
0709 13:00:23.886284 (+8433583us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:20.727946 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:20.855985 (+128039us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:20.859226 (+  3241us) write_op.cc:435] Acquired schema lock
0709 13:00:20.863050 (+  3824us) tablet.cc:662] Decoding operations
0709 13:00:20.903264 (+ 40214us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:20.909725 (+  6461us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:20.911354 (+  1629us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:20.942008 (+ 30654us) tablet.cc:701] Row locks acquired
0709 13:00:20.942044 (+    36us) write_op.cc:260] PREPARE: finished
0709 13:00:20.948642 (+  6598us) write_op.cc:270] Start()
0709 13:00:20.948830 (+   188us) write_op.cc:276] Timestamp: P: 1783602020948560 usec, L: 0
0709 13:00:20.948865 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:00:20.949910 (+  1045us) log.cc:844] Serialized 541 byte log entry
0709 13:00:21.756325 (+806415us) op_driver.cc:464] REPLICATION: finished
0709 13:00:22.075697 (+319372us) write_op.cc:301] APPLY: starting
0709 13:00:22.075779 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:22.075874 (+    95us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:22.075992 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:22.076234 (+   242us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:22.076422 (+   188us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:22.076495 (+    73us) write_op.cc:312] APPLY: finished
0709 13:00:22.076744 (+   249us) log.cc:844] Serialized 23 byte log entry
0709 13:00:22.089756 (+ 13012us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:22.103413 (+ 13657us) write_op.cc:454] Released schema lock
0709 13:00:22.106515 (+  3102us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:22.190619 (+ 84104us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:22.196308 (+  5689us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1625,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":314500,"apply.run_cpu_time_us":2123,"apply.run_wall_time_us":54463,"num_ops":1,"prepare.queue_time_us":253179,"prepare.run_cpu_time_us":32224,"prepare.run_wall_time_us":366400,"raft.queue_time_us":392547,"raft.run_cpu_time_us":38459,"raft.run_wall_time_us":200193,"replication_time_us":798439,"thread_start_us":694845,"threads_started":4,"wal-append.queue_time_us":244419,"wal-append.run_cpu_time_us":18408,"wal-append.run_wall_time_us":1872699}]]}
I20260709 13:00:24.065717  3875 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.189808  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:24.189888  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:23.889284  1458 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 2 to 3, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 3 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.190480  1628 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.190943  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:23.890172  1634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2102) took 2196 ms (client timeout 2766 ms). Trace:
I20260709 13:00:24.191756  1636 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:24.191870  1634 rpcz_store.cc:269] 0709 13:00:21.693631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.713829 (+ 20198us) service_pool.cc:224] Handling call
0709 13:00:23.889863 (+2176034us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b540017b954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:23.890080 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172136832}
W20260709 13:00:23.891956  1674 connection.cc:256] RPC call timeout handler was delayed by -1.9770899200000258s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 13:00:23.899613  2345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 365) took 3412 ms (client timeout 1531 ms). Trace:
W20260709 13:00:24.192869  1674 connection.cc:256] RPC call timeout handler was delayed by -1.9532980119999479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
W20260709 13:00:24.192948  2345 rpcz_store.cc:269] 0709 13:00:20.487356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.492755 (+  5399us) service_pool.cc:224] Handling call
0709 13:00:23.899259 (+3406504us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b54006bd354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:23.899465 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54072064}
I20260709 13:00:23.886411  3915 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:23.911381  3917 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:23.912039  3790 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:23.984989  3526 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:23.995464  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1578) took 1882 ms. Trace:
I20260709 13:00:24.002708  3825 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.000864  3697 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:23.998363  3800 consensus_queue.cc:1175] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
I20260709 13:00:24.002664  3752 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.008420  3870 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:24.013290  3888 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:23.921002  3565 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.017498  3896 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:24.028673  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:24.029628  3881 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.034482  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:24.032260  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.030349  2347 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:24.030010  1555 rpcz_store.cc:276] 0709 13:00:22.317931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.343844 (+ 25913us) service_pool.cc:224] Handling call
0709 13:00:23.551948 (+1208104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.030078  3778 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:24.039228  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2022) took 1693 ms (client timeout 1810 ms). Trace:
W20260709 13:00:24.038576  1707 rpcz_store.cc:269] 0709 13:00:22.338642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.339190 (+   548us) service_pool.cc:224] Handling call
0709 13:00:24.038172 (+1698982us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.072386  3782 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:24.046717  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1542) took 1709 ms (client timeout 1671 ms). Trace:
I20260709 13:00:24.047405  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:24.250931  1710 rpcz_store.cc:269] 0709 13:00:22.345846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.607340 (+1261494us) service_pool.cc:224] Handling call
0709 13:00:24.039143 (+431803us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.256333  2348 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
W20260709 13:00:24.040185  1709 rpcz_store.cc:269] 0709 13:00:21.633989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.634118 (+   129us) service_pool.cc:224] Handling call
0709 13:00:23.634872 (+2000754us) spinlock_profiling.cc:243] Waited 88.1 ms on lock 0x7b5400077954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:23.635045 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":202733312}
I20260709 13:00:24.048921  1557 rpcz_store.cc:276] 0709 13:00:21.047677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.048374 (+   697us) service_pool.cc:224] Handling call
0709 13:00:21.048729 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:21.083375 (+ 34646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.679655 (+2596280us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b5400484ca0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27651dc0 00007fda276dda02 00007fda276d9dd6 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:00:23.679959 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43017472}
W20260709 13:00:24.256074  1705 rpcz_store.cc:269] 0709 13:00:22.337039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.593376 (+1256337us) service_pool.cc:224] Handling call
0709 13:00:24.046627 (+453251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.070011  3900 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:24.053808  3734 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.047323  3797 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.046716  3599 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:24.057638  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.614s	user 0.015s	sys 0.013s
I20260709 13:00:23.701093  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1535) took 1638 ms. Trace:
I20260709 13:00:24.063575  3874 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.064471  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:24.065083  3830 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:24.039817  3539 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:24.061020  1701 rpcz_store.cc:269] 0709 13:00:22.336710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.697034 (+1360324us) service_pool.cc:224] Handling call
0709 13:00:24.060565 (+363531us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.071389  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:24.286008  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.286777  1555 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:24.286859  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:24.286934  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 215675us
I20260709 13:00:24.025189  3901 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:24.192452  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.191425  1458 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 6 to 7, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:00:24.294452  1458 catalog_manager.cc:5697] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 3, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 3 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.245000  3808 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:24.196660  3907 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.197224  3805 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:24.065935  3857 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.197834  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.198650   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:24.056689  3820 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:24.148895  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:24.198805  1703 rpcz_store.cc:269] 0709 13:00:22.061314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.061792 (+   478us) service_pool.cc:224] Handling call
0709 13:00:23.751936 (+1690144us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54001e9954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:23.752069 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366953856}
I20260709 13:00:24.199671  2353 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:24.182528  1704 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:24.195403  1708 rpcz_store.cc:276] 0709 13:00:21.067405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.067532 (+   127us) service_pool.cc:224] Handling call
0709 13:00:21.067853 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:21.270277 (+202424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.839101 (+2568824us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400246b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:23.839321 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417604352}
W20260709 13:00:24.199703  2349 rpcz_store.cc:269] 0709 13:00:21.793768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.793877 (+   109us) service_pool.cc:224] Handling call
0709 13:00:23.717859 (+1923982us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b540054a454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:23.718030 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448232832}
I20260709 13:00:24.198038  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:24.074165  1558 rpcz_store.cc:269] 0709 13:00:22.292834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.392123 (+ 99289us) service_pool.cc:224] Handling call
0709 13:00:24.073550 (+1681427us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.203586  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:24.204741  1556 rpcz_store.cc:276] 0709 13:00:22.113602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.114130 (+   528us) service_pool.cc:224] Handling call
0709 13:00:23.995087 (+1880957us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54002526d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:23.995388 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3941336448}
W20260709 13:00:24.143129  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:24.207237  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1543) took 1869 ms (client timeout 2127 ms). Trace:
W20260709 13:00:24.161275  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 126608us
I20260709 13:00:24.261471  3896 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.240500  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:24.305948  1702 rpcz_store.cc:269] 0709 13:00:22.337832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.745377 (+1407545us) service_pool.cc:224] Handling call
0709 13:00:24.207139 (+461762us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.243142  1635 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:24.244017  3790 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.189399  3715 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Initialized, Role: LEARNER
I20260709 13:00:24.246556  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:24.182554  3783 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.075366  3816 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.039731  1554 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:24.252728  2350 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:24.254020  2344 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:24.229247  3828 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.187988  3898 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.255119  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:24.312518  1674 connection.cc:256] RPC call timeout handler was delayed by -1.2739183559999674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 13:00:24.323982  1707 raft_consensus.cc:2376] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3. Re-sending same reply.
W20260709 13:00:24.051215  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:24.325460  2318 connection.cc:256] RPC call timeout handler was delayed by -2.1369090239999764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 13:00:24.325877  2318 connection.cc:256] RPC call timeout handler was delayed by -2.0208090239999592s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 13:00:24.326129  2318 connection.cc:256] RPC call timeout handler was delayed by -1.8273090239999874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
I20260709 13:00:24.258144  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.258140  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.326746  1705 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 13:00:24.326830  1557 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 1.
I20260709 13:00:24.072969  3801 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.180725  3597 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.262346  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:24.263149  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 1.8197e+06us
I20260709 13:00:24.330219  1710 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:24.263165  1706 rpcz_store.cc:276] 0709 13:00:22.062701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.191738 (+129037us) service_pool.cc:224] Handling call
0709 13:00:22.192048 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.420879 (+228831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.700731 (+1279852us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b540016c454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:23.701010 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426152832}
I20260709 13:00:24.244976  3870 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.072837  3826 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.283891  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:24.339038  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.339874  1709 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.066848  3877 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.250494  3774 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:24.194979  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.189262  1627 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:24.254529  3778 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.203474  3854 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.264460  3782 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.306411  1632 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:24.201977  3852 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.161870  3772 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.210282  3917 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.303676  2353 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:24.196074  3652 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:24.307329  3805 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.210125  3915 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.261665  3912 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.307962  1636 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:24.312270  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.311966  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.070462  3873 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.314393  3896 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.205961  3856 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.318377  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.290222  3707 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.319449  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.255323  3881 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.310101   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:24.299218  3752 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:24.324093  1674 connection.cc:256] RPC call timeout handler was delayed by -1.1531159289999664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
I20260709 13:00:24.048908  3785 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:24.327282  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.303141  3697 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.329598  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.331180  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:24.264828  3797 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.312403  3565 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.264977  3900 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.342901  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.311702  3871 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.571882  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:24.572464  3871 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.572881  3921 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:24.574990  3871 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.350888  1553 raft_consensus.cc:2410] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:24.351536  1633 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:24.352438  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.353981  3539 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:24.579102  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.579483  3539 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.580328  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.352943  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.581368  1557 raft_consensus.cc:2368] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:24.391811  3526 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.384495  3597 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.547674  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.582636  3921 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:24.342537  3825 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.206761  3913 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.201794  3909 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.312388  1628 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:24.316498  3837 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.378388  3911 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.395747  3778 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:24.497547  2374 debug-util.cc:398] Leaking SignalData structure 0x7b0800185c60 after lost signal to thread 434
I20260709 13:00:24.390929  3790 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.562731  3917 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.563395  3912 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.563825  3900 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:24.564118  3852 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:24.358628  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:24.564688  2353 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.659250  2353 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:24.659235  1629 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:24.660058  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 301507us
I20260709 13:00:24.340976  3898 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.355295  1561 raft_consensus.cc:2410] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:24.566286  1705 raft_consensus.cc:2368] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:24.565659  3752 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:24.566412  3785 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:24.502547  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:24.505659  3857 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.567236  3652 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.650308  3825 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:24.567713  1707 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:00:24.391491  3681 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.376423  3799 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:24.352632  3774 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.678093  3799 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.679673  3774 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.680702  3752 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.681382  3826 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.681604  3790 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:24.682339  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.682238  3790 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.684406  3790 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.686618  3909 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.687417  3785 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.688884  3652 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.689635  3913 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.691049  3837 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.569051  3797 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.692071  3599 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:24.695161  3899 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.002s
I20260709 13:00:24.697876  3805 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:24.698683  3912 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.698873  3881 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.699222  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.363799  3715 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:24.569247  1704 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:24.570070  3772 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.570174  3856 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:24.569931  3697 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.570396  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.504634  3854 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:24.561077  1634 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:24.571300  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.546077  1709 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:24.582396  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.191501  3875 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:24.308048  3901 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:00:24.583086  1627 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
W20260709 13:00:24.400374  2318 connection.cc:256] RPC call timeout handler was delayed by -1.7343623250000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
I20260709 13:00:24.354635  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.585340  1635 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:24.585688  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2008) took 2284 ms. Trace:
I20260709 13:00:24.315699  3820 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.346637  1555 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 2: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:24.374202  1560 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:24.383559  3877 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:24.630566  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.611060  1558 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:24.344977  3874 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:24.636595  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 287618us
I20260709 13:00:24.638407  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:24.858875  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.584438  1556 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 13:00:24.861804  1557 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:24.861178  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.862834  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2016) took 1137 ms. Trace:
I20260709 13:00:24.863088  1710 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:24.863224  1556 rpcz_store.cc:276] 0709 13:00:23.725350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.312036 (+586686us) service_pool.cc:224] Handling call
0709 13:00:24.862762 (+550726us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:24.728406  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.863998  1553 rpcz_store.cc:276] 0709 13:00:22.300866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.764604 (+1463738us) service_pool.cc:224] Handling call
0709 13:00:24.585451 (+820847us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54002ff654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:24.585618 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358462848}
I20260709 13:00:24.864533  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.865168  1556 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:24.796695  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:24.647513  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 75698.5us
I20260709 13:00:24.651772  1701 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 1.
I20260709 13:00:24.560501  3923 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.565704  3565 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:24.873979  3923 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:24.668557  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 337527us
I20260709 13:00:24.673466  1552 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:24.690946  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.699911  1707 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:24.670759  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:24.568585  3922 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:24.878834  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.880692  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1540) took 2783 ms. Trace:
I20260709 13:00:24.881814  1709 rpcz_store.cc:276] 0709 13:00:22.097508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.879630 (+2782122us) service_pool.cc:224] Handling call
0709 13:00:24.880003 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:24.880260 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:24.880298 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:24.880356 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:24.880416 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:24.880491 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:24.880623 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:24.862226  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.863348  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:24.636783  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.883060  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:24.883474  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 212776us
I20260709 13:00:24.883852  1557 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 2: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:24.884019  1556 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:24.865422  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.640911  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:24.666577  3917 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:24.866330  2318 connection.cc:256] RPC call timeout handler was delayed by -1.4953189049999764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
W20260709 13:00:24.782419  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:24.891403  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.866768  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:24.866994  1561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2003) took 2574 ms (client timeout 2766 ms). Trace:
I20260709 13:00:24.841355  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:24.892095  1561 rpcz_store.cc:269] 0709 13:00:22.292431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.739316 (+1446885us) service_pool.cc:224] Handling call
0709 13:00:24.860404 (+1121088us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400271fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:24.866942 (+  6538us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370099968}
I20260709 13:00:24.892632  1629 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:24.870645  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1545) took 2532 ms. Trace:
I20260709 13:00:24.870939  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:24.648334  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 59984.4us
I20260709 13:00:24.893563  1708 rpcz_store.cc:276] 0709 13:00:22.338383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.796289 (+2457906us) service_pool.cc:224] Handling call
0709 13:00:24.870581 (+ 74292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.894333  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:24.894776  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:24.766763  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:24.896234  1704 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
W20260709 13:00:24.722736  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 1996) took 2454 ms (client timeout 2013 ms). Trace:
I20260709 13:00:24.872370  1702 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:24.872525  1628 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:24.765785  3801 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.897747  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:24.897778  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:24.898517  2346 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 13:00:24.898734  1628 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:24.901140  3923 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.901566  3912 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:24.902204  3912 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:24.717255  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.702811  1630 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:24.903079  3912 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.903591  3805 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:24.903712  3922 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:24.904275  1630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2093) took 3219 ms (client timeout 1992 ms). Trace:
I20260709 13:00:24.867502  1706 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
W20260709 13:00:24.905041  1630 rpcz_store.cc:269] 0709 13:00:21.684667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.689989 (+  5322us) service_pool.cc:224] Handling call
0709 13:00:24.904068 (+3214079us) spinlock_profiling.cc:243] Waited 2.8 s on lock 0x7b5400343d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:24.904193 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6441320320}
I20260709 13:00:24.905889  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.906384  2350 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:24.906786  1561 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:24.868330  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:24.790194  2374 debug-util.cc:398] Leaking SignalData structure 0x7b08001b8ba0 after lost signal to thread 1493
I20260709 13:00:24.875408  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.884231  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:24.668663  3828 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.880589  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1536) took 2812 ms. Trace:
I20260709 13:00:24.968840  1701 rpcz_store.cc:276] 0709 13:00:22.068164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.879486 (+2811322us) service_pool.cc:224] Handling call
0709 13:00:24.879875 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:24.880143 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:24.880181 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:24.880244 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:24.880304 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:24.880377 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:24.880512 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.886452  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2017) took 1159 ms. Trace:
W20260709 13:00:24.970381  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.971120  1631 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:24.881156  1705 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:24.887336  1703 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.904498  3805 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:24.875378  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:24.766955  3915 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.908160  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.908368  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:24.869350  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:24.983135  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 113861us
W20260709 13:00:24.896984  1554 rpcz_store.cc:269] 0709 13:00:22.267888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.366271 (+ 98383us) service_pool.cc:224] Handling call
0709 13:00:24.722431 (+2356160us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400312754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:24.722602 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553166080}
I20260709 13:00:24.910392  1702 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:00:24.983398  3816 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.001s
I20260709 13:00:24.984750  3927 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:00:24.981014  3805 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.965372  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:24.999754  3896 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:25.001494  3783 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:25.008451  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.890347  3917 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.985596  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2028) took 1145 ms. Trace:
I20260709 13:00:24.992952  3921 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:24.875983  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.992385  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.010913  3921 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.011149  1634 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:25.011332  3921 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:25.011397  2346 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 13:00:25.012427  3922 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.912103  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.007395  3697 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.013315  1561 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:24.991981  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.014146  1629 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:25.014322  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.014953  1561 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:25.015726  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.016248  1629 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:24.859860  3782 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:24.912200  1628 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:24.702852  3825 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:25.018262  1629 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 13:00:25.018466  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 134322us
W20260709 13:00:24.912212  1552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2001) took 2622 ms (client timeout 2048 ms). Trace:
W20260709 13:00:25.019236  1552 rpcz_store.cc:269] 0709 13:00:22.290036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.548915 (+1258879us) service_pool.cc:224] Handling call
0709 13:00:24.912019 (+1363104us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54002cbd54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:24.912135 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":689401728}
I20260709 13:00:25.019969  1631 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.019860  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.020442  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.020747  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:25.021275  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:25.021799  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.022545  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.022922  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.023154  3926 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:24.913291  3899 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:25.023667  3926 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:25.023988  3801 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.024255  3926 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:24.971201  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.023559  1552 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
W20260709 13:00:25.025429  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.889143  1707 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:25.027344  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2026) took 1188 ms. Trace:
I20260709 13:00:25.027693  1707 rpcz_store.cc:276] 0709 13:00:23.838834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.882837 (+1044003us) service_pool.cc:224] Handling call
0709 13:00:24.889007 (+  6170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.027087 (+138080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.027284 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.921136  1559 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:24.970345  1560 rpcz_store.cc:276] 0709 13:00:23.727094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.749991 (+ 22897us) service_pool.cc:224] Handling call
0709 13:00:24.886312 (+1136321us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400330ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:24.886401 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":408791808}
I20260709 13:00:25.009982  1702 rpcz_store.cc:276] 0709 13:00:23.840087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.909754 (+1069667us) service_pool.cc:224] Handling call
0709 13:00:24.910068 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:24.985273 (+ 75205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:24.985534 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.906399  1633 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:25.022578  1630 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:24.907351  1558 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:24.771332  3873 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:25.043503  1634 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:24.713972  3734 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:25.047072  3734 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:25.048192  3734 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:25.049203  3734 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.050079  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.050963  3899 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:25.051394  3899 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:25.051654  3801 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:24.976091  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.052564  1635 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:24.979084  3565 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.053112  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2012) took 1329 ms. Trace:
I20260709 13:00:24.980669  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.053772  1555 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:25.053521  1559 rpcz_store.cc:276] 0709 13:00:23.724204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.026739 (+302535us) service_pool.cc:224] Handling call
0709 13:00:25.052972 (+1026233us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b54003304d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:25.053057 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1537667328}
I20260709 13:00:24.668267  3898 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:24.792488  3799 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:25.055433  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1582) took 2865 ms. Trace:
I20260709 13:00:24.766134  3772 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:24.878249  3874 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:25.055816  3799 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:25.064375  3772 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:25.064730  3799 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:24.892271  2318 connection.cc:256] RPC call timeout handler was delayed by -1.4026903959999686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
I20260709 13:00:25.066406  3825 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.067499  3772 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.067636  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:25.067524  3799 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.010156  3917 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:25.070662  3898 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:25.071434  3778 raft_consensus.cc:2764] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:00:24.741897  3715 ts_tablet_manager.cc:1434] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent starting tablet: real 0.723s	user 0.011s	sys 0.017s
I20260709 13:00:24.838886  3870 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:25.019608  1629 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:24.991968  3816 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:24.967809  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:24.974064  1710 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:25.079106  1636 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 1.
I20260709 13:00:25.021390  1631 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:25.009212  1554 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:25.027931  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.080576  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2013) took 1356 ms. Trace:
W20260709 13:00:25.080933  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 57540.5us
I20260709 13:00:25.082055  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:24.985827  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:25.007330  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 96626.2us
I20260709 13:00:25.083870  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:25.040078  1556 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:25.085858  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 270) took 1040 ms. Trace:
I20260709 13:00:25.080977  1554 rpcz_store.cc:276] 0709 13:00:23.724498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.984505 (+1260007us) service_pool.cc:224] Handling call
0709 13:00:25.080496 (+ 95991us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.086364  1556 rpcz_store.cc:276] 0709 13:00:24.045852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.882710 (+836858us) service_pool.cc:224] Handling call
0709 13:00:25.085768 (+203058us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:25.075476  3715 ts_tablet_manager.cc:1467] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Tablet startup took 1077ms
I20260709 13:00:25.087406  3898 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:25.087790  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.088451  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:25.089084  1554 raft_consensus.cc:2376] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4. Re-sending same reply.
I20260709 13:00:25.089968  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:24.772051  3752 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.024899  3901 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:25.045023  1633 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:25.045897  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.051764  3899 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.055433  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1581) took 2921 ms. Trace:
I20260709 13:00:24.914742  1602 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:25.055830  1559 rpcz_store.cc:276] 0709 13:00:22.189809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.054593 (+2864784us) service_pool.cc:224] Handling call
0709 13:00:25.054937 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.055112 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.055165 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.055198 (+    33us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:25.055230 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.055281 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.055373 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56192}
I20260709 13:00:25.058213  2353 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
W20260709 13:00:24.866044  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.888175  3907 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:24.971989  1553 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:25.106206  2353 pending_rounds.cc:85] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 7
W20260709 13:00:25.106451  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 191808us
I20260709 13:00:25.106683  2353 pending_rounds.cc:107] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:00:25.107474  2353 raft_consensus.cc:2889] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:25.107887  1602 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:25.106350  1555 rpcz_store.cc:276] 0709 13:00:22.133709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.054538 (+2920829us) service_pool.cc:224] Handling call
0709 13:00:25.054937 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.055113 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.055149 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.055199 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:25.055229 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.055282 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.055373 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40064}
I20260709 13:00:25.108644  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 636) took 1082 ms. Trace:
W20260709 13:00:25.063218  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:25.110095  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:25.110565  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.919937  3877 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:25.074059  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:25.074640  1560 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:25.016402  1561 raft_consensus.cc:2368] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5.
I20260709 13:00:25.085065  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:25.127046  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:25.128001  1674 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:25.127871  1554 raft_consensus.cc:2376] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2. Re-sending same reply.
I20260709 13:00:25.091555  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.130517  1556 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:25.089696  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.116087  1552 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:25.117430  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:25.120792  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.133049  2352 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:25.121497  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.134449  2349 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:25.121786  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.135237  2348 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 13:00:25.121874  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.122244  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:25.138049  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 48436us
I20260709 13:00:25.139051  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.126261  1525 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:25.132460  2350 raft_consensus.cc:2376] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3. Re-sending same reply.
I20260709 13:00:25.109164  2353 rpcz_store.cc:276] 0709 13:00:24.026309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.035003 (+  8694us) service_pool.cc:224] Handling call
0709 13:00:24.199335 (+164332us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b5800003c98. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:00:24.199540 (+   205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.108284 (+908744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.108560 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9011968}
I20260709 13:00:25.147569  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.139515  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.149144  1555 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 13:00:25.150812  1555 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:25.151579  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1994) took 2884 ms. Trace:
I20260709 13:00:25.152004  1555 rpcz_store.cc:276] 0709 13:00:22.267365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.150093 (+2882728us) service_pool.cc:224] Handling call
0709 13:00:25.150565 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.151267 (+   702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.151505 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.155372  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.155977  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1999) took 2882 ms. Trace:
I20260709 13:00:25.156582  1559 raft_consensus.cc:2410] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:25.156558  1561 rpcz_store.cc:276] 0709 13:00:22.273831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.154891 (+2881060us) service_pool.cc:224] Handling call
0709 13:00:25.155300 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.155551 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.155590 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.155655 (+    65us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:25.155703 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.155781 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.155915 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.161516  1561 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:00:25.162413  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2002) took 2870 ms. Trace:
I20260709 13:00:25.162837  1561 rpcz_store.cc:276] 0709 13:00:22.292147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.160162 (+2868015us) service_pool.cc:224] Handling call
0709 13:00:25.160588 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.162048 (+  1460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.162334 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.097151  3816 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:25.123332  2374 debug-util.cc:398] Leaking SignalData structure 0x7b0800341ca0 after lost signal to thread 1585
I20260709 13:00:25.171943  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2000) took 2883 ms. Trace:
I20260709 13:00:25.175043  1559 rpcz_store.cc:276] 0709 13:00:22.287936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.157624 (+2869688us) service_pool.cc:224] Handling call
0709 13:00:25.169548 (+ 11924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.170438 (+   890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.170479 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.170539 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:25.170597 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.170685 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.170851 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.178084  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.179215  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.092689  3898 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:25.181641  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.183491  1707 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:25.183740  1559 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.184684  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2006) took 2886 ms. Trace:
I20260709 13:00:25.123080  3565 raft_consensus.cc:2764] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:00:25.138166  3873 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:25.138679  3915 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:25.106323  3901 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:25.145946  3783 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:25.092458  3652 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:25.195194  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.195835  3898 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:25.197193  3874 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.197628  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.197940  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1576) took 3094 ms. Trace:
W20260709 13:00:25.213286  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 230204us
I20260709 13:00:25.226320  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.229918  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.229581  1560 rpcz_store.cc:276] 0709 13:00:22.103238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.043990 (+2940752us) service_pool.cc:224] Handling call
0709 13:00:25.044361 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.127038 (+ 82677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.197688 (+ 70650us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b5400312254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:25.197880 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69020032}
I20260709 13:00:25.164901  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2005) took 2870 ms. Trace:
I20260709 13:00:25.145761  3875 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:25.154565  3852 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:25.154507  3912 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:25.168385  3907 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:25.238512  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 110604us
I20260709 13:00:25.154937  3799 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:25.238798  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.239355  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:25.239934  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.240471  1527 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.240674  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.240764  1707 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:25.209111  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2007) took 2910 ms. Trace:
I20260709 13:00:25.241766  1553 rpcz_store.cc:276] 0709 13:00:22.299022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.208289 (+2909267us) service_pool.cc:224] Handling call
0709 13:00:25.208613 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.208799 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.208825 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.208869 (+    44us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:25.208906 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.208958 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.209052 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.243130  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.244123  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 267) took 1199 ms. Trace:
I20260709 13:00:25.173316  3816 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.222505  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.245944  1704 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:00:25.244670  1553 rpcz_store.cc:276] 0709 13:00:24.044764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.243094 (+1198330us) service_pool.cc:224] Handling call
0709 13:00:25.243502 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.243744 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.243784 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.243845 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:25.243892 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.243961 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.244069 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.231459  1560 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:25.247165  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2015) took 1522 ms. Trace:
I20260709 13:00:25.232613  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.247848  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 268) took 1202 ms. Trace:
I20260709 13:00:25.247663  1560 rpcz_store.cc:276] 0709 13:00:23.725073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.230980 (+1505907us) service_pool.cc:224] Handling call
0709 13:00:25.231284 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.246913 (+ 15629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.247108 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.251179  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2011) took 2912 ms. Trace:
I20260709 13:00:25.226500  1703 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
W20260709 13:00:25.226945  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 152949us
I20260709 13:00:25.248446  1553 rpcz_store.cc:276] 0709 13:00:24.044972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.246782 (+1201810us) service_pool.cc:224] Handling call
0709 13:00:25.247182 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.247425 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.247464 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.247527 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:25.247573 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.247645 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.247778 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.251623  1554 rpcz_store.cc:276] 0709 13:00:23.723700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.222011 (+1498311us) service_pool.cc:224] Handling call
0709 13:00:25.250657 (+ 28646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.250836 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.250860 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.250935 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:25.250977 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.251027 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.251125 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:25.227716  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 115970us
I20260709 13:00:25.236095  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.237466  3870 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.124830  3752 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:25.237677  3875 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:25.192651  3772 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:00:25.231638  1561 rpcz_store.cc:276] 0709 13:00:22.294492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.163810 (+2869318us) service_pool.cc:224] Handling call
0709 13:00:25.164220 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.164468 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.164506 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.164568 (+    62us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:25.164616 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.164690 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.164830 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:25.284017  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44741.9us
I20260709 13:00:25.284453  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.285168  2348 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:25.285102  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:25.287869  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:25.229596  3652 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:25.288821  1632 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:25.287346  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:25.259156  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.200197  3873 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:25.118680  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.258425  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.248226  2350 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:25.212144  3783 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:25.087507  3715 ts_tablet_manager.cc:1469] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Trace:
0709 13:00:15.636025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.636334 (+   309us) service_pool.cc:224] Handling call
0709 13:00:15.956191 (+319857us) ts_tablet_manager.cc:934] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Initiating tablet copy from peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
0709 13:00:17.809968 (+1853777us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:00:18.142256 (+332288us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:18.153394 (+ 11138us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:00:19.226152 (+1072758us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:00:19.287995 (+ 61843us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:19.337118 (+ 49123us) ts_tablet_manager.cc:989] Replica registered.
0709 13:00:19.357066 (+ 19948us) inbound_call.cc:177] Queueing success response
0709 13:00:21.000178 (+1643112us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:00:23.998033 (+2997855us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:00:24.052510 (+ 54477us) tablet_replica.cc:261] Starting instrumentation
0709 13:00:24.053325 (+   815us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":548,"raft.queue_time_us":1352,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":86526,"tablet-copy.queue_time_us":226344,"tablet-download-pool.queue_time_us":3831,"tablet-download-pool.run_cpu_time_us":2238,"tablet-download-pool.run_wall_time_us":1203808,"thread_start_us":266400,"threads_started":2}
I20260709 13:00:25.299750  3873 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:25.304206  3931 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:00:25.301877  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.185060  1559 rpcz_store.cc:276] 0709 13:00:22.298722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.176088 (+2877366us) service_pool.cc:224] Handling call
0709 13:00:25.183489 (+  7401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.184335 (+   846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.184603 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.305683  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.306679  2349 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 2: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:25.303028  3783 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.293359  3896 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:25.310981  3896 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.311367  3896 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:25.301808  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.314246  2353 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:25.314961  3752 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:25.268864  1710 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:25.319161  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2024) took 1482 ms. Trace:
I20260709 13:00:25.319620  1710 rpcz_store.cc:276] 0709 13:00:23.836477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.868973 (+1032496us) service_pool.cc:224] Handling call
0709 13:00:25.319100 (+450127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.320539  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:25.272866  3801 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:25.206606  3901 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.281891  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.322494  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:25.282406  3805 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:25.327066  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:25.306718  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:25.328569  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.254897  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.290102  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:25.290486  1628 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:00:25.329227  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:25.290828  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.291618  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.330168  1631 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:25.330435  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.292498  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.331017  1633 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:25.331522  1561 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:00:25.294216  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.331871  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.332548  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.333762  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.334532  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:25.335515  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:25.335808  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.337592  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:25.295871  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:25.342067  1635 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:25.297670  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.238967  3907 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:25.257638  3912 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:25.251267  3816 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:25.293933  3652 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.319063  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:25.320602  1704 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:00:25.325893  2374 debug-util.cc:398] Leaking SignalData structure 0x7b08001bcb20 after lost signal to thread 1660
I20260709 13:00:25.321161  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.354017  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:25.354377  1703 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:00:25.348995  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7472909869999853s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 13:00:25.355196  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4241909870000313s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2169s for a response
I20260709 13:00:25.323071  1701 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.263163  1554 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:25.355659  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4202121060000081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
I20260709 13:00:25.324550  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:25.356696  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 274) took 1283 ms. Trace:
W20260709 13:00:25.225615  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 99423.9us
I20260709 13:00:25.255738  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.357090  1554 rpcz_store.cc:276] 0709 13:00:24.073716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.262536 (+1188820us) service_pool.cc:224] Handling call
0709 13:00:25.262928 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.356424 (+ 93496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.356618 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:25.357863  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 102175us
I20260709 13:00:25.358072  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:25.358382  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:25.358780  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.360558  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:25.322902  3801 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.287649  3539 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:25.346657  1708 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:25.347132  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.347222  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.333190  1629 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:25.342998  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.350747  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.250655  3799 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.351488  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.352540  2318 leader_election.cc:400] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:25.352984  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.355697  1703 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:25.357297  1702 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:25.358937  1554 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.331039  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.344918  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:25.325778  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.369102  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:25.368799  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:25.370909  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:25.384608  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.350811  3907 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.494261  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.555862  1633 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:25.398005  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.551541  1559 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:25.552435  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:25.393235  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.370956  1527 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:25.447885  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.553308  1636 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:25.558003  1630 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:25.557065  1635 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:25.507072  1627 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:25.550948  1704 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:25.559186  2318 leader_election.cc:403] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:25.551091  1710 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:25.559525  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:25.550729  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0251121059999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
I20260709 13:00:25.392406  1701 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
W20260709 13:00:25.560211  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0251121059999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
W20260709 13:00:25.561013  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 208629us
I20260709 13:00:25.560263  1704 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
W20260709 13:00:25.353873  2374 debug-util.cc:398] Leaking SignalData structure 0x7b0800224f00 after lost signal to thread 1661
I20260709 13:00:25.561452  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:25.560923  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0098121059999698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
I20260709 13:00:25.562247  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:25.564316  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0098121059999698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
I20260709 13:00:25.564494  1628 raft_consensus.cc:2376] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3. Re-sending same reply.
I20260709 13:00:25.552083  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.371471  3918 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:25.565462  3915 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.565847  3918 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:25.566022  3915 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.563709  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:25.562125  1710 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:25.566435  1634 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:25.353392  3912 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.349282  3923 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:25.561520  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:25.383388  3707 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.748s	user 0.008s	sys 0.005s
I20260709 13:00:25.574301  3912 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.574687  1701 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 1.
I20260709 13:00:25.574994  3707 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:25.575786  1555 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.576598  3707 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:25.577119  3707 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.573416  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.350106  3917 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:25.592796  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.383523  3820 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:25.362761  3852 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:25.579718  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:25.622910  1703 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:25.566294  3918 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:25.385921  3828 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:25.367318  3801 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.385211  3800 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:25.629942  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:25.566759  3921 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.012s
I20260709 13:00:25.556800  3799 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:25.601845  1557 raft_consensus.cc:2368] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:25.554960  3734 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:00:25.420812  3783 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:25.608669  3901 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:25.638015  1558 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:25.638532  3901 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.598764  3899 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:25.639180  3901 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:25.640461  3901 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:25.640820  3901 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:25.641173  3899 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:25.579631  1554 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:25.578193  1630 raft_consensus.cc:2393] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:25.612269  3896 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:25.566532  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:25.618883  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:25.619354  3926 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.595s	user 0.013s	sys 0.023s
I20260709 13:00:25.619469  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.620066  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.385805  3922 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:25.352864  3825 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.232s
I20260709 13:00:25.577561  3707 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:25.626636  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 256388us
I20260709 13:00:25.630333  3828 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:25.628427  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.629324  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.377987  3805 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:25.670153  1634 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:25.667824  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 269974us
I20260709 13:00:25.633957  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.670209  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:25.672020  1632 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:25.635566  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.634292  3921 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:25.643491  3783 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:25.611327  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.567219  3816 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:25.376262  3875 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:25.646688  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 355922us
W20260709 13:00:25.647454  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 276733us
I20260709 13:00:25.631006  3820 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:25.650336  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.574697  3923 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:25.655593  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1997) took 3387 ms. Trace:
I20260709 13:00:25.663012  3922 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:25.657711  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 92608.1us
I20260709 13:00:25.622232  3852 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:25.659178  1631 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:25.659703  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.631837  3881 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:25.707489  3881 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:25.707916  3881 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:25.662659  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1577) took 3556 ms. Trace:
I20260709 13:00:25.710521  1556 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:25.710209  1558 rpcz_store.cc:276] 0709 13:00:22.106348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.044003 (+2937655us) service_pool.cc:224] Handling call
0709 13:00:25.044361 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.638503 (+594142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.662455 (+ 23952us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b540026b1d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:25.662607 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1364775680}
I20260709 13:00:25.701886  1527 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:25.654520  2374 debug-util.cc:398] Leaking SignalData structure 0x7b08003878e0 after lost signal to thread 1734
I20260709 13:00:25.668100  1633 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:25.670816  1707 raft_consensus.cc:2368] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:25.602932  3917 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.672132  1636 raft_consensus.cc:2376] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2. Re-sending same reply.
I20260709 13:00:25.673672  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.674773  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2113) took 1382 ms. Trace:
I20260709 13:00:25.678381  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1544) took 3340 ms. Trace:
I20260709 13:00:25.712540  1527 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.681344  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1584) took 1424 ms. Trace:
I20260709 13:00:25.713666  1557 rpcz_store.cc:276] 0709 13:00:24.256423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.907851 (+651428us) service_pool.cc:224] Handling call
0709 13:00:25.681013 (+773162us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b5400311d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:25.681275 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1424684800}
I20260709 13:00:25.663239  3752 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:25.655601  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.717896  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:25.712770  1627 rpcz_store.cc:276] 0709 13:00:24.292722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.866531 (+573809us) service_pool.cc:224] Handling call
0709 13:00:25.674591 (+808060us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b54002dbed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:25.674708 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972464896}
I20260709 13:00:25.718606  1558 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:25.695906  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.695708  1555 rpcz_store.cc:276] 0709 13:00:22.268165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.153020 (+2884855us) service_pool.cc:224] Handling call
0709 13:00:25.153419 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.576174 (+422755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.653259 (+ 77085us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54001c0054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:25.655522 (+  2263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":970893184}
W20260709 13:00:25.719458  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 358295us
I20260709 13:00:25.719892  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:25.661787  3926 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:25.669023  3828 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:25.717815  3943 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:25.717549  3752 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:25.700799  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:25.668680  3778 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:00:25.711113  1705 raft_consensus.cc:2368] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4.
W20260709 13:00:25.665823  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 410957us
I20260709 13:00:25.734369  3652 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:25.731523  3801 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:25.627497  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.714867  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.756973  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:25.753233  3652 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:25.679250  3782 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:25.713090  1709 rpcz_store.cc:276] 0709 13:00:22.338124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.882759 (+2544635us) service_pool.cc:224] Handling call
0709 13:00:24.967066 (+ 84307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:24.967317 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:25.139206 (+171889us) log.cc:844] Serialized 14 byte log entry
0709 13:00:25.677667 (+538461us) spinlock_profiling.cc:243] Waited 6.18 ms on lock 0x7b18003604c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a
0709 13:00:25.677910 (+   243us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.677976 (+    66us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:25.678045 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.678166 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.678315 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":269833,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":1062,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":7519,"spinlock_wait_cycles":14222592,"thread_start_us":576291,"threads_started":2,"wal-append.queue_time_us":401450}
I20260709 13:00:25.716461  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:25.720948  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.633461  3873 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:25.723798  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:25.724396  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 160813us
I20260709 13:00:25.724738  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:25.696743  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.726382  1558 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:25.798961  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 82566.3us
I20260709 13:00:25.763331  2352 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:25.800585  1555 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:25.758239  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.696635  3820 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:25.727200  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.799319  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:25.741117  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:25.746695  1554 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:25.750102  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:25.702838  3922 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.840164  1553 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.745678  1556 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.761742  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:25.862406  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:25.862704  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 60906.5us
I20260709 13:00:25.857026  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:25.862352  1459 catalog_manager.cc:5697] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:25.753813   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:25.761618  3898 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:25.864993  3898 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.863848  3828 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:25.865365  3898 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:25.865763  2349 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:25.866124  1555 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:25.750207  3539 raft_consensus.cc:2764] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:25.866674  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2021) took 1352 ms. Trace:
I20260709 13:00:25.866848  3828 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.803260  3820 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.867053  1555 rpcz_store.cc:276] 0709 13:00:24.514562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.865585 (+1351023us) service_pool.cc:224] Handling call
0709 13:00:25.865834 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.866459 (+   625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.866624 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.739812  3752 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:25.870247  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 272) took 1823 ms. Trace:
W20260709 13:00:25.871254  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 113186us
I20260709 13:00:25.864377  1557 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:25.871399  2348 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 1.
I20260709 13:00:25.872270  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2019) took 1495 ms. Trace:
I20260709 13:00:25.872690  1557 rpcz_store.cc:276] 0709 13:00:24.377115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.860845 (+1483730us) service_pool.cc:224] Handling call
0709 13:00:25.864082 (+  3237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.872043 (+  7961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.872218 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.870661  1553 rpcz_store.cc:276] 0709 13:00:24.046454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.252677 (+1206223us) service_pool.cc:224] Handling call
0709 13:00:25.253100 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.840752 (+587652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.870036 (+ 29284us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b54001ce154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:25.870194 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1349703424}
I20260709 13:00:25.841444  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1585) took 1584 ms. Trace:
I20260709 13:00:25.876194  1556 rpcz_store.cc:276] 0709 13:00:24.256646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.745155 (+1488509us) service_pool.cc:224] Handling call
0709 13:00:25.745504 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.841105 (+ 95601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.841371 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.798182  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 277) took 1600 ms. Trace:
I20260709 13:00:25.710642  3852 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:25.804383  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.878926  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:25.894629  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.893543  1627 raft_consensus.cc:2368] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3.
I20260709 13:00:25.896173  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.809808  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:25.879679  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1587) took 1591 ms. Trace:
I20260709 13:00:25.879876  1630 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.898243  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:25.899477  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.866676  3874 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:25.736364  3943 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:25.881619  1526 leader_election.cc:400] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4.
I20260709 13:00:25.909415  1526 leader_election.cc:403] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:25.738216  2374 debug-util.cc:398] Leaking SignalData structure 0x7b080030c420 after lost signal to thread 2377
I20260709 13:00:25.887440  1633 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:25.888891  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.911697  1631 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:25.889648  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.890373  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.889686  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.878248  1558 rpcz_store.cc:276] 0709 13:00:24.198081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.725962 (+1527881us) service_pool.cc:224] Handling call
0709 13:00:25.726230 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.797877 (+ 71647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.798118 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.901218  2352 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 13:00:25.899135  1554 rpcz_store.cc:276] 0709 13:00:24.288711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.746256 (+1457545us) service_pool.cc:224] Handling call
0709 13:00:25.746529 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.879333 (+132804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.879609 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.903164  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.915541  2348 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 13:00:25.898469  1459 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 5 to 6, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 6 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:25.906703  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.906912  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:25.739392  3921 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:25.741282  3870 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:25.907642  1458 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 269) took 7211 ms. Trace:
I20260709 13:00:25.797636  3917 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:25.907145  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:25.919498  1632 raft_consensus.cc:2368] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5.
I20260709 13:00:25.803650  3652 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:25.922636  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 172625us
I20260709 13:00:25.926175  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.926692  1675 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:25.924875  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 183823us
I20260709 13:00:25.927790  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:00:25.922642  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 49754.5us
I20260709 13:00:25.920303  1459 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:25.928030  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:25.940055  1630 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:25.932571  3942 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:25.894536  3899 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:25.934686  3950 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:25.893690  3825 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:25.924697  3870 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:25.935948  3917 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:25.936344  3652 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.937071  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:25.936774  3707 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:25.931382  3873 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:25.855770  3907 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:25.881295  3772 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:00:25.893311  3783 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:00:25.893501  3828 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:25.942368  3565 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:25.945990  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:25.947908  1675 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:25.947896  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:25.953169  1602 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:25.977154  1634 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:25.736014  3816 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.734789  3896 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:25.739173  3915 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:25.800879  3912 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:25.999765  1560 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.958511  1630 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:26.002012  1634 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:25.959725  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.003648  1632 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:25.894320  3935 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:25.955924  1676 leader_election.cc:336] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.531s (SENT)
I20260709 13:00:25.972263  3828 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.921260  1458 rpcz_store.cc:276] 0709 13:00:18.687940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.724362 (+ 36422us) service_pool.cc:224] Handling call
0709 13:00:25.898786 (+7174424us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:24.767801 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:24.768679 (+   878us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:24.768734 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:00:24.768762 (+    28us) tablet.cc:662] Decoding operations
0709 13:00:24.769583 (+   821us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:24.769674 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:24.769715 (+    41us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:00:24.770173 (+   458us) tablet.cc:701] Row locks acquired
0709 13:00:24.770200 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:00:24.770416 (+   216us) write_op.cc:270] Start()
0709 13:00:24.770574 (+   158us) write_op.cc:276] Timestamp: P: 1783602024770343 usec, L: 0
0709 13:00:24.770604 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:00:24.873615 (+103011us) log.cc:844] Serialized 1317 byte log entry
0709 13:00:25.565415 (+691800us) op_driver.cc:464] REPLICATION: finished
0709 13:00:25.709376 (+143961us) write_op.cc:301] APPLY: starting
0709 13:00:25.709463 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:25.709619 (+   156us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:25.709763 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:25.714141 (+  4378us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:25.741358 (+ 27217us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:00:25.741404 (+    46us) write_op.cc:312] APPLY: finished
0709 13:00:25.741663 (+   259us) log.cc:844] Serialized 35 byte log entry
0709 13:00:25.741817 (+   154us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:25.741906 (+    89us) write_op.cc:454] Released schema lock
0709 13:00:25.741972 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:25.742294 (+   322us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:25.742322 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13648,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":143690,"apply.run_cpu_time_us":2262,"apply.run_wall_time_us":32914,"num_ops":3,"prepare.queue_time_us":454119,"prepare.run_cpu_time_us":91306,"prepare.run_wall_time_us":322881,"raft.queue_time_us":354384,"raft.run_cpu_time_us":16283,"raft.run_wall_time_us":107143,"replication_time_us":794657,"thread_start_us":906376,"threads_started":4,"wal-append.queue_time_us":221125}]]}
I20260709 13:00:25.916700  3874 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:26.031904  1561 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:25.965806  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:25.916905  3943 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.925375  3801 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:25.806517  3901 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.061851  1559 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:25.973547  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:25.975715  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 94265.6us
I20260709 13:00:25.803308  3800 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:25.976609  2351 raft_consensus.cc:2376] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3. Re-sending same reply.
I20260709 13:00:25.954682  3950 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.969815  3946 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.977895  2318 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:26.081758  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 104037us
I20260709 13:00:25.989130  1674 leader_election.cc:400] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3.
I20260709 13:00:25.983147  1675 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:26.083029  1674 leader_election.cc:403] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 13:00:26.083406  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 100319us
W20260709 13:00:26.083910  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 94928.3us
I20260709 13:00:26.084821  1675 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.962658  3870 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:25.962878  3652 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:26.020644  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 293309us
I20260709 13:00:25.954160  3805 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.086999  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:26.088160  1525 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.004667  3941 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.020205  3935 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:26.089699  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:26.090463  1705 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.021881  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:25.989138  2344 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:00:26.094849  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.946070  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.067055  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:26.036706  3828 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:26.069406  3922 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.104884  1702 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:26.070214  3952 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:26.073613  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 271) took 2027 ms. Trace:
I20260709 13:00:26.109211  1560 rpcz_store.cc:276] 0709 13:00:24.046170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.249684 (+1203514us) service_pool.cc:224] Handling call
0709 13:00:25.250609 (+   925us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.005116 (+754507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.073380 (+ 68264us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b540038a9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:26.073559 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1722371456}
I20260709 13:00:26.004542  3734 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:26.061998  3874 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:25.972431  3899 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:25.997731  3707 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:26.034474  2349 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:26.033723  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.111192  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.032778  2348 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:26.079787  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.082578  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:25.998500  3947 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:25.961024  3898 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:26.093186  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:26.126797  1703 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:26.027433  3816 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.062310  3918 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.095554  3935 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.067695  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:26.103825  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:26.085050  3950 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.115098  3801 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:26.115669  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.115983  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.117307  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.098634  3783 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:26.099112  3856 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:26.094281  3896 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3.
I20260709 13:00:26.125636  2318 leader_election.cc:400] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
W20260709 13:00:26.136734  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 183627us
I20260709 13:00:26.097929  1675 leader_election.cc:400] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5.
I20260709 13:00:26.164294  1675 leader_election.cc:403] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:26.165103  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 67377.8us
I20260709 13:00:26.165657  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:26.166014  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:26.098866  3936 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:26.101271  3942 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:26.139962  1710 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:26.147595  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.113057  3805 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.108853  3952 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:26.153940  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 226206us
I20260709 13:00:26.156145  1553 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:26.154737  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.155706  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:26.208973  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 53387us
I20260709 13:00:26.115960  3565 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:26.112787  3734 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:26.167510  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:26.209694  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.162358  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:26.162108  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 206219us
W20260709 13:00:26.210834  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.210902  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.161962  1705 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:26.172927  3960 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5.
I20260709 13:00:26.088009  3917 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:26.253413  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:26.254211  1704 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:26.251493  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:26.255571  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.256146  1634 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.256461  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.217039  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:26.211851  3734 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:26.212666  3707 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:26.238515  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.168463  3772 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.114727  3799 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:26.151049  3539 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:00:26.273846  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:26.204798  3942 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:26.208721  3936 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.154961  3783 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.211884  3939 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:26.252104  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.140446  3926 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:26.163236  3825 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:26.098258  3946 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.139778  3940 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:26.257355  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.257974  1708 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:26.155850  3956 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.162199  3856 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:26.318105  3856 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:26.212224  3565 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.275393  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:26.319008  3921 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:26.319638  3921 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:26.320003  1636 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:26.275226  1710 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:26.212595  3801 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:26.246148  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.222131  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:26.322378  2348 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 1.
I20260709 13:00:26.319054  3565 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:26.213555  3952 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:26.283695  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.285934  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:26.293862  1552 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:26.299647  3825 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:26.336725  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:26.299932  3957 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:26.300091  3800 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.301210  3922 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:26.280784  3947 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:00:26.304194  3918 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:26.280586  3941 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:26.379808  3854 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:26.379897  3941 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.380359  3854 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:26.380627  3941 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.291601  3936 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:26.285593  1646 tablet_copy_service.cc:342] P ba0548b7d2a94e8598475a46c279fe0e: Request end of tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-316c6fe30c8f416c942b4338f4e6eab9 received from {username='slave'} at 127.0.0.1:48246
I20260709 13:00:26.307332  3946 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:26.350422  3953 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:00:26.311038  3939 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:26.320688  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2048) took 1085 ms. Trace:
I20260709 13:00:26.388022  1706 rpcz_store.cc:276] 0709 13:00:25.235653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.235859 (+   206us) service_pool.cc:224] Handling call
0709 13:00:26.320520 (+1084661us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001e9454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:26.320638 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2386464128}
I20260709 13:00:26.263399  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:26.333911  3896 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.276634  3955 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.391812  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1993) took 4124 ms. Trace:
I20260709 13:00:26.392267  1552 rpcz_store.cc:276] 0709 13:00:22.267064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.132505 (+2865441us) service_pool.cc:224] Handling call
0709 13:00:25.174220 (+ 41715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.336206 (+1161986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.391527 (+ 55321us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400207954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:26.391747 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2574384000}
I20260709 13:00:26.393821  1557 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:26.341495  2349 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:00:26.347834  3881 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.001s
I20260709 13:00:26.352718  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:26.294556  3783 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:26.405356  3937 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
I20260709 13:00:26.405658  3918 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:26.406226  3918 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:26.384519  3942 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:26.406700  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 653) took 1115 ms. Trace:
I20260709 13:00:26.407379  3942 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.340586  3782 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.381350  3854 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.307734  3940 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:26.350692  3915 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:26.387321  3956 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:00:26.327208  2317 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:26.434691  3946 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:26.434765  3875 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:26.435163  3946 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.435237  3875 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:26.435623  3946 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.320772  3921 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:26.385802  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50403.8us
I20260709 13:00:26.369431  3800 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:26.390332  1646 tablet_copy_service.cc:434] P ba0548b7d2a94e8598475a46c279fe0e: ending tablet copy session 95fc6a4cf5874b7eb91280dffcd3b3fb-316c6fe30c8f416c942b4338f4e6eab9 on tablet 316c6fe30c8f416c942b4338f4e6eab9 with peer 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:26.408263  3782 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:26.419386  3940 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:26.292908  3899 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.389464  3801 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:00:26.409382  3881 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.407086  2347 rpcz_store.cc:276] 0709 13:00:25.291558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.292066 (+   508us) service_pool.cc:224] Handling call
0709 13:00:25.292426 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:25.319113 (+ 26687us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.319149 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:25.327114 (+  7965us) log.cc:844] Serialized 26 byte log entry
0709 13:00:25.629964 (+302850us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:25.630019 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.630088 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:26.406479 (+776391us) raft_consensus.cc:1685] finished
0709 13:00:26.406511 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.406639 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26271,"raft.queue_time_us":32873,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":20017,"thread_start_us":283056,"threads_started":1,"wal-append.queue_time_us":623547}
I20260709 13:00:26.426189  3939 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.383222  3926 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:26.433822  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1606) took 1054 ms. Trace:
I20260709 13:00:26.490912  1557 rpcz_store.cc:276] 0709 13:00:25.379491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.907751 (+528260us) service_pool.cc:224] Handling call
0709 13:00:26.393561 (+485810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.401853 (+  8292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.433594 (+ 31741us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b5400207950. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:26.433762 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1078424832}
I20260709 13:00:26.438272  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:26.492347  1630 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
W20260709 13:00:26.438915  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 111764us
I20260709 13:00:26.493727  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:00:26.401160  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 48527.5us
I20260709 13:00:26.496641  2353 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:26.497676  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 246265us
I20260709 13:00:26.498672  3962 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:26.501762  3944 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:26.502347  3944 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:26.502784  3881 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:26.505237  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:26.505645  1676 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 2.121s (SENT)
I20260709 13:00:26.506331  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:26.490289  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:26.515123  1629 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.010s	sys 0.008s
W20260709 13:00:26.531716  1629 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.012s	sys 0.008s
W20260709 13:00:26.486608  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 239515us
W20260709 13:00:26.531152  1632 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.527s	user 0.014s	sys 0.000s
I20260709 13:00:26.533798  1632 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
W20260709 13:00:26.546288  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:26.546986  1459 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.003s	sys 0.009s
I20260709 13:00:26.549059  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:26.549425  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:26.549748  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 381412us
I20260709 13:00:26.503561  3921 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.552246  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:26.553007  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:26.550583  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:26.554386  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:26.501358  3962 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:00:26.504530  3926 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:26.555253  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:26.407104  3918 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:26.557338  1557 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:26.530957  3898 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:26.544094  3959 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:26.436159  3875 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:26.560493  1601 leader_election.cc:400] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:26.567047  1601 leader_election.cc:403] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:00:26.544648  3951 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:00:26.560334  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 343272us
I20260709 13:00:26.562186  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:26.570153  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:26.570170  1553 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
W20260709 13:00:26.547387  1459 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.003s	sys 0.009s
W20260709 13:00:26.564533  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66060.9us
I20260709 13:00:26.571434  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:26.571748  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 475363us
I20260709 13:00:26.571931  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:26.573733  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.554947  3957 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:00:26.588857  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.562711  3799 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:26.590927  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.591135  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:26.593115  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:26.595077  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.563459  3959 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.591641  2318 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:26.598953  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.601116  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.601282  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:26.601742  1634 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:26.602105  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:26.602752  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.612181  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.613947  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:26.616492  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.616896  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.503347  3944 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.590770  3956 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:26.619321  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.578962  3898 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:26.555442  3963 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:00:26.620858  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.670209  1630 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.617899  1634 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:26.624505  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.623019  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.591938  3873 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:26.604610  3959 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.594775  3974 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:26.694196  3974 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.694648  3974 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.626636  3955 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:26.629874  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:26.607800  3954 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.609048  3915 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.682597  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.621447  3799 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:26.621663  3801 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:26.683549  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.684355  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.687048  1627 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:26.653970  3971 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:00:26.720239  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.681627  3873 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:26.722818  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.590340  3967 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:26.700330  1630 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:26.709760  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.707742  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:26.726428  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:26.727526  1627 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:26.728754  3852 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.788s	user 0.018s	sys 0.005s
W20260709 13:00:26.706228  3820 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.839s	user 0.014s	sys 0.009s
W20260709 13:00:26.733134  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 140918us
W20260709 13:00:26.707031  3870 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.009s
I20260709 13:00:26.734709  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.741174  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:26.736240  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 175895us
W20260709 13:00:26.738560  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 161269us
W20260709 13:00:26.743463  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.699780  3898 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:26.723522  3752 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.854s	user 0.006s	sys 0.017s
I20260709 13:00:26.744654  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:26.625651  3946 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:26.729616  3950 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 13:00:26.662490  3960 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:00:26.729888  3873 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:26.723289  3707 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:26.776700  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 185123us
I20260709 13:00:26.823143  2318 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:26.737367  3881 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:26.742931  3874 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.001s
I20260709 13:00:26.680557  3970 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.129s
W20260709 13:00:26.742632  3828 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.630s	user 0.006s	sys 0.006s
I20260709 13:00:26.824506  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:26.794059  3951 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:26.774101  3943 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.682s	user 0.005s	sys 0.006s
I20260709 13:00:26.741966  3782 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:26.686367  3856 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.773751  3772 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.723022  3734 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:26.661334  3926 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:26.872990  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 126637us
W20260709 13:00:26.873579  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 49139.4us
I20260709 13:00:26.724480  3955 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:26.874570  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 133620us
W20260709 13:00:26.882553  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 303559us
I20260709 13:00:26.887445  3825 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:26.888613  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:26.792779  3950 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:26.742251  3820 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.890817  3907 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:26.739392  3964 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:26.894307  1627 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:26.746557  3870 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.953433  1674 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:26.953843  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:26.991125  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 93653.9us
I20260709 13:00:26.991778  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:26.821611  3958 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.954794  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:26.820475  3957 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:26.816273  3966 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:26.775324  3752 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:26.741668  3852 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:26.969580  3783 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.018s
I20260709 13:00:26.995638  1560 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:27.132157  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.001s	sys 0.023s
W20260709 13:00:27.170948  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.002s	sys 0.023s
W20260709 13:00:27.005771  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51061.8us
I20260709 13:00:27.174505  3825 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.133846  1628 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:27.016875  1632 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:27.004556  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51207.7us
I20260709 13:00:27.143766  1635 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:27.175765  3825 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:27.035950  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.042922  2346 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:27.148517  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:27.148605  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:27.150221  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.151608  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:27.152626  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 187088us
I20260709 13:00:26.956795  3782 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.060263  3820 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:27.086341  3901 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:27.062719  3852 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:26.891202  3828 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.204969  1527 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:27.206262  1552 raft_consensus.cc:2368] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:27.206925  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:26.876662  3874 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.212214  3820 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:27.215794  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:27.216125  1560 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:27.216356  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:27.216856  1557 raft_consensus.cc:2368] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:27.217223  1708 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.217990  3828 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:26.959859  3881 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:27.217927  4001 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:27.223193  2351 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:26.998104  3898 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.037849  3912 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:27.044545  3935 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.881s	user 0.008s	sys 0.002s
I20260709 13:00:27.226750  3935 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.227505  3854 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.228390  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:27.094481  3941 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.713s	user 0.014s	sys 0.007s
I20260709 13:00:27.234707  3935 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:27.234467  3941 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:27.032473  3942 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.624s	user 0.005s	sys 0.006s
I20260709 13:00:27.066072  3946 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:27.091673  3950 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:27.034514  3951 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:27.097343  3952 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.726s	user 0.014s	sys 0.000s
I20260709 13:00:27.278753  3952 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.043252  3955 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.050374  3956 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:27.006304  3957 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:27.290102  3957 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.291967  3957 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.015215  3958 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.095455  3959 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:27.296893  3958 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:27.003736  3960 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.958532  3963 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:26.988494  3967 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:27.059237  3968 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260709 13:00:27.070506  3970 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:27.065912  3972 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 13:00:27.071348  3973 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260709 13:00:27.095422  3974 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:27.058841  3975 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:00:26.997200  3976 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:27.062047  3977 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.042703  3979 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:26.956743  3980 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.119s
I20260709 13:00:27.087283  3981 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:27.099740  3982 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:00:26.989750  3984 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:27.091723  3986 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:00:27.093649  3989 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:00:27.459740  3989 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:27.460160  3989 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:27.460695  3970 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:27.461161  3970 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:27.071496  4002 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:27.467073  4002 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:27.159636  3734 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:27.180447  3899 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
W20260709 13:00:27.183353  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.182981  3998 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:00:27.183781  1674 leader_election.cc:400] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5.
W20260709 13:00:27.504877  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 321557us
I20260709 13:00:27.184607  3954 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:27.185496  3782 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.186619  3870 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:27.188149  2353 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.190404  3783 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.071431  3856 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:26.897383  3943 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.513754  3998 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:27.515547  3965 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:27.205057  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:27.515966  3965 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:27.516356  3939 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:27.215795  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:27.257014  3936 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.818s	user 0.009s	sys 0.003s
I20260709 13:00:27.524128  1710 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.524039  3939 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.524178  3936 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.259024  3825 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:27.526443  3939 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:27.527161  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:27.528604  3734 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:27.261989  3752 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:27.286684  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:27.515906  3783 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:27.529135  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:27.516017  3943 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:27.329731  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:27.332494  3652 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.231s	user 0.002s	sys 0.022s
I20260709 13:00:27.530928  2350 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:27.531212  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:27.530972  3652 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.532115  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:27.532488  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:27.532836  2350 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 2: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:27.332644  2318 leader_election.cc:400] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:27.533375  3652 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:27.533620  2318 leader_election.cc:403] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:00:27.533867  3783 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:27.534149  3652 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:27.342437  1631 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:27.304534  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2054) took 1239 ms. Trace:
I20260709 13:00:27.304557  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 660) took 1030 ms. Trace:
W20260709 13:00:27.348039  3915 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.612s	user 0.008s	sys 0.008s
I20260709 13:00:27.299289  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 397) took 1356 ms. Trace:
I20260709 13:00:27.350195  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:27.305576  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2132) took 1416 ms. Trace:
W20260709 13:00:27.341778  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 47595.1us
I20260709 13:00:27.340775  1636 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:27.534696  3783 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:27.325225  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 419) took 1048 ms. Trace:
I20260709 13:00:27.537178  1635 rpcz_store.cc:276] 0709 13:00:26.258980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.259242 (+   262us) service_pool.cc:224] Handling call
0709 13:00:26.259514 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.176675 (+917161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.307193 (+130518us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54001b7454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:27.307409 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2032002432}
I20260709 13:00:27.281203  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:27.299394  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:27.299075  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:27.297381  3959 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:27.539149  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:27.539731  1706 raft_consensus.cc:2368] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 2. Current term is 3.
I20260709 13:00:27.540365  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:27.535169  1707 rpcz_store.cc:276] 0709 13:00:26.064922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.065557 (+   635us) service_pool.cc:224] Handling call
0709 13:00:26.093075 (+ 27518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.093566 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.093639 (+    73us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:26.108924 (+ 15285us) log.cc:844] Serialized 26 byte log entry
0709 13:00:26.567196 (+458272us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:26.567238 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.567310 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:27.304075 (+736765us) raft_consensus.cc:1685] finished
0709 13:00:27.304113 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.304321 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5491,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":14006,"thread_start_us":447447,"threads_started":1,"wal-append.queue_time_us":454900}
I20260709 13:00:27.540838  4001 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:27.541138  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:27.541525  4001 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:27.338672  3974 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:27.306208  3941 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:27.381076  3778 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:27.383980  3800 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.899s	user 0.010s	sys 0.006s
I20260709 13:00:27.384461  3881 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:00:27.298692  3963 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:27.299811  3967 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.353811  3958 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.387197  3981 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:27.388236  3799 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.642s	user 0.012s	sys 0.003s
I20260709 13:00:27.388603  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:27.393328  3852 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:00:27.396134  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2142) took 1407 ms. Trace:
I20260709 13:00:27.537827  3783 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.380076  3952 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:27.410418  3873 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.583s	user 0.014s	sys 0.003s
W20260709 13:00:27.409288  3816 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.258s	user 0.019s	sys 0.011s
I20260709 13:00:27.437656  3979 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:27.439826  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:27.439548  2346 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 655) took 1778 ms (client timeout 1770 ms). Trace:
I20260709 13:00:27.441013  3875 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.442871   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:27.447538  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:27.454726  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 423) took 1193 ms. Trace:
I20260709 13:00:27.563926  3873 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.526412  3936 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:27.466327  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:27.466235  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:27.485786  3940 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.966s	user 0.000s	sys 0.017s
W20260709 13:00:27.473006  3801 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.747s	user 0.007s	sys 0.010s
I20260709 13:00:27.461277  3989 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:27.483527  3565 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.139s	user 0.006s	sys 0.011s
I20260709 13:00:27.511286  3782 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:27.250038  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 295078us
I20260709 13:00:27.519100  2353 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
W20260709 13:00:27.521658  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 316909us
I20260709 13:00:27.522588  1555 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:27.522421  3965 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.523195  3950 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:27.529788  1705 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.328567  1708 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
W20260709 13:00:27.534312  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 202133us
I20260709 13:00:27.302497  3874 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:27.564086  3970 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.538261  1632 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:27.534986  3539 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:27.538800  1704 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:27.341552  3921 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.684s	user 0.014s	sys 0.000s
I20260709 13:00:27.307423  3976 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.542253  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:27.535291  1709 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:27.536386  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 420) took 1277 ms. Trace:
I20260709 13:00:27.535771  2351 rpcz_store.cc:276] 0709 13:00:25.942832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.162098 (+219266us) service_pool.cc:224] Handling call
0709 13:00:27.298991 (+1136893us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400796b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:27.299197 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2328184192}
I20260709 13:00:27.535908  1710 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:27.536105  1701 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.541816  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:27.558638  3943 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:27.560336  3981 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.560210  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:27.411069  3977 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:27.561790  3856 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:27.563329  1635 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:00:27.536243  1628 rpcz_store.cc:276] 0709 13:00:25.889443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.890144 (+   701us) service_pool.cc:224] Handling call
0709 13:00:27.305290 (+1415146us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400196754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:27.305457 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2805500288}
I20260709 13:00:27.673024  1635 pending_rounds.cc:85] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 2
I20260709 13:00:27.673333  1635 pending_rounds.cc:107] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:00:27.673607  1635 raft_consensus.cc:2889] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:27.567233  3816 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4.
I20260709 13:00:27.567458  3805 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:27.062475  3993 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.576846  3778 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.578788  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:27.580036  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:27.580295  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 133118us
I20260709 13:00:27.580796  1526 leader_election.cc:400] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 2 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 2. Current term is 3.
I20260709 13:00:27.570977  3799 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:27.535640  3915 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:27.567291  3852 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:27.561053  3800 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.585098  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:27.587805  4010 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:00:27.588068  2345 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:27.590426  3941 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:27.570717  2346 rpcz_store.cc:269] 0709 13:00:25.660933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.662040 (+  1107us) service_pool.cc:224] Handling call
0709 13:00:27.439133 (+1777093us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400550354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:27.439328 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2851893504}
I20260709 13:00:27.591147  3801 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:27.592141  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 311179us
I20260709 13:00:27.581703  2349 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.571303  1633 rpcz_store.cc:276] 0709 13:00:26.260920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.261785 (+   865us) service_pool.cc:224] Handling call
0709 13:00:26.262091 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.562970 (+300879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.563002 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:26.573876 (+ 10874us) log.cc:844] Serialized 26 byte log entry
0709 13:00:26.774415 (+200539us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:26.774455 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.774533 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:27.099039 (+324506us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:27.454176 (+355137us) raft_consensus.cc:1685] finished
0709 13:00:27.454212 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.454553 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40863,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":96,"spinlock_wait_cycles":6397696,"thread_start_us":459043,"threads_started":2,"wal-append.queue_time_us":356927}
I20260709 13:00:27.573082  3960 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:27.562069  1629 rpcz_store.cc:276] 0709 13:00:25.966465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.966594 (+   129us) service_pool.cc:224] Handling call
0709 13:00:25.967088 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.532146 (+565058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.532188 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:26.547969 (+ 15781us) log.cc:844] Serialized 26 byte log entry
0709 13:00:26.612056 (+ 64087us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:26.612088 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.612151 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:27.307497 (+695346us) raft_consensus.cc:1685] finished
0709 13:00:27.328814 (+ 21317us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.373988 (+ 45174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128269,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":11297,"thread_start_us":609944,"threads_started":2,"wal-append.queue_time_us":217563}
I20260709 13:00:27.611280  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1592) took 2342 ms. Trace:
I20260709 13:00:27.676420  3976 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:27.671459  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 62457.8us
I20260709 13:00:27.563593  4001 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:27.607512  3873 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:27.679051  1707 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:27.675035  1631 rpcz_store.cc:276] 0709 13:00:26.259248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.259686 (+   438us) service_pool.cc:224] Handling call
0709 13:00:26.259917 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.534937 (+1275020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.536079 (+  1142us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001b6a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:27.536336 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353337088}
I20260709 13:00:27.607676  3782 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:00:27.677990  1527 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:27.667840  3943 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.596699  3939 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:27.685248  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:27.686295  1627 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 2: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:27.690230  1632 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:27.671669  3977 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.676487  3856 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:27.690615  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:27.645351  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.700496  3993 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.595175  3958 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.686820  3852 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:00:27.703471  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 138606us
W20260709 13:00:27.705457  4013 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 13:00:27.706105  2345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 403) took 1761 ms (client timeout 2054 ms). Trace:
I20260709 13:00:27.707224  3800 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:27.708357  1633 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:27.694159  1555 rpcz_store.cc:276] 0709 13:00:25.268467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.870908 (+602441us) service_pool.cc:224] Handling call
0709 13:00:25.871738 (+   830us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.606234 (+1734496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.610912 (+  4678us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400207454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:27.611193 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3783366528}
I20260709 13:00:27.714234  1628 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:27.597179  3565 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.715689  1630 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.659984  3940 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.608968  4002 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.790053  4002 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:27.693883  3960 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:27.776273  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:27.777524  1708 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:27.793848  1710 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:00:27.781252  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 119531us
I20260709 13:00:27.777292  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:27.721764  3943 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:27.780681  2345 rpcz_store.cc:269] 0709 13:00:25.944611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.206546 (+261935us) service_pool.cc:224] Handling call
0709 13:00:27.705881 (+1499335us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b54004bf0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:27.706044 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2194750208}
I20260709 13:00:27.535292  2344 rpcz_store.cc:276] 0709 13:00:26.273834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.324054 (+ 50220us) service_pool.cc:224] Handling call
0709 13:00:26.324371 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.324755 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.324819 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:26.350675 (+ 25856us) log.cc:844] Serialized 26 byte log entry
0709 13:00:26.597038 (+246363us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:26.600116 (+  3078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.607356 (+  7240us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:27.303805 (+696449us) raft_consensus.cc:1685] finished
0709 13:00:27.303855 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.304334 (+   479us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8403,"raft.queue_time_us":1219,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":16520,"thread_start_us":173912,"threads_started":1,"wal-append.queue_time_us":308399}
I20260709 13:00:27.788391  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:27.784899  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:27.715453  3941 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:27.791018  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:27.778271  1706 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:27.785538  3816 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:27.793401  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:27.798622  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:27.798534  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:27.802469  2348 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
W20260709 13:00:27.817227  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 60775.4us
I20260709 13:00:27.780738  3539 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:00:27.779017  3874 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:27.812454  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:27.806314  4016 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:00:27.806046  3852 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.808662  1701 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:27.810169  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:27.795219  2318 leader_election.cc:400] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
W20260709 13:00:27.815344  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.835249  2318 leader_election.cc:403] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:27.835613  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.709748  3963 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.777719  3921 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:00:27.833312  1629 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:27.831965  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:27.838518  1632 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.840785  1707 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:27.847679  1674 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:27.842569  1627 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:27.876955  3963 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:27.880072  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 69960.3us
I20260709 13:00:27.880491  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:27.881418  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.824535  3950 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:27.882131  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 94908.5us
I20260709 13:00:27.886859  2349 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:27.833847  3816 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:27.891052  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 52160.9us
I20260709 13:00:27.889842  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:27.889998  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:27.890414  3801 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.889653  1552 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:27.889845  1631 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:27.815968  3881 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:00:27.904645  1634 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:27.792985  3734 consensus_queue.cc:1175] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba0548b7d2a94e8598475a46c279fe0e" responder_term: 2 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:27.906286  1559 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:27.881091  3915 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:27.853809  3940 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:27.925354  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 244851us
I20260709 13:00:27.904673  3899 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:27.930789  1705 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:27.837069  3874 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.891688  3854 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:27.937188  1704 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:00:27.937891  1708 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:27.938135  1555 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:27.938140  1560 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 2: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:27.930785  3825 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:27.931941  3899 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.930596  3852 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:27.946542  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66174.8us
I20260709 13:00:27.948647  3915 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:27.949569  4012 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.350s	user 0.000s	sys 0.002s
I20260709 13:00:27.949518  3816 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.950242  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:27.950894  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61182.7us
I20260709 13:00:27.996801  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:27.953903  3960 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.955402  3993 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:28.005100  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 156359us
I20260709 13:00:28.005237  3915 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:27.939982  3856 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.010155  3856 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.957739  3984 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:27.960049  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:27.960759  3778 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:27.960649  3981 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.963310  1553 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:27.933605  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:27.967952  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:27.938716  3954 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:27.968142  3899 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.972159  1632 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:27.973454  1555 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:00:27.974251  1557 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:00:27.983003  3565 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:27.983425  3801 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:27.990899  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 407) took 1402 ms. Trace:
I20260709 13:00:27.994099  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:28.029655  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:28.030315  1708 raft_consensus.cc:2368] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:27.999079  1630 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:28.025596  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:28.033808  3917 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:28.034260  3917 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.034298  1704 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:28.034706  1559 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:00:28.035019  3981 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:28.035300  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:28.035715  3998 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:28.029896  3825 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:00:28.036940  3801 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:28.029975  2348 rpcz_store.cc:276] 0709 13:00:26.588336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.588575 (+   239us) service_pool.cc:224] Handling call
0709 13:00:27.990699 (+1402124us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540054a454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:27.990838 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2760267008}
I20260709 13:00:27.995016  3782 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:28.046002  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:28.049815  1561 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
W20260709 13:00:28.048518  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 91830us
W20260709 13:00:28.051431  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:28.052871  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 102775us
I20260709 13:00:28.058871  1553 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 2: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:28.034729  3917 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.061249  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:27.945380  3941 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.038045  3954 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:28.043371  3958 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:28.045274  3984 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.068783  3800 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:28.103189  3800 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:28.104301  3800 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.070878  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:28.070335  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:28.074210  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:28.070895  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:28.075199  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:28.076148  1705 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:28.076717  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:28.076928  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:28.112082  1559 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:28.077306  3965 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:28.078297  3954 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.084025  3825 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:28.109530  1554 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:28.089991  1527 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:28.090946  1632 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:28.135793  1632 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:28.091450  1561 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:28.091910  1708 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:28.093555  3940 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:00:28.092291  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.052445  3993 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:28.138890  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:28.040951  3967 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.068090  3782 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.099047  3958 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:28.133693  1556 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.134104  1555 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:28.143296  1632 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.144881  1632 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.137670  1630 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:28.084084  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:28.138562  3957 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:28.162649  3960 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:28.144654  3898 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:00:28.190845  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 101896us
W20260709 13:00:28.191037  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 101149us
W20260709 13:00:28.191349  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 122879us
W20260709 13:00:28.167764  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2052) took 2103 ms (client timeout 2238 ms). Trace:
W20260709 13:00:28.198892  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 238990us
W20260709 13:00:28.198969  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 129815us
W20260709 13:00:28.192128  1709 rpcz_store.cc:269] 0709 13:00:26.064316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.065625 (+  1309us) service_pool.cc:224] Handling call
0709 13:00:28.167417 (+2101792us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54002a67d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:28.167663 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3328407936}
I20260709 13:00:28.138803  3940 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:28.221221  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:28.218624  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:28.232020  1629 raft_consensus.cc:2368] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 4.
I20260709 13:00:28.232335  3874 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:28.199321  3898 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:28.233043  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:28.229127  1676 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:28.167001  3936 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:28.201038  3957 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.242506  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:28.215670  3958 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:28.245915  1707 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:28.238481  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:28.248135  2348 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:28.249986  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:28.251838  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.252343  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:28.252494  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.254520  2349 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:28.234695  3970 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:28.258436  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:28.236434  3801 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:28.263198  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:28.263132  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:28.220523  3825 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:00:28.259802  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:28.264554  2347 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:28.265368  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:28.263162  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:28.268827  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:28.262823  1634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 1940) took 1635 ms (client timeout 1636 ms). Trace:
I20260709 13:00:28.271211  1627 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:28.273566  1628 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:28.273725  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:28.274386  1633 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:28.274825  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:28.275401  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.275784  1705 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:28.270201  1634 rpcz_store.cc:269] 0709 13:00:26.627450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.684057 (+ 56607us) service_pool.cc:224] Handling call
0709 13:00:28.246112 (+1562055us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400197b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:28.262768 (+ 16656us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2540187904}
I20260709 13:00:28.244876  3984 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:28.231101  3915 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:00:28.226380  4025 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:00:28.278489  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:28.250932  3734 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:28.282339  1554 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:28.284449  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.262571  3820 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:28.284937  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:28.291075  3967 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:28.292671  3565 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:28.293143  3565 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:28.293586  3565 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:28.296474  3899 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.287175  1560 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 2: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:28.288296  3989 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:28.310647  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 75085.2us
I20260709 13:00:28.311616  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:28.313380  1708 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:28.313601  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.307350  1561 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.309254  3957 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:28.307966  1631 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 2: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:28.313114  3898 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.268975  3958 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:28.315399  1559 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:28.285087  1556 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
W20260709 13:00:28.316249  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 247364us
I20260709 13:00:28.321776  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:28.317581  1632 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:00:28.323302  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 100673us
I20260709 13:00:28.278865  3816 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:00:28.328734  3951 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:28.329396  3957 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:28.315371  3782 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:28.333539  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:28.333926  3977 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.005s
I20260709 13:00:28.334329  3977 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:28.334859  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.319902  3943 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.336298  3977 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:28.334864  1601 leader_election.cc:400] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:28.337267  1601 leader_election.cc:403] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:28.337777  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.338160  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.338454  1704 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:28.338593  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.339358  1701 raft_consensus.cc:2368] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:28.340134  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.340790  1701 raft_consensus.cc:2368] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:28.340898  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:28.341687  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:28.341617  3825 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:28.342353  1701 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 1.
I20260709 13:00:28.341859  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:28.343137  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.349373  1704 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:28.284616  3936 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.353629  1707 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:28.353854  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:28.322248  3917 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:28.354465  3976 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.561s	user 0.000s	sys 0.013s
I20260709 13:00:28.378279  1561 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
W20260709 13:00:28.379218  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 119876us
I20260709 13:00:28.315035  3940 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.355091  3977 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:28.353857  3801 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:00:28.386091  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 54551.9us
W20260709 13:00:28.386209  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 45494.6us
I20260709 13:00:28.354923  3957 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:28.392390  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.392958  1702 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.392998  3899 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:28.405550  3940 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:28.380985  3958 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.410079  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:28.382609  3782 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:28.411227  3977 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:28.411547  3801 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:28.411830  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:28.412532  3801 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.424844  1702 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
W20260709 13:00:28.424116  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 47593.7us
I20260709 13:00:28.381608  3976 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:28.440017  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50186.9us
I20260709 13:00:28.442975  1702 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:28.423823  3977 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:28.444001  1675 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:28.442546  3783 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
W20260709 13:00:28.446808  4030 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:00:28.458357  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:28.459578  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:28.460261  1705 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:28.460391  3816 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:28.460532  3783 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:28.462651  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.464597  1704 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.470046  1707 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:28.474491  1556 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:28.474696  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:28.477380  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:28.478545  3899 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:28.487265  4027 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:28.488523  4027 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:28.488881  4027 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:00:28.491036  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 211451us
I20260709 13:00:28.493587  3825 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:28.493126  2317 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:28.497309  3856 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:28.497622  3915 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:28.498073  3915 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.498418  3915 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.499930  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:28.501122  3852 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:28.501650  3950 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:28.501796  3852 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:28.501668  3998 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:28.502972  3998 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:28.503340  3998 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:28.503211  3852 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.504333  3998 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.501806  3951 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:00:28.506163  3951 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:28.506562  3951 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:28.507572  3565 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.507542  3951 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.509845  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:28.509306  3951 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.510479  1633 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:28.511924  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:28.510043  1632 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:28.516083  1710 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:00:28.502777  3950 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:00:28.521507  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.531167  1559 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:28.534893  2317 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:28.536317  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 78036.2us
I20260709 13:00:28.472062  3783 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:28.461100  3816 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.538502  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:28.519294  3998 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:28.542903  3963 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:28.543202  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:28.550122  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:28.543962  1629 raft_consensus.cc:2410] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:28.503405  3967 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:00:28.541100  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.493235  3940 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:00:28.571511  1704 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:28.539459  3782 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:28.477144  3898 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:28.574205  1554 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:28.556901  3967 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:28.559319  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 82264.9us
I20260709 13:00:28.537096  3899 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:28.575574  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:28.605671  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 46865.4us
I20260709 13:00:28.585182  1559 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:28.558943  3954 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:28.605618  1675 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:28.615505  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:28.616585  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:28.617926  1633 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:28.573760  3963 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:28.577193  3936 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:28.578900  3734 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.601563  3565 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:28.628391  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:28.633396  1634 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:28.612015  3954 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.634296  2349 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:28.636590  1559 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:28.614249  4002 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.641040  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:28.619967  3782 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:28.633085  3856 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:28.639654  3954 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.640849  3970 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:28.704726  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.669096  3921 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.633872  3734 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.712950  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:28.715337  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:28.720136  1636 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:28.720038  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:28.725839  1631 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:28.697602  3856 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:28.727751  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 122281us
I20260709 13:00:28.706776  3970 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.735100  2317 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:28.734113  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:28.738665  1635 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:28.561827  4027 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.742743  3942 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.739328  3963 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.732923  3976 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:28.745357  3958 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:28.745714  3854 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:28.747131  3734 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:28.737630  3970 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.748662  3816 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:28.730880  3856 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.750252  1636 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:28.742779  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:28.767118  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:28.676780  3783 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.696991  4002 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:28.635210  3898 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.740823  3950 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.727569  3940 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:00:28.710947  3967 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.795105  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:28.795764  2345 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:28.796366  4040 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:00:28.727730  3801 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:00:28.786029  3782 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.789724  3943 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:28.790155  3734 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:28.791906  3950 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.714813  3921 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.756313  3852 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:00:28.788271  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:28.805778  1559 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:28.807297  1634 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:28.772066  3874 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:00:28.809130  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:28.817523  1561 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:28.809755  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:28.821959  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:28.816619  3734 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:28.827416  2345 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:28.827193  3734 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.805258  4049 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:00:28.829957  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:28.831288  3950 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:28.831341  3801 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:28.832140  3801 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:28.832480  3874 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:28.833192  3874 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:28.833705  3874 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:28.833105  3801 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.834560  3874 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.835960  3874 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.813055  3921 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:28.837886  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.840085  2318 leader_election.cc:400] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:28.781001  3942 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.844182  1559 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:28.847098  2318 leader_election.cc:400] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:28.846997  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:28.848088  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:28.849753  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:28.773067  3899 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.855883  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.869189  1561 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:28.804695  3993 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:00:28.823866  3957 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.872601  1705 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:28.877997  4002 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:28.879439  3965 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:28.879876  3965 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:28.813611  3915 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.891577  1556 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
W20260709 13:00:28.893033  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 72655.4us
I20260709 13:00:28.895430  1701 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:28.871291  3940 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:28.880795  3965 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.904829  3782 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:28.905546  3993 consensus_queue.cc:1243] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:00:28.906123  3993 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:28.875144  3854 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:28.919530  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.886442  3957 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.912313  3856 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:28.914888  3976 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:28.909437  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:28.922572  1634 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:28.926503  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:28.931114  1627 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:28.935194  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:28.938290  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:28.939399  1705 raft_consensus.cc:2368] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:28.922585  3782 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.940840  3963 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:28.943197  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:28.903158  3940 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:28.943887  1629 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:28.947799  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:28.926340  3993 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.954512  1705 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:28.952894  1707 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:28.954339  3940 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:28.960786  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:28.961063  4050 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:00:28.970023  1627 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:28.972898  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:28.941401  3782 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:28.978753  1561 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:28.982537  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.984318  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:28.984592  1702 raft_consensus.cc:2376] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2. Re-sending same reply.
I20260709 13:00:28.984961  1704 raft_consensus.cc:2376] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4. Re-sending same reply.
I20260709 13:00:28.977586  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.979926  3957 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:28.989902  1674 leader_election.cc:400] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 4.
I20260709 13:00:28.990252  1674 leader_election.cc:403] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:00:28.992144  2347 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:28.981798  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:28.996443  3960 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.833s	user 0.014s	sys 0.004s
I20260709 13:00:28.997818  1552 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:28.998502  1708 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:28.999516  4057 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 13:00:29.000119  4041 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.281s	user 0.002s	sys 0.000s
I20260709 13:00:29.001853  2353 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:29.001646  4060 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:28.975960  3993 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.003949  3970 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:29.041632  1703 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:29.006287  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:29.009208  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:29.011242  1633 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:29.013157  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.013424  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:29.027079  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:29.026893  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:28.987715  3965 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.031586  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:29.048729  1552 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:28.992693  3898 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:29.039122  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.044129  1702 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:29.081259  1558 raft_consensus.cc:2368] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3.
W20260709 13:00:29.082690  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.080835  2348 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
W20260709 13:00:29.038918  4053 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.118s	user 0.001s	sys 0.001s
I20260709 13:00:29.088129  1602 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:29.089403  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:28.990724  4049 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:00:29.032356  4061 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.031929  3942 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:29.091259  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:29.092368  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:29.092438  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:29.193110  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:29.193415  2353 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:29.193110  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:29.200224  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 101879us
I20260709 13:00:29.200657  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:29.084762  3960 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:29.209117  1558 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:29.090512  3963 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:29.214669  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:29.215497  2348 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:00:29.091835  3941 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.256086  4063 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:29.265969  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 56678.7us
W20260709 13:00:29.270179  1601 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.0.107.254:43857, user_credentials={real_user=slave}} blocked reactor thread for 47093us
I20260709 13:00:29.275796  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:29.229455  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:29.200043  3898 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.208871  3951 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:29.286736  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:29.210979  3960 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.217165  3734 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.330350  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:29.325340  3856 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:29.325711  3898 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.326333  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:29.350776  2348 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.352857  1703 raft_consensus.cc:2368] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3.
I20260709 13:00:29.189771  4061 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:29.364593  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:29.375746  1556 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:29.376919  1706 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:29.321936  3951 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.328426  3915 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:29.417096  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:29.328758  3960 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.436868  2348 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
W20260709 13:00:29.354816  3874 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.518s	user 0.010s	sys 0.003s
I20260709 13:00:29.371923  3963 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.333027  3856 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.439169  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:29.447608  1633 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:29.448189  1703 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:00:29.449600  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 459751us
I20260709 13:00:29.452307  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:29.469164  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:29.469379  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:29.319707  3941 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.354307  3734 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.441128  3951 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.479525  2348 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEARNER]: Advancing to term 4
W20260709 13:00:29.480568  2316 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.001s	sys 0.022s
W20260709 13:00:29.481611  2316 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.002s	sys 0.022s
I20260709 13:00:29.491145  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:29.493285  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:29.494154  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.493192  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:29.527431  2348 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:29.527741  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:29.530323  1703 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:29.460050  3856 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.478031  3957 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:29.533212  2353 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:29.484025  3782 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:29.537283  1552 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:29.492218  3970 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:29.550786  1558 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:00:29.563244  3854 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.645s	user 0.007s	sys 0.003s
I20260709 13:00:29.563843  3854 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:29.564728  2345 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:29.564980  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 814) took 1012 ms. Trace:
I20260709 13:00:29.565325  2349 rpcz_store.cc:276] 0709 13:00:28.552900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.553041 (+   141us) service_pool.cc:224] Handling call
0709 13:00:28.553313 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.638449 (+ 85136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.564741 (+926292us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b5400690854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:29.564915 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185850624}
I20260709 13:00:29.539525  3921 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:29.671917  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:29.677207  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 183993us
W20260709 13:00:29.677361  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 210254us
W20260709 13:00:29.678606  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 185489us
I20260709 13:00:29.573101  3993 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:29.679872  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:29.669991  3954 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.941s	user 0.005s	sys 0.006s
I20260709 13:00:29.683393  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2057) took 1157 ms. Trace:
I20260709 13:00:29.684463  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:29.667349  3998 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.668687  3921 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:29.664384  3942 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:00:29.691159  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:29.697448  1703 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:29.695719  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:29.686390  1554 rpcz_store.cc:276] 0709 13:00:28.525020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.534952 (+  9932us) service_pool.cc:224] Handling call
0709 13:00:29.678451 (+1143499us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b54002ff654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:29.678578 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53569792}
I20260709 13:00:29.703745  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:29.694257  3921 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:29.690711  3993 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.696828  3950 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:00:29.709969  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 142971us
I20260709 13:00:29.742030  4089 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.742205  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:29.742791  1703 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:29.743052  4062 log.cc:927] Time spent T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:00:29.742704  4089 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:29.743640  4070 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:00:29.704988  3782 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:29.745162  4089 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:29.745687  3782 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:29.745736  4089 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:29.746446  3782 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.746673  4089 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:29.749925  3856 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:29.737882  3963 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.750799  3963 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.751535  4063 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 4.
I20260709 13:00:29.752784  3963 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:29.742677  4080 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:00:29.760398  4073 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:00:29.766945  3989 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:29.760349  4076 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.770529  1459 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 185) took 5592 ms. Trace:
I20260709 13:00:29.739403  4081 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:29.785905  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:29.772713  3940 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.785s	user 0.002s	sys 0.006s
I20260709 13:00:29.702255  4061 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:29.748575  3951 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.776741  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:29.777738  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:29.857631  1704 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:29.857839  1705 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:00:29.778614  4072 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.294s	user 0.000s	sys 0.000s
I20260709 13:00:29.793315  1602 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:29.796988  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:29.710093  3898 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:29.837910  2347 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.867662  4002 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:29.868122  4002 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:29.690147  3954 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.763013  1703 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:29.700659  3998 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:29.874182  1555 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:29.760946  2344 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:29.783167  3950 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:29.781803  3941 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:29.873554  1704 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:29.796463  3989 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:29.877085  2347 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:29.783041  3734 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:29.798161  3957 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:29.794513  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 90804.6us
W20260709 13:00:29.880280  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 184613us
I20260709 13:00:29.760179  4077 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.779549  4086 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.889519  1703 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:00:29.881107  3965 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.801s	user 0.012s	sys 0.000s
I20260709 13:00:29.799140  3993 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.870440  4091 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.895308  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:29.798313  3958 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:00:29.763113  4084 consensus_queue.cc:1059] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.148s
I20260709 13:00:29.893240  1676 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:29.869009  4002 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:29.862692  3816 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:00:29.856312  3940 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.837378  4076 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:29.909281  1601 leader_election.cc:400] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3.
I20260709 13:00:29.910714  1601 leader_election.cc:403] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:00:29.911576  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2086) took 1079 ms. Trace:
I20260709 13:00:29.911981  1709 rpcz_store.cc:276] 0709 13:00:28.832622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.850849 (+ 18227us) service_pool.cc:224] Handling call
0709 13:00:28.874255 (+ 23406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:28.911586 (+ 37331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:28.920369 (+  8783us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:28.933591 (+ 13222us) log.cc:844] Serialized 26 byte log entry
0709 13:00:29.215530 (+281939us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:29.215571 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.215645 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:29.911358 (+695713us) raft_consensus.cc:1685] finished
0709 13:00:29.911378 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:29.911510 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9020,"raft.queue_time_us":20018,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":267816,"threads_started":2,"wal-append.queue_time_us":457039}
I20260709 13:00:29.899749  4096 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.918884  3989 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.920414  3954 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:29.920593  3989 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.922217  3950 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:29.919504  4095 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:29.908037  3965 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:00:29.928388  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 135176us
I20260709 13:00:29.910638  3943 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:29.914573  4089 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:29.933277  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:29.914201  3734 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:29.941856  1628 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:29.946466  1636 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:29.796546  1459 rpcz_store.cc:276] 0709 13:00:24.178138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.178550 (+   412us) service_pool.cc:224] Handling call
0709 13:00:24.286504 (+107954us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b5400051d10. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b60b2 00007fda2d52fa89 00007fda2d4c7da6 00007fda2d499088 00007fda2d59737e 00007fda2ba582a4 00007fda2ba58232 00007fda2ba58161 00007fda2ba580dc
0709 13:00:29.770311 (+5483807us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:26.721826 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:26.952623 (+230797us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:26.958387 (+  5764us) write_op.cc:435] Acquired schema lock
0709 13:00:26.960016 (+  1629us) tablet.cc:662] Decoding operations
0709 13:00:27.054582 (+ 94566us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:27.064881 (+ 10299us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:27.070135 (+  5254us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:00:27.090504 (+ 20369us) tablet.cc:701] Row locks acquired
0709 13:00:27.091014 (+   510us) write_op.cc:260] PREPARE: finished
0709 13:00:27.092895 (+  1881us) write_op.cc:270] Start()
0709 13:00:27.093015 (+   120us) write_op.cc:276] Timestamp: P: 1783602027092839 usec, L: 0
0709 13:00:27.093036 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:00:27.160104 (+ 67068us) log.cc:844] Serialized 1302 byte log entry
0709 13:00:27.559983 (+399879us) op_driver.cc:464] REPLICATION: finished
0709 13:00:27.821791 (+261808us) write_op.cc:301] APPLY: starting
0709 13:00:27.848466 (+ 26675us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:27.904744 (+ 56278us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:27.904885 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:27.905761 (+   876us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:27.906120 (+   359us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:00:27.906230 (+   110us) write_op.cc:312] APPLY: finished
0709 13:00:27.906606 (+   376us) log.cc:844] Serialized 35 byte log entry
0709 13:00:27.906806 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:27.906923 (+   117us) write_op.cc:454] Released schema lock
0709 13:00:27.906977 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:27.907452 (+   475us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:27.907483 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":68556,"reported_tablets":7,"spinlock_wait_cycles":36322688,"child_traces":[["op",{"apply.queue_time_us":216725,"apply.run_cpu_time_us":3290,"apply.run_wall_time_us":116091,"num_ops":3,"prepare.queue_time_us":715204,"prepare.run_cpu_time_us":28429,"prepare.run_wall_time_us":648882,"raft.queue_time_us":115856,"raft.run_cpu_time_us":23794,"raft.run_wall_time_us":36944,"replication_time_us":466731,"thread_start_us":778350,"threads_started":4,"wal-append.queue_time_us":166982,"wal-append.run_cpu_time_us":21105,"wal-append.run_wall_time_us":1595967}]]}
I20260709 13:00:29.928622  4084 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.953739  4089 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:29.956281  4043 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.236s	user 0.010s	sys 0.005s
I20260709 13:00:29.958894  3940 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:29.961072  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:29.938354  4095 raft_consensus.cc:697] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:29.901372  4086 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:29.901993  3957 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:29.936786  3943 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:30.019521  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.010813  1707 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:29.944914  3734 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:29.944245  3965 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:30.097572  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:30.058871  3960 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.003s
I20260709 13:00:29.906929  4091 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:30.193869  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.198361  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 410936us
W20260709 13:00:30.199779  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 519927us
I20260709 13:00:30.200739  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:30.202208  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 262420us
W20260709 13:00:30.203845  2316 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.003s	sys 0.010s
W20260709 13:00:30.204264  2316 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.003s	sys 0.010s
I20260709 13:00:30.093813  4063 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:30.226397  1633 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:30.090806  3940 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.149s
W20260709 13:00:30.244585  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.237066  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 447316us
I20260709 13:00:30.221187  4099 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.251034  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:30.254180  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:30.234947  3856 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:30.279136  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46315.4us
I20260709 13:00:30.280298  2353 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:30.271440  2345 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:30.272465  4107 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:30.273123  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:30.286413  2352 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:30.244241  3981 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:30.277149  2349 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:30.199827  4087 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.166s
I20260709 13:00:30.242372  3565 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:30.341009  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 70216.2us
I20260709 13:00:30.285225  1636 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:30.245380  3998 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.200522  3960 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:30.225697  3951 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.278992  3856 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.344964  1702 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:30.279786  3915 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:30.323598  4075 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:30.350288  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.323741  4105 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:30.326113  1525 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:30.244011  4063 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.058781  4084 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.328408  1634 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:30.361853  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.263381  3852 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:29.927170  4096 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.295799  3981 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:30.353237  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:30.364217  1636 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:30.255990  3921 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.280025  4078 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3.
I20260709 13:00:30.342218  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:30.372428  1627 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:30.377972  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.350576  3801 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.253086  3943 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.350935  3915 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:30.352195  3998 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.388411  4104 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:00:30.389700  3993 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.398056  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.400717  4116 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:30.401049  3936 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:30.401870  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:30.277056  4099 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.362334  3951 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:30.410512  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 129756us
I20260709 13:00:30.421583  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:30.429350  1634 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:30.411334  1628 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:00:30.430125  1628 pending_rounds.cc:85] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 2
I20260709 13:00:30.430357  1628 pending_rounds.cc:107] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:00:30.430581  1628 raft_consensus.cc:2889] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:30.283972  3734 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.354346  3565 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.406051  3915 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:30.407395  4108 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 13:00:30.404359  3782 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.000s
I20260709 13:00:30.383751  3852 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:30.432205  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.420192  2352 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:30.404657  4115 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.470240  4115 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.472410  4115 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.450608  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:30.473686  1634 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:30.434731  3993 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:30.341426  3965 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.360503  4075 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.418915  4076 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:30.456780  3782 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:30.363129  4105 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.383512  4107 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:00:30.432081  4078 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:30.419478  4116 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.432798  3960 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:30.461418  3957 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.421030  3801 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.463395  3852 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:30.463881  4099 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.433014  3941 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:30.465858  4084 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.467430  3921 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:30.468689  4091 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.354914  4095 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.514612  3917 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.516757  3565 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.469552  3856 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:30.479031  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 153014us
I20260709 13:00:30.479542  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.412957  3998 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.485677  1636 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:30.519016  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1578) took 1490 ms. Trace:
I20260709 13:00:30.519338  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.519971  3998 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.520085  1636 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:30.520716  1634 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:30.456302  3950 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.521418  1705 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:30.521780  3950 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.454581  3954 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:30.523193  3954 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.523630  3954 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:30.491837  4078 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.476687  3993 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:30.524923  3993 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.484622  3936 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:30.493959  4093 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:00:30.549942  4071 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.010s	user 0.013s	sys 0.019s
I20260709 13:00:30.552594  3941 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:30.553073  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2066) took 1634 ms. Trace:
I20260709 13:00:30.499910  4086 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.480561  3951 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.553517  1559 rpcz_store.cc:276] 0709 13:00:28.918331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.918482 (+   151us) service_pool.cc:224] Handling call
0709 13:00:28.918870 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:28.919211 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:28.919239 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:28.919657 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:00:29.342096 (+422439us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:29.342138 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.342219 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:30.549034 (+1206815us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:30.552772 (+  3738us) raft_consensus.cc:1685] finished
0709 13:00:30.552795 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:30.553009 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3429,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7574,"spinlock_wait_cycles":292572416,"thread_start_us":421970,"threads_started":1,"wal-append.queue_time_us":559181}
I20260709 13:00:30.556339  3951 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.558301  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:30.559080  1634 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:30.559816  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.560264  3915 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.560811  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:30.561545  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 43556.3us
I20260709 13:00:30.510481  4113 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.562731  4113 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.507987  4063 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:30.511025  4097 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.358s	user 0.000s	sys 0.001s
I20260709 13:00:30.564940  4113 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:30.510892  4081 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.647s	user 0.007s	sys 0.002s
I20260709 13:00:30.519436  1707 rpcz_store.cc:276] 0709 13:00:29.028834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.275466 (+246632us) service_pool.cc:224] Handling call
0709 13:00:30.518820 (+1243354us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b54002e90d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:30.518923 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080412928}
I20260709 13:00:30.565712  3801 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.513331  4105 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.547959  3936 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.565722  4081 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:30.566896  2347 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:30.567240  1633 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:30.567849  3936 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.521165  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.568890  1636 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:30.562117  4096 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.564976  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 1958) took 1988 ms. Trace:
I20260709 13:00:30.570164  1636 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:30.570665  1636 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:30.570745  4066 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.309s	user 0.009s	sys 0.000s
I20260709 13:00:30.569777  1630 rpcz_store.cc:276] 0709 13:00:28.576838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.611402 (+ 34564us) service_pool.cc:224] Handling call
0709 13:00:29.737637 (+1126235us) spinlock_profiling.cc:243] Waited 5.36 ms on lock 0x7b5800135698. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:00:29.737838 (+   201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:29.763217 (+ 25379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:29.763269 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:29.763973 (+   704us) log.cc:844] Serialized 26 byte log entry
0709 13:00:29.870643 (+106670us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:29.870708 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.870814 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:30.564723 (+693909us) raft_consensus.cc:1685] finished
0709 13:00:30.564744 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:30.564921 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20196,"raft.queue_time_us":26921,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"spinlock_wait_cycles":12336384,"thread_start_us":106080,"threads_started":1,"wal-append.queue_time_us":153281}
I20260709 13:00:30.571388  1636 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.554659  3950 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.571987  1630 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:30.511919  4101 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:00:30.573302  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.573225  3942 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:30.574075  4123 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:30.574329  3942 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:30.561446  1634 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:30.575440  4123 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:30.575773  4123 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:30.579193  4075 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.579767  3941 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.576570  4123 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.582338  4123 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:30.585812  4083 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:30.586229  4083 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:30.587074  4083 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.588160  4083 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:30.562547  3915 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.509740  4116 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.584313  3954 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.572273  1636 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:30.609897  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1609) took 2054 ms. Trace:
I20260709 13:00:30.610298  1555 rpcz_store.cc:276] 0709 13:00:28.555424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.555571 (+   147us) service_pool.cc:224] Handling call
0709 13:00:28.576138 (+ 20567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:29.874702 (+1298564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:30.609676 (+734974us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003048d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:30.609839 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2984687872}
I20260709 13:00:30.626446  4116 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:30.626896  3954 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.578265  3993 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.884346  1633 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:30.779304  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1575) took 1769 ms. Trace:
W20260709 13:00:30.768616  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.007s	sys 0.014s
I20260709 13:00:30.885838  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.778258  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1570) took 1817 ms. Trace:
I20260709 13:00:30.886639  1633 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
W20260709 13:00:30.886238  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.018s	sys 0.017s
W20260709 13:00:30.648267  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.785557  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.772537  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 430322us
W20260709 13:00:30.645615  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.891041  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.892153  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.886756  1708 rpcz_store.cc:276] 0709 13:00:28.961037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.975646 (+ 14609us) service_pool.cc:224] Handling call
0709 13:00:28.982178 (+  6532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:29.017787 (+ 35609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:30.778028 (+1760241us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b5400078354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:30.778195 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37135744}
I20260709 13:00:30.830654  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:30.690495  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.885342  1702 rpcz_store.cc:276] 0709 13:00:29.009975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.192873 (+182898us) service_pool.cc:224] Handling call
0709 13:00:30.778968 (+1586095us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000cb2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:30.779234 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2637149440}
I20260709 13:00:30.649263  1636 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:00:30.779820  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.720587  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:30.893766  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 63223.1us
W20260709 13:00:30.904031  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.578451  4078 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:30.904036  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.649048  4077 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.741s	user 0.010s	sys 0.000s
I20260709 13:00:30.909936  2347 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:30.910503  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:30.915918  4077 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:30.916633  2345 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:30.917042  2348 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:30.917160  3915 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:30.917631  2352 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:30.917068  3942 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:30.719061  4089 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:30.917793  3915 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.918411  4089 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.918895  3942 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:30.919342  4122 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:30.919463  3942 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.920051  3915 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.913831  1636 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:30.912777  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.908246  1634 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:30.788702  3801 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.627024  4116 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.919902  4122 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.935199  3981 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.911926  3816 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.920544  4104 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:30.921025  4089 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.912129  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.936681  3816 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.937098  1633 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:30.938104  2318 leader_election.cc:400] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3.
I20260709 13:00:30.938777  3816 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.938968  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:30.940037  4120 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.940759  3783 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:30.940517  4120 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:30.942688  4090 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:30.905838  3950 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:30.914474  1602 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:30.927469  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.927424  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:30.946154  1634 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:30.946549  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:30.929441  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:30.930804  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:30.903772  4105 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.903710  4081 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.931305  3936 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.917802  3898 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.933869  3942 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.913561  4113 consensus_queue.cc:1059] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:00:30.938032  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.865855  4106 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:30.950161  1631 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:30.936239  4122 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:30.954589  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:30.855731  4099 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 13:00:30.956152  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:30.958406  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:30.914337  4095 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:00:30.960148  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:30.949326  4090 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:30.648864  4100 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:30.961385  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:30.961290  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.979938  1636 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:30.980754  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:30.980973  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.981823  1636 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:30.981932  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:30.982223  1558 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:30.982584  1554 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:30.963266  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:30.942699  4120 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:30.964728  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:30.945087  3950 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.026178  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:30.970160  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:31.027211  1559 raft_consensus.cc:2410] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:00:30.712054  4049 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.906582  1674 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:30.976222  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:30.949393  4089 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:30.974432  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:31.032771  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:31.033327  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 58965.1us
I20260709 13:00:30.936271  3981 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:31.040552  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 75931us
W20260709 13:00:31.022068  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 62007.1us
W20260709 13:00:30.990264  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.016s	sys 0.030s
W20260709 13:00:31.041411  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.030s	sys 0.046s
W20260709 13:00:31.041759  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 808424us
I20260709 13:00:30.962999  4077 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:00:31.042806  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:31.024186  1553 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:30.981379  1555 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:31.025602  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:31.043934  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:30.983665  4122 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:31.029523  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:31.026081  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:31.026849  1557 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:30.946596  3801 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.035086  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:30.960062  4096 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:00:30.967389  3898 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:30.968573  2349 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
W20260709 13:00:31.052731  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 139495us
I20260709 13:00:31.041841  1634 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:31.046721  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:30.964731  3936 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.048467  1560 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:30.981092  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:30.963804  4105 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.030730  3950 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:31.061353  1701 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:31.057602  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:31.033694  4087 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:31.038444  3993 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:31.046459  4107 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:31.042083  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:31.076891  1556 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:31.047319  4120 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:30.977067  4106 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.073612  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:30.964291  4081 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.078289  1552 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:31.079319  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:31.081269  1701 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:31.084649  1555 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:31.088025  2352 raft_consensus.cc:2410] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:31.055198  4099 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:31.090499  1558 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:31.057426  3852 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:31.055421  4089 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:31.056454  4122 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:31.097065  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:31.094030  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52768.8us
W20260709 13:00:31.107667  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 201126us
I20260709 13:00:31.111879  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:31.026242  4100 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.079962  4107 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:31.120196  4107 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:31.120929  4100 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:31.122355  1558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:00:31.123111  1558 raft_consensus.cc:606] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:31.123466  1558 raft_consensus.cc:614] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:31.043265  4049 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.126235  4049 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:31.080268  4120 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.092409  3993 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:31.141790  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:31.140606  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:31.149142  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:31.110263  4135 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.154084  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:31.101339  4087 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:31.084913  3856 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:00:31.192960  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:31.128746  4103 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:31.235641  2349 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:31.150270  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:31.193311  1701 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:31.163640  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:31.134606  4099 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:31.141058  4096 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:00:31.232720  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:31.233189  3943 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:31.206929  1675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.583s (SENT) [suppressed 39 similar messages]
I20260709 13:00:31.151813  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:31.242813  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:31.243151  2351 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:31.247296  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:31.136854  4120 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:31.247910  1701 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:31.262284  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:31.248816  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:31.263219  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.150520  4049 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:31.149326  4129 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:31.245920  2349 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:31.128863  3954 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.245431  3993 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.263173  2347 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:31.235847  4103 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:31.233512  4107 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.241583  4116 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:31.287533  1632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.005s	sys 0.008s
I20260709 13:00:31.290364  4131 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:00:31.292874  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:31.290333  3954 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:31.297825  1632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.005s	sys 0.009s
I20260709 13:00:31.281935  4049 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.284077  4129 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.318681  4129 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:31.296831  4085 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:31.321990  4085 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.322404  4085 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:31.322824  4131 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.323314  4131 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.291229  4122 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:31.305228  4134 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 13:00:31.306182  4027 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:31.329872  4123 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.747s	user 0.006s	sys 0.006s
I20260709 13:00:31.296679  4096 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:31.280903  2345 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
W20260709 13:00:31.338927  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:31.296804  4127 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 13:00:31.196511  4135 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.312639  3898 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:31.313347  3801 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:31.317420  1634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 13:00:31.444572  1634 raft_consensus.cc:606] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:31.445013  1634 raft_consensus.cc:614] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:31.317836  3936 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:31.318222  4049 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:31.108925  4090 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.346105  1703 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:00:31.359239  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 217538us
I20260709 13:00:31.325565  4131 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:31.297523  4138 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:31.342105  4130 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 13:00:31.457134  1601 leader_election.cc:400] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 2 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3.
W20260709 13:00:31.425799  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:31.446141  4083 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.858s	user 0.007s	sys 0.006s
I20260709 13:00:31.475407  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:31.469660  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.480127  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 1967) took 1184 ms. Trace:
I20260709 13:00:31.480715  1628 rpcz_store.cc:276] 0709 13:00:30.296032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.341036 (+ 45004us) service_pool.cc:224] Handling call
0709 13:00:30.351123 (+ 10087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:30.434840 (+ 83717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.479831 (+1044991us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x7b540010c2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:31.480057 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124448640}
I20260709 13:00:31.485920  3801 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.486629  3801 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.490001  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:31.491755  4138 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:31.498023  4136 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 13:00:31.500756  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:31.510522  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:31.455477  3981 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.489152  3801 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:31.286803  4089 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.599547  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:31.643719  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2165) took 1217 ms. Trace:
I20260709 13:00:31.644150  1635 rpcz_store.cc:276] 0709 13:00:30.426042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.426153 (+   111us) service_pool.cc:224] Handling call
0709 13:00:30.449500 (+ 23347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:30.461736 (+ 12236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:30.461769 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:30.462230 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:00:30.738560 (+276330us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:30.738622 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:30.738719 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:31.643506 (+904787us) raft_consensus.cc:1685] finished
0709 13:00:31.643530 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:31.643660 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44340,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":30163,"thread_start_us":275923,"threads_started":1,"wal-append.queue_time_us":470165}
I20260709 13:00:31.645336  1455 catalog_manager.cc:5697] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 4 to 5, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:00:31.646973  1455 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:31.500608  4103 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.649093  1455 catalog_manager.cc:5697] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 2 to 3, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:31.647488  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:31.651227  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:31.653805  4133 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.511s	user 0.005s	sys 0.007s
I20260709 13:00:31.653983  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:31.655133  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:31.656426  1455 catalog_manager.cc:5697] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:00:31.461439  4144 raft_consensus.cc:2955] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:31.658169  1455 catalog_manager.cc:5697] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 13:00:31.659581  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.662087  1455 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 1 to 2, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:31.663686  4135 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:31.659837   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:31.665825   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:31.667821  3993 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:31.651678  4106 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:31.598901  4107 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:31.630193  4105 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:31.676712  1705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.006s	sys 0.037s
I20260709 13:00:31.678306  3856 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:31.689211  1705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.006s	sys 0.037s
I20260709 13:00:31.693259  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:31.532876  3981 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.699064  1706 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:31.699999  1705 raft_consensus.cc:2410] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:31.518992  4099 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.454110  4139 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.631652  4149 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:31.704159  3967 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:31.715812  4148 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:31.714715  4149 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:31.699738  4075 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.705993  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 412) took 2703 ms. Trace:
I20260709 13:00:31.734848  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 1974) took 1337 ms. Trace:
I20260709 13:00:31.735257  1627 rpcz_store.cc:276] 0709 13:00:30.397593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.407932 (+ 10339us) service_pool.cc:224] Handling call
0709 13:00:30.421324 (+ 13392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:30.450098 (+ 28774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:30.450127 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:30.454069 (+  3942us) log.cc:844] Serialized 26 byte log entry
0709 13:00:30.505086 (+ 51017us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:30.505248 (+   162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.723963 (+1218715us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b54003b83d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:31.724030 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:31.724099 (+    69us) raft_consensus.cc:1685] finished
0709 13:00:31.724117 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:31.734787 (+ 10670us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45183,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":7892,"spinlock_wait_cycles":65424640,"thread_start_us":41691,"threads_started":1,"wal-append.queue_time_us":327753}
I20260709 13:00:31.703321  3856 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.764995  4112 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:31.732197  4075 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.717041  4139 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.770925  4149 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.775550  4105 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:31.776322  4105 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.783094  3954 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:31.797107  4100 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.053s	sys 0.086s
W20260709 13:00:31.798028  1704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.003s	sys 0.011s
W20260709 13:00:31.798548  1675 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.000s	sys 0.014s
W20260709 13:00:31.798097  4100 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.053s	sys 0.086s
W20260709 13:00:31.800366  1704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.003s	sys 0.012s
I20260709 13:00:31.733011  2344 rpcz_store.cc:276] 0709 13:00:28.972942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.984556 (+ 11614us) service_pool.cc:224] Handling call
0709 13:00:29.083205 (+ 98649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:29.885684 (+802479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.634599 (+1748915us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b5400762d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:31.676735 (+ 42136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1416421248}
W20260709 13:00:31.803268  1675 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.000s	sys 0.014s
W20260709 13:00:31.816880  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 609878us
W20260709 13:00:31.836735  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.007s	sys 0.012s
W20260709 13:00:31.837323  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.007s	sys 0.012s
W20260709 13:00:31.839633  1600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.003s	sys 0.016s
W20260709 13:00:31.840349  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 725819us
W20260709 13:00:31.840031  1600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.003s	sys 0.016s
W20260709 13:00:31.842428  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.860268  1674 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:31.830560  4049 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.873049  4159 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.878588  4139 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:31.865461  4105 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:31.885077  1676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.000s	sys 0.042s
W20260709 13:00:31.885510  1676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.000s	sys 0.042s
W20260709 13:00:31.879673  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 369226us
I20260709 13:00:31.899508  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:31.917513  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:31.918537  2352 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:31.875726  4159 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.921437  4164 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:31.922051  4164 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:31.923313  4159 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:31.923133  4164 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:31.879238  4157 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:31.914750  3950 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.836s	user 0.005s	sys 0.005s
I20260709 13:00:31.910270  4077 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:31.926086  4152 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:00:31.937124  1633 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:00:31.938468  3950 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.939275  3951 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:31.939644  3951 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:31.940729  3950 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:31.940407  3951 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.941262  4077 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.942239  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 451) took 1191 ms. Trace:
I20260709 13:00:31.942595  1630 rpcz_store.cc:276] 0709 13:00:30.750946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.751100 (+   154us) service_pool.cc:224] Handling call
0709 13:00:30.789247 (+ 38147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:30.919820 (+130573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:30.919851 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:30.919890 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:30.919922 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.941998 (+1022076us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540017b454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:31.942057 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:31.942184 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299869440}
I20260709 13:00:31.943413  4077 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:31.926959  3856 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:31.909667  4148 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:00:31.945003  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:31.945742  2344 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:31.945314  3954 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:31.946878  4151 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 13:00:31.944591  4085 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.622s	user 0.008s	sys 0.003s
I20260709 13:00:31.950547  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 681) took 1676 ms. Trace:
I20260709 13:00:31.952091  2348 rpcz_store.cc:276] 0709 13:00:30.271155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.271264 (+   109us) service_pool.cc:224] Handling call
0709 13:00:30.271522 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:30.917458 (+645936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.947382 (+1029924us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54004ecad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:31.947562 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484140288}
I20260709 13:00:31.960012  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:31.968571  4126 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.935s	user 0.007s	sys 0.004s
I20260709 13:00:31.971035  1709 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:31.972940  1709 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:31.973991  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 1976) took 1572 ms. Trace:
I20260709 13:00:31.958959  4085 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.974550  1629 rpcz_store.cc:276] 0709 13:00:30.401385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.401506 (+   121us) service_pool.cc:224] Handling call
0709 13:00:30.401817 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:30.410208 (+  8391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:30.410258 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:30.436633 (+ 26375us) log.cc:844] Serialized 26 byte log entry
0709 13:00:30.557905 (+121272us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:30.557944 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:30.558007 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:31.968260 (+1410253us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b240029a2d8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda244739c4 00007fda24471acb 00007fda2462cdc9 00007fda2462ca55 00007fda2462c09d 00007fda2725e10e 00007fda2727588c 00007fda27274bb6 00007fda272747d8 00007fda2727e221
0709 13:00:31.973622 (+  5362us) raft_consensus.cc:1685] finished
0709 13:00:31.973650 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:31.973914 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18336,"raft.queue_time_us":8291,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":18023,"spinlock_wait_cycles":17326976,"thread_start_us":42935,"threads_started":1,"wal-append.queue_time_us":494519}
I20260709 13:00:31.976714  1636 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:00:31.977125  1636 pending_rounds.cc:85] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 6
I20260709 13:00:31.977488  1636 pending_rounds.cc:107] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:00:31.977823  1707 raft_consensus.cc:2393] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:31.978158  4138 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:31.977850  1636 raft_consensus.cc:2889] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:31.977025  4085 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:31.982124  4124 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.956261  3951 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:31.947290  3954 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:31.987519  3954 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:31.926163  4157 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.989192  4049 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:31.983376  3852 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:31.982825  4124 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:31.984848  4100 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.933773  4099 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:00:31.999933  1709 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:31.988543  4120 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.701s	user 0.007s	sys 0.000s
I20260709 13:00:32.001801  4100 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:31.990330  4131 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:32.011143  4123 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.011540  4104 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.011612  1527 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:31.998485  3852 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:32.013000  3954 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.025875  4120 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.015106  3856 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:32.032054  3954 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.032349  3856 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:32.033659  4129 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.714s	user 0.009s	sys 0.003s
I20260709 13:00:32.034162  3954 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:32.016125  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2069) took 1046 ms. Trace:
W20260709 13:00:32.018488  4027 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.001s
I20260709 13:00:32.020167  4081 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:32.032629  4100 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:32.030113  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:32.035753  1561 rpcz_store.cc:276] 0709 13:00:30.969261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.969379 (+   118us) service_pool.cc:224] Handling call
0709 13:00:32.015940 (+1046561us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b54001c0054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:32.016069 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466368384}
I20260709 13:00:32.028564  4104 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.012977  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:31.954169  4089 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:00:32.035154  4106 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:00:32.043684  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:32.030943  4122 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:32.041138  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.047725  1707 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:32.027150  3942 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:00:32.048918  3941 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:32.049747  3954 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:32.043320  4100 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:32.050868  4100 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.051230  4100 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:32.053987  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.044863  1527 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:32.055231  1707 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:32.056710  3941 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:32.045759  4104 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:32.046423  2350 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:32.057914  4025 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.024384  3967 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:32.047516  4122 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:32.058382  4025 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.064054  4122 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:32.064109  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.065670  2346 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:32.062953  4104 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.068813  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:32.069059  4104 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.070694  4113 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:32.071405  4113 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:32.072307  4113 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.050390  4157 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:32.011608  3943 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:32.073791  4104 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:32.073897  3943 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.076099  3943 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:32.054497  3981 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:32.030637  4123 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.060770  1552 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:00:32.076921  1552 pending_rounds.cc:85] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 5
I20260709 13:00:32.077171  1552 pending_rounds.cc:107] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:00:32.077394  1552 raft_consensus.cc:2889] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:32.078069  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.078267  4123 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:32.060948  2348 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:32.061864  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2180) took 1154 ms. Trace:
I20260709 13:00:32.079795  1633 rpcz_store.cc:276] 0709 13:00:30.907788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.038713 (+130925us) service_pool.cc:224] Handling call
0709 13:00:31.051539 (+ 12826us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:31.937770 (+886231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.061269 (+123499us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b54002dbed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:32.061792 (+   523us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2035799936}
I20260709 13:00:32.067847  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:32.034518  3856 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:32.073271  4113 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:32.097054  4089 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:32.099258  4089 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:32.100338  4089 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.102732  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.104151  4120 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:32.108955  4002 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.110196  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:32.110347  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.111138  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:32.111435  3943 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:32.111873  3943 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.112253  3943 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:32.112843  3942 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:32.113255  3942 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.113632  3942 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:32.113806  3852 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.116039  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:32.116763  2352 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:32.104540  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.063232  3967 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:32.104984  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.117975  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.118561  1556 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:32.034498  4129 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.119480  1557 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:32.119917  1703 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:32.120112  3852 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:32.106407  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.122076  3941 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:32.122753  1558 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:32.107151  4089 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:32.107643  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.125445  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.124060  1555 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:32.134461  4129 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:32.124598  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.135533  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:32.137571  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2103) took 1045 ms. Trace:
I20260709 13:00:32.118552  3967 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.139179  3967 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:32.139276  2349 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:32.137943  1704 rpcz_store.cc:276] 0709 13:00:31.092341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.092542 (+   201us) service_pool.cc:224] Handling call
0709 13:00:31.132439 (+ 39897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.132650 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:31.249870 (+117220us) log.cc:844] Serialized 14 byte log entry
0709 13:00:31.811053 (+561183us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.051677 (+240624us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b180040dbc0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1
0709 13:00:32.053148 (+  1471us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b180040dbc0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:00:32.137024 (+ 83876us) spinlock_profiling.cc:243] Waited 6.67 ms on lock 0x7b180040dbc0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:00:32.137180 (+   156us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:32.137240 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:32.137310 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.137375 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.137521 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6385,"prepare.queue_time_us":202889,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":84,"raft.queue_time_us":5872,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3630,"spinlock_wait_cycles":42043648,"thread_start_us":653287,"threads_started":2,"wal-append.queue_time_us":583393}
I20260709 13:00:32.141127  2351 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:32.144940  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 294104us
I20260709 13:00:32.145219  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:32.146085  2348 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:32.147215  3989 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.146127  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:32.147689  3989 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.149075  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:32.151158  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:32.152585  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.154243  1602 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:32.156037  3799 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:32.157847  4175 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.158284  4175 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.158670  4175 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:32.160995  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:32.163007  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:32.154320  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:32.164837  2348 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:32.166605  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.167296  1702 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:32.168170  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.168695  1702 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:32.168823  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.170158  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.171859  1709 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:32.173430  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.173754  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:32.174408  3989 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:32.174472  1552 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:32.176147  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:32.177126  1557 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:32.180351  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.147010  3993 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:32.181667  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:32.189819  1702 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:32.182623  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:32.191118  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:32.183276  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.180395  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:32.192170  1555 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:32.183884  4049 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.179131  4095 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:32.192806  4049 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.179404  3940 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.193218  4095 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.193778  4095 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.166821  3898 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:00:32.155069  3917 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.185904  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.180182  3936 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:00:32.193011  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.194630  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 456) took 1211 ms. Trace:
I20260709 13:00:32.191864  3993 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.196357  1704 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:32.219457  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.193638  3940 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.201802  3799 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:32.195708  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.289325  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:32.275767  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.227423  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:32.271025  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:32.228288  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.266867  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:32.311244  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:32.312147  2348 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:32.308048  1703 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:32.307624  1709 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
W20260709 13:00:32.317214  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.193267  4049 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:32.306250  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.325048  1555 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:32.309083  1602 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:32.320639  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:32.311048  1634 rpcz_store.cc:276] 0709 13:00:30.983244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.494935 (+511691us) service_pool.cc:224] Handling call
0709 13:00:31.495333 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:31.915503 (+420170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:31.915543 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:31.916042 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:00:31.964220 (+ 48178us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:31.964746 (+   526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.964968 (+   222us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:32.194398 (+229430us) raft_consensus.cc:1685] finished
0709 13:00:32.194417 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.194572 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15204,"raft.queue_time_us":209265,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":422706,"threads_started":2,"wal-append.queue_time_us":173409}
I20260709 13:00:32.188104  4138 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
W20260709 13:00:32.342913  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 191835us
I20260709 13:00:32.345549  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:32.345822  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 182852us
I20260709 13:00:32.357599  1702 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:32.368557  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:32.372783  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:32.377560  4157 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:32.377974  4157 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:32.376804  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 67763.7us
I20260709 13:00:32.378890  4077 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.379794  4077 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.379892  4131 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.380221  4077 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:32.380786  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7469757179999874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.193s for a response
W20260709 13:00:32.377144  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.260088  4001 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.381110  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:32.388420  1601 connection.cc:256] RPC call timeout handler was delayed by -1.2519757179999829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.248s for a response
I20260709 13:00:32.378324  4157 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:32.214689  4175 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.314090  3993 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:32.378326  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:32.350121  3989 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:32.390345  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:32.394403  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48890.1us
I20260709 13:00:32.395185  1556 raft_consensus.cc:2410] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:32.380443  1634 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:32.334950  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:32.396265  2318 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:00:32.374501  4170 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:00:32.357076  3856 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.289637  4122 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.219071  3917 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.350847  2346 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:32.402501  4142 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:32.406129  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.425467  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:32.425639  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:32.448086  1556 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:32.418488  3856 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.424535  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.357743  3963 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:00:32.389838  3965 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:32.432379  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:32.432999  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.432384  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.388294  3951 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:32.428100  2344 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:32.460923  4069 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:32.344293  3852 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:00:32.469805  3951 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.470759  3940 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:32.380383  4131 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.472054  3951 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:32.472944  4131 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:32.437194  4105 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:32.433593  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.444710  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:32.477015  1557 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:32.445698  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:32.403844  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.479121  1704 raft_consensus.cc:2410] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:32.461098  1629 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:32.461609  1554 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:32.433820  4081 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:32.434813  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.463677  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:32.464867  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.476441  1627 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:32.394012  4001 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.466967  3965 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.468106  3943 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.451889  3989 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:32.468573  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:32.434137  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 65700us
I20260709 13:00:32.469553  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:32.437126  4087 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.460222  1634 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:32.480568  1635 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:32.465842  3936 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:32.481398  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.436409  4162 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:00:32.486760  1708 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:32.487639  4075 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:32.469234  3942 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.459131  4142 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.734479  1631 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:32.735172  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.736164  1632 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:32.460363  3856 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:32.458211  3981 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.487821  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.733856  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:32.743201  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:32.733086  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 231958us
I20260709 13:00:32.736658  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.744397  1627 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:32.728693  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.747224  1630 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:32.699154  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:32.697170  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.461088  4120 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:32.768620  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.752166  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:32.752635  1634 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:32.774189  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.775048  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:32.491128  4063 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:32.766470  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.480213  4060 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.780562  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.480039  4105 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.800607  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.807613  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:32.807914  2344 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:32.488979  3917 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:32.808604  1559 raft_consensus.cc:2393] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:32.488099  3783 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.818928  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:32.729058  4122 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:32.812947  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.814278  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:32.815181  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.744112  4087 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.700286  3940 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.810695  1556 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:32.826282  1525 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:32.787173  4001 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:32.803773  4105 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.835282  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:32.839895  4144 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:32.842168  4183 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.405s	user 0.000s	sys 0.001s
I20260709 13:00:32.842087  1525 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:32.767568  4140 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.772320  4075 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.837314  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.833240  1552 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:32.834132  4087 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:32.838587  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 95478.9us
W20260709 13:00:32.838891  4181 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.376s	user 0.000s	sys 0.001s
I20260709 13:00:32.833907  3783 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.839316  4063 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:32.839586  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 87489.8us
I20260709 13:00:32.866838  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:32.695932  4002 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.128s
I20260709 13:00:32.867137  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.867977  1552 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:32.727514  4161 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.375s
I20260709 13:00:32.819623  4131 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.860848  1556 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:32.858771  3957 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.870963  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.864712  4175 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:32.863286  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:32.872479  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:32.836349  4103 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:32.801059  4120 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:32.840983  3967 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.701s	user 0.010s	sys 0.000s
I20260709 13:00:32.872479  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:32.929339  4100 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.878s	user 0.011s	sys 0.000s
I20260709 13:00:32.913986  3957 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.930245  4100 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.931103  3783 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:32.932197  4100 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:32.933674  3936 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:32.934897  4186 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:00:32.931753  3957 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:32.841194  4162 consensus_queue.cc:1243] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:00:32.808825  4060 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.870934  4063 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:32.993938  1705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.023s	sys 0.148s
W20260709 13:00:32.936738  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 64322.3us
W20260709 13:00:32.998533  1705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.024s	sys 0.150s
I20260709 13:00:32.859535  3940 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:32.860618  4075 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:32.940441  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.026s	sys 0.027s
I20260709 13:00:32.878795  1632 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
W20260709 13:00:33.027645  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.028s	sys 0.027s
W20260709 13:00:33.028075  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 680496us
W20260709 13:00:32.861513  4113 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.738s	user 0.000s	sys 0.011s
I20260709 13:00:32.940007  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.914808  4144 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:33.036332  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:32.884860  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:32.881321  3989 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.881390  4001 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:32.868884  4122 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.137s
I20260709 13:00:33.040692  1704 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 2.
I20260709 13:00:33.040311  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:32.999619  4063 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:33.049924  1556 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:00:33.044169  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.051612  1632 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
W20260709 13:00:33.055851  1557 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.574s	user 0.009s	sys 0.000s
I20260709 13:00:33.049459  1629 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:32.880888  3852 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:33.048163  4191 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 13:00:32.932808  4100 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.117s
I20260709 13:00:33.029037  4075 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:32.941519  4169 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.019s	sys 0.080s
I20260709 13:00:33.060137  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:33.060361  1557 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:33.060983  4113 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:33.061452  3852 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.084154  4060 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:33.079011  4169 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.032s	sys 0.090s
W20260709 13:00:33.090229  4089 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.948s	user 0.008s	sys 0.001s
I20260709 13:00:33.072818  3917 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:33.090946  3917 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:33.091346  3917 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:33.094913  3852 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.096366  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:33.097565  1556 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:00:33.097468  4049 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.735s	user 0.007s	sys 0.000s
I20260709 13:00:33.101752  4100 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:33.059264  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:33.046090  3942 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.109170  3942 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:33.077432  3943 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.109376  1703 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:33.109655  3942 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:33.110097  3942 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:33.110865  1703 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:00:33.111214  1703 pending_rounds.cc:85] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 5
I20260709 13:00:33.111449  1703 pending_rounds.cc:107] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:00:33.111506  3943 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:33.054056  3936 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:33.112511  3936 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.112944  3936 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:33.113363  3943 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:00:33.114837  4167 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.070s	user 0.015s	sys 0.018s
I20260709 13:00:32.951594  3783 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:33.045898  4025 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:32.938892  4140 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:32.999856  4115 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:00:33.048826  4189 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:33.041693  4144 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:33.068838  3954 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.012s	sys 0.054s
I20260709 13:00:33.082832  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.131162  1632 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:00:33.102160  1556 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:33.102432  4049 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.107935  3965 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:33.111807  1703 raft_consensus.cc:2889] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:33.109269  1552 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:32.935843  4131 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:32.940335  4135 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.023s	sys 0.056s
I20260709 13:00:33.118357  4189 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:33.128535  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:33.130766  3954 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.013s	sys 0.055s
I20260709 13:00:33.143401  4185 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:33.126735  4144 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:33.158070  1561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.052s	sys 0.105s
W20260709 13:00:33.234676  1561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.053s	sys 0.106s
I20260709 13:00:33.235653  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 308) took 1154 ms. Trace:
W20260709 13:00:33.159408  2348 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.002s	sys 0.001s
W20260709 13:00:33.238179  2348 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.002s	sys 0.001s
I20260709 13:00:33.236096  1561 rpcz_store.cc:276] 0709 13:00:32.080973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.096519 (+ 15546us) service_pool.cc:224] Handling call
0709 13:00:32.096909 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:32.097108 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:32.097400 (+   292us) log.cc:844] Serialized 14 byte log entry
0709 13:00:33.235188 (+1137788us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:33.235261 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:33.235320 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.235407 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.235584 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4926,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":200835,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":167493,"thread_start_us":1137271,"threads_started":2,"wal-append.queue_time_us":285262}
I20260709 13:00:33.108734  4100 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.099136  4113 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:33.146456  3951 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.243870  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 403) took 1101 ms. Trace:
I20260709 13:00:33.250671  1703 rpcz_store.cc:276] 0709 13:00:32.142236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.110328 (+968092us) service_pool.cc:224] Handling call
0709 13:00:33.110585 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.243535 (+132950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.243793 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:33.271049  1601 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.007s	sys 0.010s
W20260709 13:00:33.271581  1601 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.007s	sys 0.010s
W20260709 13:00:33.271919  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 882399us
W20260709 13:00:33.273002  1633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.817s	user 0.001s	sys 0.017s
W20260709 13:00:33.273506  1633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.001s	sys 0.018s
W20260709 13:00:33.274044  1558 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.011s	user 0.000s	sys 0.016s
W20260709 13:00:33.274461  1558 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.000s	sys 0.016s
W20260709 13:00:33.278234  1527 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.017s	sys 0.058s
W20260709 13:00:33.278708  1527 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.018s	sys 0.058s
W20260709 13:00:33.279085  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 914503us
I20260709 13:00:33.286152  1561 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:00:33.229333  4135 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.030s	sys 0.063s
W20260709 13:00:33.286083  4095 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.068s	user 0.002s	sys 0.008s
I20260709 13:00:33.228850  4185 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:33.234436  3943 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:00:33.250075  4104 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:33.228515  4189 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:33.378151  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 97914.1us
I20260709 13:00:33.345985  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.392530  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2101) took 2308 ms. Trace:
I20260709 13:00:33.406893  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2107) took 2257 ms. Trace:
I20260709 13:00:33.252040  4100 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:33.396381  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 480277us
W20260709 13:00:33.374004  1631 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.000s
W20260709 13:00:33.410272  2350 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 425) took 2255 ms (client timeout 2843 ms). Trace:
I20260709 13:00:33.412362  1631 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:33.413040  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2189) took 1012 ms. Trace:
W20260709 13:00:33.396915  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.413672  1631 rpcz_store.cc:276] 0709 13:00:32.400615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.468322 (+ 67707us) service_pool.cc:224] Handling call
0709 13:00:33.412983 (+944661us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.275276  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2086) took 1102 ms. Trace:
I20260709 13:00:33.406656  1632 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:33.406647  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:33.412762  2350 rpcz_store.cc:269] 0709 13:00:31.150502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.150614 (+   112us) service_pool.cc:224] Handling call
0709 13:00:33.405422 (+2254808us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b540049b454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:33.405601 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1886800640}
I20260709 13:00:33.407320  1705 rpcz_store.cc:276] 0709 13:00:31.149669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.704677 (+555008us) service_pool.cc:224] Handling call
0709 13:00:31.705044 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:31.705628 (+   584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:31.705666 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:31.715001 (+  9335us) log.cc:844] Serialized 26 byte log entry
0709 13:00:32.191505 (+476504us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b180040dd40. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:00:32.999093 (+807588us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:33.000009 (+   916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.000105 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:33.384074 (+383969us) raft_consensus.cc:1685] finished
0709 13:00:33.384096 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.406833 (+ 22737us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":10430,"spinlock_wait_cycles":8504192,"thread_start_us":807343,"threads_started":1,"wal-append.queue_time_us":1428377}
I20260709 13:00:33.407051  1710 rpcz_store.cc:276] 0709 13:00:31.084051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.084218 (+   167us) service_pool.cc:224] Handling call
0709 13:00:31.114557 (+ 30339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.304910 (+190353us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:31.674084 (+369174us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.165987 (+491903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:32.166068 (+    81us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:32.166136 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.392154 (+1226018us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400224f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:33.392220 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.392402 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":218160,"prepare.queue_time_us":25791,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":256627,"spinlock_wait_cycles":445000448,"thread_start_us":600936,"threads_started":2,"wal-append.queue_time_us":357259,"wal-append.run_cpu_time_us":1345,"wal-append.run_wall_time_us":1092789}
I20260709 13:00:33.390297  1552 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:00:33.331876  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 489844us
I20260709 13:00:33.416839  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:33.425845  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:33.432410  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:33.434576  1705 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:33.437463  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:33.423588  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:33.454865  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:00:33.406334  4087 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.542s	user 0.003s	sys 0.007s
I20260709 13:00:33.492268  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:33.495254  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 839) took 1134 ms. Trace:
I20260709 13:00:33.498355  2348 rpcz_store.cc:276] 0709 13:00:32.360659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.369625 (+  8966us) service_pool.cc:224] Handling call
0709 13:00:32.370045 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:32.386557 (+ 16512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:32.386619 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:32.387516 (+   897us) log.cc:844] Serialized 26 byte log entry
0709 13:00:33.238638 (+851122us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:33.238692 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.238779 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:33.494967 (+256188us) raft_consensus.cc:1685] finished
0709 13:00:33.494994 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.495183 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4622,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":8885,"thread_start_us":803429,"threads_started":1,"wal-append.queue_time_us":1086482}
I20260709 13:00:33.386016  4120 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.406040  4060 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:33.386022  4079 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:33.501199  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 75448.2us
I20260709 13:00:33.299254  3941 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.171s
I20260709 13:00:33.463635  1527 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:33.506443  3856 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.715s	user 0.020s	sys 0.000s
I20260709 13:00:33.415951  1558 rpcz_store.cc:276] 0709 13:00:32.173261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.173380 (+   119us) service_pool.cc:224] Handling call
0709 13:00:32.186954 (+ 13574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:32.187139 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:32.187466 (+   327us) log.cc:844] Serialized 14 byte log entry
0709 13:00:33.274836 (+1087370us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:33.274907 (+    71us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:33.274964 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.275050 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.275210 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":209,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":183056,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":14132,"spinlock_wait_cycles":1920,"thread_start_us":1086864,"threads_started":2,"wal-append.queue_time_us":188162}
I20260709 13:00:33.502832  4049 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:33.508634  3942 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:33.513996  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:33.410137  3967 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:33.494983  4175 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:33.522269  4001 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.504390  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:33.457060  4188 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:33.416649  4075 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:33.555117  4157 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.149s	user 0.010s	sys 0.001s
I20260709 13:00:33.548226  3954 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:33.497345  3936 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.590262  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.591328  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 1997) took 1155 ms. Trace:
I20260709 13:00:33.591763  1634 rpcz_store.cc:276] 0709 13:00:32.436171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.589943 (+1153772us) service_pool.cc:224] Handling call
0709 13:00:33.591159 (+  1216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.593080  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.594051  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:33.595038  1710 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:33.596884  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 1978) took 2630 ms. Trace:
I20260709 13:00:33.556720  4198 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:33.568046  4113 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.549342  1632 raft_consensus.cc:2368] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 3. Current term is 4.
I20260709 13:00:33.285499  3951 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.544929  2317 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:33.394150  3981 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:33.565798  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1581) took 4114 ms (client timeout 1930 ms). Trace:
I20260709 13:00:33.556979  3967 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:33.560461  4175 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:33.569794  2318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.021s	sys 0.064s
W20260709 13:00:33.659858  2318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.021s	sys 0.064s
I20260709 13:00:33.580543  4195 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:33.550027  4120 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.526328  4197 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:33.627683  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.654526  3874 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:33.499490  4087 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.623479  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2095) took 2563 ms. Trace:
I20260709 13:00:33.626453  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2190) took 1219 ms. Trace:
I20260709 13:00:33.665297  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.665441  1629 rpcz_store.cc:276] 0709 13:00:32.407319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.479904 (+ 72585us) service_pool.cc:224] Handling call
0709 13:00:33.626247 (+1146343us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54003cdcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:33.626383 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691573120}
I20260709 13:00:33.665949  1632 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:00:33.597270  1636 rpcz_store.cc:276] 0709 13:00:30.959123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.044769 (+ 85646us) service_pool.cc:224] Handling call
0709 13:00:31.045041 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.980210 (+935169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.589212 (+1609002us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b5400196c54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:33.589431 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141849344}
I20260709 13:00:33.666455  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2187) took 1266 ms. Trace:
I20260709 13:00:33.621574   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:33.667055  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.668138  1636 raft_consensus.cc:2410] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:33.665324  1708 rpcz_store.cc:276] 0709 13:00:31.059961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.060075 (+   114us) service_pool.cc:224] Handling call
0709 13:00:31.078156 (+ 18081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:32.734307 (+1656151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.623149 (+888842us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400224554. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:33.623398 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3238598400}
I20260709 13:00:33.667012  2348 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:33.620417  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.625591  1557 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:33.670604  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:33.670972  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:33.671260  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 1992) took 1272 ms. Trace:
I20260709 13:00:33.628427  4198 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:33.666765  1632 rpcz_store.cc:276] 0709 13:00:32.399969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.642805 (+1242836us) service_pool.cc:224] Handling call
0709 13:00:33.666404 (+ 23599us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.671648  1636 rpcz_store.cc:276] 0709 13:00:32.400421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.666864 (+1266443us) service_pool.cc:224] Handling call
0709 13:00:33.671206 (+  4342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.579635  4157 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.417542  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 1988) took 1026 ms. Trace:
W20260709 13:00:33.646905  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.049s	sys 0.086s
W20260709 13:00:33.565037  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 101470us
I20260709 13:00:33.553098  4079 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.552358  3856 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.548695  4060 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:33.316977  3970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.925s	user 0.003s	sys 0.009s
W20260709 13:00:33.650417  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 105577us
W20260709 13:00:33.651697  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.020s	sys 0.097s
W20260709 13:00:33.599136  4077 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.180s	user 0.010s	sys 0.001s
I20260709 13:00:33.559127  4185 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:33.619591  4100 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:33.653431  3981 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:33.406024  4095 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.563568  4001 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.658046  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 1983) took 2691 ms. Trace:
W20260709 13:00:33.653815  1706 rpcz_store.cc:269] 0709 13:00:29.444692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.444820 (+   128us) service_pool.cc:224] Handling call
0709 13:00:33.559141 (+4114321us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b540014c8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:33.559248 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4574385024}
W20260709 13:00:33.641877  2319 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.045s	sys 0.055s
W20260709 13:00:33.562738  3993 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.134s	user 0.006s	sys 0.006s
I20260709 13:00:33.672612  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:33.405807  4125 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:33.674707  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.049s	sys 0.086s
I20260709 13:00:33.675249  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:33.734942  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 683385us
I20260709 13:00:33.735076  2349 raft_consensus.cc:2376] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3. Re-sending same reply.
I20260709 13:00:33.674981  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:33.646070  4049 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:33.675603  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:33.659890  4131 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.661652  3965 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:33.681726  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.693881  1556 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:33.680125  2352 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEARNER]: Advancing to term 6
W20260709 13:00:33.689522  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2091) took 1470 ms (client timeout 1583 ms). Trace:
W20260709 13:00:33.744625  1559 rpcz_store.cc:269] 0709 13:00:32.219057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.219204 (+   147us) service_pool.cc:224] Handling call
0709 13:00:33.689289 (+1470085us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54001fd454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:33.689442 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147317504}
I20260709 13:00:33.693408  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:33.678737  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.021s	sys 0.097s
I20260709 13:00:33.681130  1705 raft_consensus.cc:2410] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:33.746843  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:33.747102  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:33.746407  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 873914us
I20260709 13:00:33.746381  1634 raft_consensus.cc:2410] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:00:33.749571  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:33.750734  1675 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:33.752202  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:33.752624  2349 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:33.753350  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:33.690189  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 315) took 1581 ms (client timeout 2039 ms). Trace:
I20260709 13:00:33.753705  1553 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:33.662937  3989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.067s	sys 0.088s
I20260709 13:00:33.717319  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.758545  1632 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:33.717707  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:33.669747  4197 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.719255  4060 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:33.719851  4075 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:33.721103  3917 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.000s
I20260709 13:00:33.721154  3936 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:33.721752  4077 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.766723  1675 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:33.767279  4060 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:33.767534  4200 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:33.767927  4142 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:33.768451  4142 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:33.768527  4077 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.674059  4198 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:33.769376  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:33.770237  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1626) took 1432 ms. Trace:
I20260709 13:00:33.770596  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:33.770823  1557 rpcz_store.cc:276] 0709 13:00:32.337637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.093163 (+755526us) service_pool.cc:224] Handling call
0709 13:00:33.095796 (+  2633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.669543 (+573747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.770000 (+100457us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54002ff654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:33.770178 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217681664}
I20260709 13:00:33.771401  4200 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:33.771947  3965 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:33.772765  3965 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:33.772944  4149 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:33.770296  4085 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:33.774015  4085 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.774361  4085 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:33.774712  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:33.775556  4142 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:33.776225  4142 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:33.777148  4142 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:33.777963  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:33.778950  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.722477  4175 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:33.722756  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:33.781823  1709 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:33.723089  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:33.692176  3856 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:33.723918  4063 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.654s	user 0.002s	sys 0.007s
I20260709 13:00:33.679608  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:33.669996  3874 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:33.680490  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:33.724345  4105 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.858s	user 0.018s	sys 0.006s
I20260709 13:00:33.692842  2345 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:33.724907  4087 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.830360  4140 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:33.830626  4198 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:33.831140  4085 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:33.725152  4185 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:33.832825  4198 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.783325  3942 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.794106  4063 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.673208  4120 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.835909  4063 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:33.793849  3963 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:00:33.837006  4120 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:33.727213  4100 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.838047  2349 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:33.838774  4193 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:00:33.839598  3967 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.662866  4195 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.842280  4195 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:33.842753  4095 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:33.833179  4085 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:33.727017  4144 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.674631  1633 rpcz_store.cc:276] 0709 13:00:32.391873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.392062 (+   189us) service_pool.cc:224] Handling call
0709 13:00:32.418203 (+ 26141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:32.418408 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:32.436650 (+ 18242us) log.cc:844] Serialized 14 byte log entry
0709 13:00:33.417120 (+980470us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:33.417190 (+    70us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:33.417246 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.417333 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.417489 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142988,"prepare.queue_time_us":28749,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":3618,"raft.queue_time_us":182271,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":39469,"thread_start_us":818309,"threads_started":1,"wal-append.queue_time_us":1205185}
I20260709 13:00:33.746786  4049 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.854388  4049 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:33.855894  4144 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:33.610828  3734 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:33.733796  1636 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:33.731267  2319 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.046s	sys 0.056s
I20260709 13:00:33.681960  4113 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.736975  2351 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 2: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:33.738121  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:33.676277  4202 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.730885  1628 rpcz_store.cc:276] 0709 13:00:30.966833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.501709 (+534876us) service_pool.cc:224] Handling call
0709 13:00:31.502074 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.502297 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:31.851578 (+349281us) log.cc:844] Serialized 14 byte log entry
0709 13:00:33.657335 (+1805757us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b180044da60. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a
0709 13:00:33.657599 (+   264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:33.657664 (+    65us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:33.657721 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.657827 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.657975 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":411618,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":2388,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":5406,"spinlock_wait_cycles":5922688,"thread_start_us":385607,"threads_started":2,"wal-append.queue_time_us":146788,"wal-append.run_cpu_time_us":1083,"wal-append.run_wall_time_us":1022555}
I20260709 13:00:33.729626  4201 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.742816  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:33.862914  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 124835us
I20260709 13:00:33.864290  1628 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:33.864452  2347 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:33.743443  1710 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:33.743659  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:33.864816  1526 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:33.754009  1554 rpcz_store.cc:269] 0709 13:00:32.109501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.117657 (+  8156us) service_pool.cc:224] Handling call
0709 13:00:33.689988 (+1572331us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b5400313154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:33.690132 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":788053760}
I20260709 13:00:33.864899  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2192) took 1432 ms. Trace:
I20260709 13:00:33.865767  1628 rpcz_store.cc:276] 0709 13:00:32.432543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.863363 (+1430820us) service_pool.cc:224] Handling call
0709 13:00:33.863600 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.864680 (+  1080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.864847 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.755492  4209 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:33.754433  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 1994) took 1353 ms. Trace:
I20260709 13:00:33.759343  2353 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:33.677469  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:33.741369  4123 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.762754  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.735255  4189 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:33.765859  3917 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.766430  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:33.875934  1630 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:33.782452  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.783308  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:33.735852  3993 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:33.760396  3989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.068s	sys 0.089s
I20260709 13:00:33.795032  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:33.883726  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 100497us
I20260709 13:00:33.757426  4079 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:33.762144  4139 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:33.820026  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:33.790936  4205 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:00:33.906272  1554 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:33.797031  3874 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:33.818311  2346 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:33.826737  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:33.827168  4124 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:33.827962  2344 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:33.828275  4105 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.760283  4197 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:33.845146  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 1993) took 1444 ms. Trace:
I20260709 13:00:33.926678  1633 rpcz_store.cc:276] 0709 13:00:32.400808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.844354 (+1443546us) service_pool.cc:224] Handling call
0709 13:00:33.844680 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.844850 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:33.844874 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:33.844916 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:33.844951 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.845000 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.845096 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:33.696164  3970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.308s	user 0.003s	sys 0.012s
I20260709 13:00:33.849926  2349 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:33.788003  3951 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:33.859503  1709 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:33.696499  3954 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:00:33.867015  1634 rpcz_store.cc:276] 0709 13:00:32.401251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.669321 (+1268070us) service_pool.cc:224] Handling call
0709 13:00:33.754378 (+ 85057us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.867575  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:33.931147  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:33.931475  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64746.9us
I20260709 13:00:33.931694  1709 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:33.871564  2352 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:00:33.932644  1561 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.016s	sys 0.002s
I20260709 13:00:33.867857  1629 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
W20260709 13:00:33.932938  1561 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.016s	sys 0.002s
I20260709 13:00:33.933218  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 1996) took 1506 ms. Trace:
I20260709 13:00:33.933674  1629 rpcz_store.cc:276] 0709 13:00:32.427324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.748015 (+1320691us) service_pool.cc:224] Handling call
0709 13:00:33.933163 (+185148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.883781  1632 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:33.884771  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 462) took 1032 ms. Trace:
I20260709 13:00:33.885540  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:33.736874  4125 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:33.904531  1628 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:33.938691  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2002) took 1412 ms. Trace:
I20260709 13:00:33.938989  1628 rpcz_store.cc:276] 0709 13:00:32.526421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.866434 (+1340013us) service_pool.cc:224] Handling call
0709 13:00:33.884223 (+ 17789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.938457 (+ 54234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.938627 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.902472  4139 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:33.914688  2347 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
W20260709 13:00:33.915037  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.919077  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:33.912446  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:33.921234  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:33.874653  4113 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.874045  3941 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:33.874290  3963 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.923653  1702 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:33.954274  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:33.954900  1709 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:33.924607  1558 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:33.878146  3942 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:33.929591  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:33.926000  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 433778us
W20260709 13:00:33.899827  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.946718  1552 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:33.902148  4087 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:33.934443  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:33.936126  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 50696.6us
I20260709 13:00:33.959633  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 460) took 1108 ms. Trace:
I20260709 13:00:33.960405  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:33.960983  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48646.3us
I20260709 13:00:33.935024  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2001) took 1451 ms. Trace:
I20260709 13:00:33.936625  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2095) took 1003 ms. Trace:
I20260709 13:00:33.936735  1630 rpcz_store.cc:276] 0709 13:00:32.852054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.766143 (+914089us) service_pool.cc:224] Handling call
0709 13:00:33.884689 (+118546us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.913980  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1639) took 1213 ms. Trace:
I20260709 13:00:33.946719  2348 raft_consensus.cc:2368] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:33.875315  4201 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:33.962847  1556 rpcz_store.cc:276] 0709 13:00:32.700550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.132983 (+432433us) service_pool.cc:224] Handling call
0709 13:00:33.693660 (+560677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.744399 (+ 50739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.913724 (+169325us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b5400313150. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:33.913904 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1288559360}
I20260709 13:00:33.947352  1706 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:33.947880  1527 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:33.957928  1708 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:33.961961  1674 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:33.962396  1632 rpcz_store.cc:276] 0709 13:00:32.483640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.759074 (+1275434us) service_pool.cc:224] Handling call
0709 13:00:33.934968 (+175894us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.962482  1561 rpcz_store.cc:276] 0709 13:00:32.933063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.287297 (+354234us) service_pool.cc:224] Handling call
0709 13:00:33.299423 (+ 12126us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.299716 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:33.417801 (+118085us) log.cc:844] Serialized 14 byte log entry
0709 13:00:33.936320 (+518519us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:33.936367 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:33.936410 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.936475 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.936576 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2987,"prepare.queue_time_us":256685,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":3096,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":13180,"thread_start_us":619449,"threads_started":2}
I20260709 13:00:33.960179  1628 rpcz_store.cc:276] 0709 13:00:32.851484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.939662 (+1088178us) service_pool.cc:224] Handling call
0709 13:00:33.959019 (+ 19357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.959217 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:33.959253 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:33.959309 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:33.959353 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.959423 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.959551 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.936017  4192 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:33.894126  4123 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:33.929461  4157 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:33.981240  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:33.992821  1674 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:33.994488  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:33.995339  4142 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:33.995963  4142 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:33.996546  4142 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:33.996977  4142 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:33.984567  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:33.977571  1636 raft_consensus.cc:2410] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:33.998394  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 466) took 1117 ms. Trace:
I20260709 13:00:33.998695  1636 rpcz_store.cc:276] 0709 13:00:32.880859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.672430 (+791571us) service_pool.cc:224] Handling call
0709 13:00:33.998337 (+325907us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.999518  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:34.000170  1636 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:34.001000  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:34.001711  1627 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:34.002909  4142 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:33.989156  4123 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:34.006599  1636 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:34.007529  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:34.009490  1636 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:34.010912  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:34.008641  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:33.941267  4120 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:34.012919  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:34.015857  1636 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.015053  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:34.016441  1628 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:34.017138  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:33.954713  3941 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.021123  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:34.022235  1632 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:34.023001  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:34.023866  1630 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:33.956007  4124 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.957849  3951 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.957755  3942 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.030287  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:34.036586  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:33.877885  4202 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.034518  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:34.035621  1628 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:33.977573  4089 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.985569  4122 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.023051  4198 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:34.108831  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2004) took 1389 ms. Trace:
I20260709 13:00:34.111212  1709 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:34.013513  4087 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:33.979281  3970 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:34.111905  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 468) took 1171 ms. Trace:
I20260709 13:00:33.973984  3799 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:00:34.011762  4209 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:34.013593  3856 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:34.025198  3989 raft_consensus.cc:2764] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:00:34.027504  3941 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.112884  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:34.115964  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:34.115998  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:34.031730  3976 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:34.020979  4120 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:34.022629  4081 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 13:00:34.105786  4077 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:34.129522  1636 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:34.141316  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:34.136312  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.142560  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:34.143151  1709 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.120872  1628 rpcz_store.cc:276] 0709 13:00:32.940673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.017067 (+1076394us) service_pool.cc:224] Handling call
0709 13:00:34.035409 (+ 18342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.111582 (+ 76173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.111833 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.974071  3963 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.136860  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:34.113898  4210 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:00:34.150300  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:34.150866  3965 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:34.152046  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:34.152745  3965 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.156502  3981 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.107519  4124 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.112015  3942 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.135314  3976 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:34.161902  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.105433  3951 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.169400  3874 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.167940  4175 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:34.173867  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:34.176160  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:34.120467  3970 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:34.127342  4198 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:34.133810  4120 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.187373  3963 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:34.177790  3976 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.114236  4089 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.206739  1635 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:34.207888  3951 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:34.208531  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 1999) took 1733 ms (client timeout 2236 ms). Trace:
I20260709 13:00:34.201908  4131 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:00:34.209823  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:34.210654  1702 raft_consensus.cc:2410] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:34.211381  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:34.211997  1702 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:34.136619  4077 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.213687  4077 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.215636  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.216558  3981 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:34.216720  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.144171  4149 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.192658  1636 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:34.218245  4149 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:34.219717  4124 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:34.184286  3965 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:34.220230  4149 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.140799  4215 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.221143  1708 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:34.208971  1635 rpcz_store.cc:269] 0709 13:00:32.475578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.799064 (+323486us) service_pool.cc:224] Handling call
0709 13:00:34.208389 (+1409325us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540010b8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:34.208482 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3159450880}
I20260709 13:00:34.203964  3941 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:34.194995  4144 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:34.188311  4192 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.207988  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.188081  4201 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.202111  3917 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:34.203866  4175 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:34.226022  4105 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.220920  4215 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:34.114287  4139 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.202497  4189 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:34.222815  4211 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:00:34.137012  4157 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:34.228018  1708 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:34.259218  1708 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:00:34.224233  4214 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 13:00:34.218255  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2003) took 1638 ms. Trace:
W20260709 13:00:34.223492  4212 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:00:34.224890  1674 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:34.234097  1558 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:34.121325  4122 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:34.219735  3943 raft_consensus.cc:3055] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:34.195819  3970 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.237111  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.224848  4113 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:34.226078  4219 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:00:34.233640  4144 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.241670  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:34.245289  1709 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:34.201963  3993 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:34.238823  4192 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.263527  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2005) took 1511 ms. Trace:
I20260709 13:00:34.263800  1552 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:34.261021  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 98671.5us
W20260709 13:00:34.286705  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61871.3us
I20260709 13:00:34.289283  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.289793  3941 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.292531  1701 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:34.262290  1627 rpcz_store.cc:276] 0709 13:00:32.580013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.916036 (+1336023us) service_pool.cc:224] Handling call
0709 13:00:34.001554 (+ 85518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.002206 (+   652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.218078 (+215872us) spinlock_profiling.cc:243] Waited 85.2 ms on lock 0x7b5400197b50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:34.218204 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195921152}
I20260709 13:00:34.300479  1561 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:34.303107  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.317755  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 412) took 2146 ms. Trace:
I20260709 13:00:34.305737  1635 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.311997  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.314504  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:34.311573  1709 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:34.310297  1634 rpcz_store.cc:276] 0709 13:00:32.752515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.930936 (+1178421us) service_pool.cc:224] Handling call
0709 13:00:33.955178 (+ 24242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.955357 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:34.119722 (+164365us) log.cc:844] Serialized 14 byte log entry
0709 13:00:34.263203 (+143481us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.263253 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:34.263298 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.263358 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.263476 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":202202,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":101,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":14084,"thread_start_us":278301,"threads_started":2}
I20260709 13:00:34.319427  1704 rpcz_store.cc:276] 0709 13:00:32.171760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.482890 (+311130us) service_pool.cc:224] Handling call
0709 13:00:34.317587 (+1834697us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54000cbcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:34.317698 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671541504}
I20260709 13:00:34.322414  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.324319  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:34.284214  4125 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:34.314939  1676 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:34.304492  4217 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:00:34.297581  4139 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.330580  3993 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:34.302089  4144 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.315380  4079 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:34.293764  3943 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:34.297663  3989 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:34.330401  4115 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:34.337661  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:34.338770  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.350967  1634 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.351998  1676 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:34.297446  4202 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.327705  3967 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:34.341095  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.293426  3976 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.296644  2346 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:34.344084  4138 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:00:34.359684  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 163518us
I20260709 13:00:34.202145  4087 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.307018  4104 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:34.329905  4073 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:34.305430  4120 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:34.346701  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 398859us
I20260709 13:00:34.354318  3734 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:34.343844  3993 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.313972  4002 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.296494  4077 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.356063  1558 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:34.338259  4192 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:34.361702  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:34.284133  4157 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.349958  3989 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.295053  4122 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.350212  4115 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:34.355525  4125 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:34.367632  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:34.368418  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.369010  2348 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:34.360277  4215 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.366016  4078 consensus_queue.cc:1059] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:00:34.376708  4120 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.376157  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.377025  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:34.362816  4104 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.366549  4138 raft_consensus.cc:697] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:34.373754  4195 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.002s
I20260709 13:00:34.382781  1634 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:34.387666  4115 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.364518  4113 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:00:34.389803  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.372534  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:34.336275  4198 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.359141  3943 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:34.384655  4185 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.001s
I20260709 13:00:34.393703  1557 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:34.393903  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.391388  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:34.394354  1560 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:34.382783  3989 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:34.402711  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.371870  3993 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.373312  4190 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:00:34.438141  1634 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:00:34.374800  4002 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.405936  4138 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.406490  4157 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:34.411847  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2120) took 2707 ms. Trace:
I20260709 13:00:34.412271  4078 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:34.388633  2345 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:34.388614  4125 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.422142  1635 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:34.422976  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:34.428077  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 55617.5us
I20260709 13:00:34.428251  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:34.371142  4149 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:00:34.460541  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:34.466420  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:34.470952  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 48094.2us
I20260709 13:00:34.471076  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:34.472240  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.470630  1707 rpcz_store.cc:276] 0709 13:00:31.704593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.055856 (+351263us) service_pool.cc:224] Handling call
0709 13:00:32.056204 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:32.195093 (+138889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:32.195128 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:34.126824 (+1931696us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b2c000732a8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:00:34.127199 (+   375us) log.cc:844] Serialized 26 byte log entry
0709 13:00:34.311357 (+184158us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:34.311400 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.311514 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.411560 (+100046us) raft_consensus.cc:1685] finished
0709 13:00:34.411581 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.411786 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":260330,"raft.queue_time_us":158061,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":10755,"spinlock_wait_cycles":4816256,"thread_start_us":133113,"threads_started":1,"wal-append.queue_time_us":220212}
W20260709 13:00:34.479321  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51160.1us
I20260709 13:00:34.479848  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.483548  1674 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:34.435323  3734 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:34.491976  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.435472  3801 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:34.467917  4104 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.493273  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.442153  2344 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:34.473660  4125 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.424844  4195 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.498705  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:34.471040  4078 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:34.478881  3965 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:34.505074  3965 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:34.505467  4078 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:34.506460  4078 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.505507  3965 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.510458  4243 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:34.510936  4243 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:34.511592  4243 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:34.490942  4209 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:34.435648  4077 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.532290  3976 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.502664  3801 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:34.534001  1554 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:34.489425  4073 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:34.533906  4185 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:00:34.532177  4182 consensus_queue.cc:1059] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:00:34.535887  3734 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:00:34.540181  4182 raft_consensus.cc:3055] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:34.540616  4182 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:34.538688  3801 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:34.546888  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 1.00795e+06us
I20260709 13:00:34.555105  1557 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:00:34.554437  1634 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:00:34.558735  1560 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:34.563939  4104 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:34.567802  4002 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:34.570863  1675 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.572672  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.541420  4182 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.569258  4189 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:00:34.580644  4229 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 13:00:34.580829  4236 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:00:34.573727  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:34.586637  1557 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:34.555109  4209 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:34.587775  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 106952us
I20260709 13:00:34.545855  3942 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:34.597730  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.588186  4002 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:34.586104  4189 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:34.603168  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.592384  4185 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:34.615732  1634 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:34.606765  4175 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:34.622730  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 409) took 2472 ms. Trace:
I20260709 13:00:34.623148  1702 rpcz_store.cc:276] 0709 13:00:32.150298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.212488 (+2062190us) service_pool.cc:224] Handling call
0709 13:00:34.212785 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:34.228489 (+ 15704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.228516 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:34.228936 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 13:00:34.364099 (+135163us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:34.364140 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.364217 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.622485 (+258268us) raft_consensus.cc:1685] finished
0709 13:00:34.622513 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.622671 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15376,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":11861,"thread_start_us":101717,"threads_started":1,"wal-append.queue_time_us":238799}
I20260709 13:00:34.589150  4245 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:00:34.629858  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:34.630203  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:34.633602  2349 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:34.635089  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:34.636641  2353 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:34.641083  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.641155  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.641924  2353 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.645511  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:34.646564  2349 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:34.648334  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:34.650211  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.654592  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.655748  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.656248  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.657353  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.618641  4185 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.657297  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:34.659339  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.654793  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:34.659950  2352 raft_consensus.cc:2368] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4.
I20260709 13:00:34.664846  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:34.665555  2352 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:34.667582  2348 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:00:34.668433  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:34.677314  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 90522us
I20260709 13:00:34.681555  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.681768  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 458) took 1831 ms. Trace:
I20260709 13:00:34.681957  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.682971  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:34.683051  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.683832  2347 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:34.682308  1629 rpcz_store.cc:276] 0709 13:00:32.850859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.934281 (+1083422us) service_pool.cc:224] Handling call
0709 13:00:33.958314 (+ 24033us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:33.980090 (+ 21776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:33.980135 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:34.029290 (+ 49155us) log.cc:844] Serialized 26 byte log entry
0709 13:00:34.321230 (+291940us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:34.321269 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.321361 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.681564 (+360203us) raft_consensus.cc:1685] finished
0709 13:00:34.681583 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.681711 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2045,"raft.queue_time_us":331,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":32179,"thread_start_us":195371,"threads_started":1,"wal-append.queue_time_us":278520}
I20260709 13:00:34.684237  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:34.685930  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:34.653707  3976 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:34.690434  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:34.688215  2353 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:34.648284  4077 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.652361  4198 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:34.687191  4124 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:34.667623  4185 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.692637  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.691340  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:34.595229  4073 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.693857  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:34.699234  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:34.668222  4157 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:34.682307  3799 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:34.702658  1702 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.652098  4104 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:00:34.693145  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:34.672600  4149 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:34.705732  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.701941  1557 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:34.704154  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:34.704399  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.700613  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:34.713357  3951 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.658600  4002 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:00:34.694312  3976 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.691075  4081 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:34.685518  4202 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:00:34.739509  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:34.722424  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.741631  2353 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:34.691401  4223 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:34.757637  3734 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:34.758308  3734 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:34.746860  1707 raft_consensus.cc:2410] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:34.707373  3954 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:34.684988  3993 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:34.747727  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.687376  4135 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:00:34.749725  2352 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:34.752043  4215 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:34.745765  4139 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:00:34.707736  3965 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:34.804937  4149 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.757704  4223 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:34.746448  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64567us
I20260709 13:00:34.713934  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:34.694757  4201 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:00:34.792177  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.759197  4002 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:00:34.793601  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.767150  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:34.850548  2353 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:34.746274  3976 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:34.806484  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 84146.8us
I20260709 13:00:34.851737  1557 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:34.815227  1702 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:34.820030  4027 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:34.755694  4125 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.704226  4077 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.857738  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:34.826696  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:34.828473  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:34.698292  4189 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.791617  4195 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:34.795392  3954 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.847716  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:34.814973  4157 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:00:34.848871  3734 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:34.796962  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 166144us
I20260709 13:00:34.852285  2318 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:34.853260  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:34.819146  1602 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:34.796092  4197 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.855s	user 0.021s	sys 0.000s
I20260709 13:00:34.702291  4209 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.859488  1629 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.856642  4139 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:34.851320  4223 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.867406  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:34.867365  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 48319.9us
W20260709 13:00:34.861243  3970 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.564s	user 0.009s	sys 0.002s
W20260709 13:00:34.869553  4218 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.932s	user 0.001s	sys 0.002s
I20260709 13:00:34.902323  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:34.903589  1676 leader_election.cc:400] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4.
I20260709 13:00:34.903867  1676 leader_election.cc:403] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:34.904711  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:34.869987  3940 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:34.870383  3734 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.870790  4257 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:34.872217  4125 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:34.872679  4189 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.865072  3954 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.875509  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:34.877626  2348 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.941431  1561 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.878213  4077 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:34.878786  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:34.847026  4149 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.829840  4078 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:00:34.865533  2344 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:00:34.869673  4247 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:00:34.896246  1702 raft_consensus.cc:2410] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:00:34.899559  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 472) took 1443 ms. Trace:
I20260709 13:00:34.905308  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.892338  4131 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:00:34.891723  4197 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.909008  3970 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.901703  4027 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:34.888161  4252 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.906790  1602 leader_election.cc:400] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:34.868019  4195 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.917711  1629 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:35.000131  1602 leader_election.cc:403] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:35.000648  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 99847.3us
I20260709 13:00:35.000799  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:34.940490  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:34.913113  4124 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:00:34.934885  4256 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4.
I20260709 13:00:34.940511  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:35.003540  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 63091.6us
I20260709 13:00:35.004200  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:35.001308  1602 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:34.944622  2317 leader_election.cc:336] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.511s (SENT)
W20260709 13:00:35.005612  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61022.2us
I20260709 13:00:35.005678  1706 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:35.005224  1676 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:34.945411  3967 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:34.946254  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2109) took 3752 ms (client timeout 2218 ms). Trace:
I20260709 13:00:34.913364  4245 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:34.940847  3981 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:00:35.011721  3967 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.013672  4027 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:35.014679  4027 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.016152  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.016750  1629 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:34.952390  2353 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:34.892074  4139 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:35.018122  4261 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:35.018661  4197 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:35.018532  4139 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.019829  4139 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.021124  4001 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:34.955545  2352 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:35.021798  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 408) took 2878 ms. Trace:
I20260709 13:00:34.958567  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.022904  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.023089  3993 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:35.023759  1702 raft_consensus.cc:2410] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:35.023669  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:35.024348  3970 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:35.024561  1556 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:34.980934  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.024612  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.028743  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:34.896494  3989 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.989552  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:34.989596  1561 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:35.032379  1635 raft_consensus.cc:2393] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.032735  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:35.022164  1703 rpcz_store.cc:276] 0709 13:00:32.149969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.299483 (+1149514us) service_pool.cc:224] Handling call
0709 13:00:33.357866 (+ 58383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:33.386280 (+ 28414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:33.386309 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:33.387335 (+  1026us) log.cc:844] Serialized 26 byte log entry
0709 13:00:33.933816 (+546481us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:34.112185 (+178369us) log.cc:844] Serialized 14 byte log entry
0709 13:00:34.114742 (+  2557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.114807 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.223718 (+108911us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:34.954390 (+730672us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b24001f90e8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda244739c4 00007fda24471acb 00007fda2462cdc9 00007fda2462ca55 00007fda2462bfb0 00007fda272a2488 00007fda272a2be8 00007fda2727a398 00007fda2727576a 00007fda27274bb6
0709 13:00:35.021589 (+ 67199us) raft_consensus.cc:1685] finished
0709 13:00:35.021610 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:35.021745 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27959,"prepare.queue_time_us":234263,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":5263,"raft.queue_time_us":32554,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":19819,"spinlock_wait_cycles":31230592,"thread_start_us":517350,"threads_started":2,"wal-append.queue_time_us":513638}
I20260709 13:00:34.935492  4257 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:34.990962  1559 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:34.991454  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2188) took 2591 ms. Trace:
I20260709 13:00:35.034250  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:34.896731  3976 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:34.944144  4125 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:34.938050  4002 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:00:35.009568  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.041968  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.036902  1631 rpcz_store.cc:276] 0709 13:00:32.400341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.779832 (+1379491us) service_pool.cc:224] Handling call
0709 13:00:33.780159 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:33.780547 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:33.780578 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:33.780996 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:00:33.937999 (+157003us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:33.938038 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.949720 (+ 11682us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.851765 (+902045us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:34.947312 (+ 95547us) raft_consensus.cc:1685] finished
0709 13:00:34.947338 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.991386 (+ 44048us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":44097920,"thread_start_us":132085,"threads_started":1,"wal-append.queue_time_us":156641}
I20260709 13:00:35.044080  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.011183  2348 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
W20260709 13:00:34.951483  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 267287us
W20260709 13:00:35.008280  1701 rpcz_store.cc:269] 0709 13:00:31.194004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.262039 (+ 68035us) service_pool.cc:224] Handling call
0709 13:00:34.946043 (+3684004us) spinlock_profiling.cc:243] Waited 3.03 s on lock 0x7b5400246b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:34.946170 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6967451008}
W20260709 13:00:35.027686  4115 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.006s
I20260709 13:00:34.951835  1628 rpcz_store.cc:276] 0709 13:00:33.456643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.145702 (+689059us) service_pool.cc:224] Handling call
0709 13:00:34.184705 (+ 39003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.392218 (+207513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.392265 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.392323 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:34.392369 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.899239 (+506870us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b540010bdd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:34.899304 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.899479 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":476712320}
I20260709 13:00:35.030507  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:35.049304  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:35.050750  2350 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:35.013263  3965 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:35.048916  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.050074  1556 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:35.058259  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.059230  1556 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:35.017625  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.050081  1561 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:00:35.060823  1561 pending_rounds.cc:85] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 1
I20260709 13:00:35.061079  1561 pending_rounds.cc:107] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:00:35.061296  1561 raft_consensus.cc:2889] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:35.064150  1709 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.064175  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.064908  1556 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:35.037377  4256 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:35.047850  1636 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.012945  4073 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:35.051326  1631 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:35.066635  2349 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:35.036281  4131 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:35.068375  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.078949  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:35.071691  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.067823  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.077582  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.012300  4195 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.063628  3993 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:35.072320  4256 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.078481  4197 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:00:35.112640  3951 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:35.063800  4245 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:35.113039  2350 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:35.059561  4185 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.066107  4078 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:35.114312  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.063200  4077 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:35.160573  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:35.108366  4257 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.079396  1701 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.140030  3874 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:35.144704  3801 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:35.144928  1635 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:35.072965  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:35.166841  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 165618us
I20260709 13:00:35.171144  1636 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.173944  4077 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:35.175271  1558 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:35.144819  4182 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:35.177142  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 115535us
I20260709 13:00:35.063469  4115 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.165503  1628 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:35.080351  3970 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.081s
I20260709 13:00:35.156298  4256 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.178321  1602 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:35.179159  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:35.151480  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2197) took 1202 ms. Trace:
I20260709 13:00:35.182940  1630 rpcz_store.cc:276] 0709 13:00:33.948762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.393862 (+445100us) service_pool.cc:224] Handling call
0709 13:00:34.429416 (+ 35554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.429667 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:34.448618 (+ 18951us) log.cc:844] Serialized 14 byte log entry
0709 13:00:34.880041 (+431423us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b18000c2260. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:00:35.151005 (+270964us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:35.151079 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:35.151151 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.151244 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:35.151406 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8633,"prepare.queue_time_us":18757,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":41021,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":8331,"spinlock_wait_cycles":65275136,"thread_start_us":262007,"threads_started":2,"wal-append.queue_time_us":602397}
I20260709 13:00:35.181833  1636 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:35.165433  4267 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:35.170351  4078 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:35.189837  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.155336  3993 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.190528  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.188928  1602 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:35.192396  1628 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.192014  1703 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.195338  3989 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:35.195916  4262 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:00:35.202483  4115 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:35.174437  4223 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.187602  4125 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:35.208629  1553 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:35.166790  4245 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.191784  4157 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:35.218887  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.220901  1556 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:35.183367  4131 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:00:35.223691  1602 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:00:35.228531  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 222777us
I20260709 13:00:35.227155  1602 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:35.234439  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:35.235175  2352 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:35.219129  4125 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:35.219548  4157 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:35.250344  4157 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.235275  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:35.250741  4125 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.251360  4245 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:35.251735  2351 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:35.238502  3993 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:35.239888  4195 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:35.231578  3898 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:35.252882  3898 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:35.253979  3898 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.254933  3898 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:35.228267  4260 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.315s	user 0.004s	sys 0.000s
I20260709 13:00:35.257571  4078 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.258940  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.259121  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.259718  1634 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.260069  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.260864  1703 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.261586  4027 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:00:35.262524  4027 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:35.262921  4027 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:35.263139  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.263870  1636 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:35.264923  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.263833  4027 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.225545  4268 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:35.245808  4269 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:35.247658  4223 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:35.247974  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.249228  4185 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:35.249799  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:35.265458  1636 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:35.271997  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.272972  1636 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:35.275501  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.268623  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:35.228041  4267 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:35.216852  4077 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.284085  1631 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:00:35.292523  1676 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:35.459836  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:35.460520  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.460624  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1646) took 1278 ms. Trace:
I20260709 13:00:35.461020  1560 rpcz_store.cc:276] 0709 13:00:34.182007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.436097 (+254090us) service_pool.cc:224] Handling call
0709 13:00:35.460449 (+1024352us) spinlock_profiling.cc:243] Waited 83.5 ms on lock 0x7b5400207454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:35.460572 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192100096}
I20260709 13:00:35.461504  3941 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.462034  3941 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.382040  3951 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.463219  3951 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:35.345583  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.462344  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:35.359537  4144 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.009s	user 0.009s	sys 0.012s
I20260709 13:00:35.380041  1702 raft_consensus.cc:2410] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:35.465797  4144 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.466234  1560 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: [2.5-3.6]
I20260709 13:00:35.432683  4233 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:35.466867  1703 raft_consensus.cc:2368] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:35.466850  4233 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:35.467492  1560 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:35.467860  4233 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.468570  4144 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:35.429209  1634 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:35.437057  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:34.114223  1633 rpcz_store.cc:276] 0709 13:00:32.719680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.927850 (+1208170us) service_pool.cc:224] Handling call
0709 13:00:33.928318 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.928563 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:34.017832 (+ 89269us) log.cc:844] Serialized 14 byte log entry
0709 13:00:34.107402 (+ 89570us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.107876 (+   474us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:34.108271 (+   395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.108628 (+   357us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.108764 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6100,"thread_start_us":151844,"threads_started":2}
I20260709 13:00:35.355871  4256 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:35.471155  4253 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.720s	user 0.001s	sys 0.000s
I20260709 13:00:35.334105  4252 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:35.470893  2346 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:35.472332  4027 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.472700  4256 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:35.473176  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 330) took 1733 ms (client timeout 1525 ms). Trace:
W20260709 13:00:35.384819  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1596) took 1976 ms (client timeout 1793 ms). Trace:
W20260709 13:00:35.473706  1559 rpcz_store.cc:269] 0709 13:00:33.739815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.746585 (+  6770us) service_pool.cc:224] Handling call
0709 13:00:35.472994 (+1726409us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54001ce154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:35.473115 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2859169024}
W20260709 13:00:35.474362  1710 rpcz_store.cc:269] 0709 13:00:33.407874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.655218 (+247344us) service_pool.cc:224] Handling call
0709 13:00:35.384578 (+1729360us) spinlock_profiling.cc:243] Waited 71.8 ms on lock 0x7b540016ce54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:35.384729 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165043584}
I20260709 13:00:35.475606  4138 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:00:35.470244  4250 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.717s	user 0.002s	sys 0.000s
I20260709 13:00:35.468984  4233 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.384213  4079 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:35.347255  3967 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:35.339581  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 71036.4us
I20260709 13:00:35.286123  1628 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:35.476583  1554 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:35.479943  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.505735  1702 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:35.506235  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 213828us
I20260709 13:00:35.483433  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.507953  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:35.484665  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.511866  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:35.512480  2352 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
W20260709 13:00:35.514101  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.489866  1525 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:35.475859  4255 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.723s	user 0.002s	sys 0.000s
I20260709 13:00:35.267354  4269 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.330263  3970 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.526757  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.530280  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.492426  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:35.528827  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.534260  1709 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
W20260709 13:00:35.481683  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.475732  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:35.494208  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.535552  1710 raft_consensus.cc:2376] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3. Re-sending same reply.
I20260709 13:00:35.530864  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.359671  4238 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:35.533380  1701 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.538436  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:35.482407  4215 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:35.483358  4079 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:35.267529  4270 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.260s
I20260709 13:00:35.546187  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.573840  1628 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:35.574980  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.576256  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.352357  3954 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.549780  1702 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:35.503238  1558 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:35.513892  4144 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:35.511708  3942 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:35.544579  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48951us
I20260709 13:00:35.587839  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.267300  3993 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.260s
I20260709 13:00:35.552917  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:35.555955  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:35.488694  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:35.560719  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 52719.1us
I20260709 13:00:35.562376  2350 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:35.526288  4256 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.622277  4256 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.465801  3951 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:35.564134  2345 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:35.579720  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.626327  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.627306  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.627959  1709 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.591629  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:35.570794  4190 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:00:35.629942  4157 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.631039  4144 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:35.599345  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.599848  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2196) took 1651 ms. Trace:
I20260709 13:00:35.476008  4138 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:35.503708  4268 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.603497  4149 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:35.571295  3917 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:35.571125  4238 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:35.606525  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.645884  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.564622  4215 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.609822  3993 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:35.610105  2344 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:35.610351  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 57567us
I20260709 13:00:35.610826  4078 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:35.670365  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:35.615783  3967 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:35.616226  4185 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:35.614830  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.671255  2345 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:35.670871  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:35.672170  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.672914  1704 raft_consensus.cc:2368] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4.
I20260709 13:00:35.618733  1674 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:00:35.674268  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 55593.7us
I20260709 13:00:35.619253  3734 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:35.622658  4122 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:35.628669  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.681638  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.628801  3943 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.633809  1627 rpcz_store.cc:276] 0709 13:00:33.948600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.333383 (+384783us) service_pool.cc:224] Handling call
0709 13:00:34.333734 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:34.334078 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.334108 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:34.334539 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:00:34.624496 (+289957us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b1800299380. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:00:34.751830 (+127334us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:34.751869 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.752379 (+   510us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:35.475276 (+722897us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:35.599622 (+124346us) raft_consensus.cc:1685] finished
0709 13:00:35.599646 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:35.599790 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3305,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":5307,"spinlock_wait_cycles":105774848,"thread_start_us":127150,"threads_started":1,"wal-append.queue_time_us":417342}
I20260709 13:00:35.634703  1557 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:35.634872  1553 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:35.686089  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.686746  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:35.628687  1701 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:35.688136  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.635468  1703 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.635442  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:35.690006  1558 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
W20260709 13:00:35.639966  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 84081.5us
W20260709 13:00:35.590445  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 101830us
I20260709 13:00:35.691655  1526 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:35.546902  1636 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.643813  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.694447  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.533191  4269 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:35.644726  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.631733  4089 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:35.629881  4241 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.121s	user 0.000s	sys 0.002s
I20260709 13:00:35.624886  4079 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.645924  4197 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.650177  4157 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:35.624920  4256 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:35.554906  3970 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:35.653457  3954 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:35.646376  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.554775  4267 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:35.645540  4077 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:35.656947  4268 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.659214  4238 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.670287  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:35.670692  4078 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.620766  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.678370  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.670221  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:35.741937  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 71826.7us
I20260709 13:00:35.743729  1709 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.683482  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.744742  4089 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:35.686774  1627 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.695197  1628 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:35.692901  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:35.678879  4115 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:35.696460  4138 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.697400  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.750284  1703 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.698163  3917 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.699822  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:35.701623  4105 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:35.753791  4115 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.702564  4113 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:35.702696  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:35.787766  1553 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:35.788795  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:35.789458  1553 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.696502  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.703822  1708 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:35.703586  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 415) took 1916 ms. Trace:
I20260709 13:00:35.792613  1706 rpcz_store.cc:276] 0709 13:00:33.786903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.128340 (+341437us) service_pool.cc:224] Handling call
0709 13:00:35.703429 (+1575089us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b540036e2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:35.703537 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1961793920}
I20260709 13:00:35.705117  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:35.794334  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 89323.8us
I20260709 13:00:35.706527  3941 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:35.710029  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.801792  1635 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.711026  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.802961  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.709281  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.666268  3993 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:35.714774  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.805732  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.806341  1636 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:35.687640  4001 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:35.721848  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:35.809517  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:35.809940  1631 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:00:35.810087  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 88305us
I20260709 13:00:35.810648  1704 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.811194  1631 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:35.811689  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:35.812547  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:35.812647  1631 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:35.723526  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:35.717525  4243 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.184s	user 0.012s	sys 0.019s
I20260709 13:00:35.742964  1602 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:35.823315  2349 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:35.823489  1627 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
W20260709 13:00:35.750801  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 57968.2us
I20260709 13:00:35.684304  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.824688  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:35.824998  1702 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.753839  1558 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.753847  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.826701  1628 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
W20260709 13:00:35.823786  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 80886us
I20260709 13:00:35.738364  4275 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.679064  4270 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.741703  4078 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:35.738147  4077 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:35.793702  4113 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:35.810194  4245 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:35.753657  3954 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:35.782799  4285 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:00:35.785876  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.748689  4089 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.792991  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.892431  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.708312  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.832830  4259 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:35.807175  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:35.680410  4122 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:35.813259  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.861901  4233 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.907589  1701 raft_consensus.cc:2410] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:35.816627  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:35.817281  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:35.822613  4001 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:35.854350  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:35.839439  1553 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:35.856317  1635 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:35.817588  4105 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:35.780893  4242 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.276s	user 0.004s	sys 0.000s
I20260709 13:00:35.833899  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:35.833573  4243 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.289s	user 0.013s	sys 0.020s
W20260709 13:00:35.914007  4289 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.092s	user 0.001s	sys 0.001s
I20260709 13:00:35.914499  4243 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:35.914212  4001 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:35.914014  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 80216.9us
I20260709 13:00:35.918211  3970 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:35.918571  4201 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:35.919265  4113 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.920598  4113 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:35.920492  4122 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.837257  3874 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:35.921597  2351 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.922257  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:35.922407  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.923038  1701 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
W20260709 13:00:35.923295  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.923497  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1615) took 1064 ms. Trace:
I20260709 13:00:35.923861  1701 rpcz_store.cc:276] 0709 13:00:34.860442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.911878 (+1051436us) service_pool.cc:224] Handling call
0709 13:00:35.923445 (+ 11567us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.925767  4079 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:35.863786  4267 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.921625  4122 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.870251  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.894421  1629 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.779407  4138 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:35.934428  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:35.894340  1552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2109) took 1787 ms (client timeout 1683 ms). Trace:
I20260709 13:00:35.850749  4157 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:35.850325  3734 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:35.908833  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 92263.2us
I20260709 13:00:35.902227  4089 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.857678  4142 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.913761  1555 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:35.930119  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:35.928876  4269 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.937155  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:35.872084  4144 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:35.940105  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:35.937199  1552 rpcz_store.cc:269] 0709 13:00:34.107122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.129031 (+ 21909us) service_pool.cc:224] Handling call
0709 13:00:35.894111 (+1765080us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400206f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:35.894245 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231834496}
I20260709 13:00:35.954457  1675 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:35.954804  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:35.949366  4245 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:00:35.955956  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:35.956820  4245 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:35.956951  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:35.934960  4233 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.941419  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:35.957413  4245 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:35.909515  1703 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:35.958609  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1618) took 1092 ms. Trace:
I20260709 13:00:35.958356  4245 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:35.958928  4233 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:35.959393  4027 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:35.959801  4027 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.960152  4027 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.960469  1636 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:35.962239  3970 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.963254  2347 raft_consensus.cc:2955] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEARNER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:35.959049  1703 rpcz_store.cc:276] 0709 13:00:34.865794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.892140 (+1026346us) service_pool.cc:224] Handling call
0709 13:00:35.958545 (+ 66405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.944800  1702 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:35.935637  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:35.935297  2351 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:35.867511  1704 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:35.950243  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.895033  1561 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:35.968905  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:35.863462  4270 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.930897  4259 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:35.938968  4079 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.939179  4157 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.943212  3734 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.065490  1632 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:36.058262  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:36.066994  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:36.067315  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:36.060976  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:36.071010  1703 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
W20260709 13:00:36.061757  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.061956  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44097.4us
I20260709 13:00:36.063015  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:36.084106  1701 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:35.951099  4197 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:36.064522  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.053938  1556 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:36.086436  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:36.046531  4233 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:36.093321  1556 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.079136  4157 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:36.103364  1703 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:00:36.106918  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1604) took 1253 ms. Trace:
I20260709 13:00:36.107285  1703 rpcz_store.cc:276] 0709 13:00:34.853426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.071824 (+1218398us) service_pool.cc:224] Handling call
0709 13:00:36.103046 (+ 31222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.106627 (+  3581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.106867 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:36.103376  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:35.939314  3993 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:35.953014  4142 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.111572  1556 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.082288  3874 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:36.110409  1633 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:36.092044  3970 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:36.077751  3734 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.113549  1556 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:36.104207  4081 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:36.082894  4215 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:36.105118  4268 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:36.132968  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.097764  4197 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.122808  3993 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:36.141250  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 74027.6us
I20260709 13:00:36.117645  4115 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:36.147162  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:36.077641  4077 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.124960  4081 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:36.107836  4233 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:36.119031  3874 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:36.192183  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.193099  1558 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:36.195469  1558 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:36.197192  1558 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.173525  4115 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:36.145653  4259 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.147091  4192 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:00:36.179405  4245 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.208544  1558 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:36.213624  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.221678  4081 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.222618  3874 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:36.222965  4081 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:36.223160  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:36.223927  1628 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:36.227344  4302 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:00:36.229007  4197 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:36.216413  4245 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:36.242933  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:36.245076  1628 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:36.247532  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:36.232502  4245 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:36.249732  4268 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.251132  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:36.254285  1558 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:36.256186  1628 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:36.258862  1675 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:36.262679  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:36.255144  4115 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:36.264964  4306 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:00:36.265635  4268 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.274338  4142 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:36.278505  4215 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:36.219020  4269 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:36.241434  4073 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:36.284929  1628 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:36.279333  4251 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:00:36.297709  4251 raft_consensus.cc:3055] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:36.298065  4251 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:36.298913  4251 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.299412  4078 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.300635  4251 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:36.301245  1634 raft_consensus.cc:2368] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9.
I20260709 13:00:36.237910  4027 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.303843  4027 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:36.261868  4079 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:00:36.305538  4142 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:36.257506  4001 consensus_queue.cc:1059] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:00:36.279495  4243 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:00:36.284189  4245 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:00:36.354301  4233 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:36.355505  4257 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:36.358436  4078 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:36.331704  4142 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.306432  4073 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.359118  4142 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:36.360585  4073 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:36.341049  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 125662us
I20260709 13:00:36.368940  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:36.357077  4089 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:36.394771  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 253988us
I20260709 13:00:36.396376  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1612) took 1540 ms. Trace:
I20260709 13:00:36.396723  1703 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:36.396812  1709 rpcz_store.cc:276] 0709 13:00:34.856250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.681314 (+825064us) service_pool.cc:224] Handling call
0709 13:00:36.396154 (+714840us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b54000cb2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:36.396307 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129289984}
I20260709 13:00:36.397364  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:36.398963  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:36.406069  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:36.406847  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:36.422865  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:36.424405  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:36.425597  1709 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:36.343372  4079 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:36.369539  3856 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:00:36.428807  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:36.430083  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:36.420869  4077 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:36.421500  4310 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:36.429945  4105 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:36.429039  4079 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:36.430266  4144 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:36.441670  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.446026  3856 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:36.447232  4027 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.498572  4027 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:36.427260  3734 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:36.440812  4105 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:36.442191  4077 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:36.524062  1706 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:36.528249  3967 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 13:00:36.528901  3967 raft_consensus.cc:1068] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:00:36.530135  4308 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:00:36.530402  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.531087  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.532016  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.533334  1629 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.534539  1629 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:36.535436  1526 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.882s (SENT) [suppressed 22 similar messages]
I20260709 13:00:36.537668  1709 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:36.538244  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1609) took 1683 ms. Trace:
I20260709 13:00:36.538529  1709 rpcz_store.cc:276] 0709 13:00:34.854681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.431501 (+1576820us) service_pool.cc:224] Handling call
0709 13:00:36.537512 (+106011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.538027 (+   515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.538197 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:36.541353  1709 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:36.522140  4105 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:36.543052  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 113045us
I20260709 13:00:36.547237  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2127) took 1596 ms. Trace:
I20260709 13:00:36.513679  4027 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:36.548009  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.540431  4309 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:00:36.547685  1709 rpcz_store.cc:276] 0709 13:00:34.946427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.539246 (+1592819us) service_pool.cc:224] Handling call
0709 13:00:36.541122 (+  1876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:36.541764 (+   642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.542433 (+   669us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:36.566084  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 138596us
I20260709 13:00:36.570490  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:36.575089  4142 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:36.575532  1560 raft_consensus.cc:2955] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:36.577721  4089 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:36.578094  4142 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:36.579470  4138 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.001s
I20260709 13:00:36.580878  4201 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.581405  4201 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.555004  4144 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:36.585702  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1628) took 1054 ms. Trace:
I20260709 13:00:36.586087  1704 rpcz_store.cc:276] 0709 13:00:35.531094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.673607 (+142513us) service_pool.cc:224] Handling call
0709 13:00:36.585539 (+911932us) spinlock_profiling.cc:243] Waited 72.4 ms on lock 0x7b5400224f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:36.585650 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166615936}
I20260709 13:00:36.537515  4079 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:36.587364  1704 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:36.587713  4079 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.587957  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2129) took 1641 ms. Trace:
I20260709 13:00:36.588311  1704 rpcz_store.cc:276] 0709 13:00:34.946933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.586857 (+1639924us) service_pool.cc:224] Handling call
0709 13:00:36.587167 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:36.587722 (+   555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.587909 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:36.558102  4303 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.378s	user 0.003s	sys 0.000s
W20260709 13:00:36.590562  4113 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.669s	user 0.013s	sys 0.000s
I20260709 13:00:36.591197  4113 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:36.591852  1631 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:36.591706  4113 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:36.592121  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1650) took 2409 ms. Trace:
I20260709 13:00:36.592499  1555 rpcz_store.cc:276] 0709 13:00:34.183062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.491733 (+308671us) service_pool.cc:224] Handling call
0709 13:00:36.591955 (+2100222us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540038b8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:36.592048 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3268252544}
I20260709 13:00:36.593077  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2031) took 1087 ms. Trace:
I20260709 13:00:36.593605  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:36.576905  4157 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.602273  1629 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:00:36.631584  1629 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:36.632908  1629 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:36.593488  1631 rpcz_store.cc:276] 0709 13:00:35.507513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.821499 (+313986us) service_pool.cc:224] Handling call
0709 13:00:35.893154 (+ 71655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.592267 (+699113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.592845 (+   578us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b540017be54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:36.593020 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1606041984}
I20260709 13:00:36.605947  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:36.634326  1555 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:36.611286  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2132) took 1656 ms. Trace:
I20260709 13:00:36.633276  1629 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:36.637975  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.635406  1704 rpcz_store.cc:276] 0709 13:00:34.954786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.589054 (+1634268us) service_pool.cc:224] Handling call
0709 13:00:36.589454 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.610841 (+ 21387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:36.610885 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:36.610941 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:36.610985 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.611057 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.611214 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:36.640329  1629 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.655906  1704 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:36.669516  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2134) took 1668 ms. Trace:
I20260709 13:00:36.669984  1704 rpcz_store.cc:276] 0709 13:00:35.000999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.641131 (+1640132us) service_pool.cc:224] Handling call
0709 13:00:36.655645 (+ 14514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.656470 (+   825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.669435 (+ 12965us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:36.671247  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 67615us
W20260709 13:00:36.671765  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 424) took 1985 ms (client timeout 1908 ms). Trace:
I20260709 13:00:36.671917  1701 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:00:36.672369  1701 pending_rounds.cc:85] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 4
W20260709 13:00:36.672159  1707 rpcz_store.cc:269] 0709 13:00:34.685961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.686075 (+   114us) service_pool.cc:224] Handling call
0709 13:00:36.671260 (+1985185us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b5400336b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:36.671689 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127837952}
I20260709 13:00:36.672813  1701 pending_rounds.cc:107] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:00:36.673082  1701 raft_consensus.cc:2889] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:36.678098  1704 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:36.682086  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:36.682881  1635 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:36.684388  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2135) took 1645 ms. Trace:
W20260709 13:00:36.685067  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.685235  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0248677910000197s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
I20260709 13:00:36.684809  1704 rpcz_store.cc:276] 0709 13:00:35.038774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.670784 (+1632010us) service_pool.cc:224] Handling call
0709 13:00:36.677822 (+  7038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:36.678531 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.684337 (+  5806us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:36.686066  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:36.676043  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 105783us
I20260709 13:00:36.687223  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:36.688036  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:36.691540  1704 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:36.692358  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2136) took 1640 ms. Trace:
I20260709 13:00:36.692739  1704 rpcz_store.cc:276] 0709 13:00:35.051572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.686224 (+1634652us) service_pool.cc:224] Handling call
0709 13:00:36.691301 (+  5077us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:36.692067 (+   766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.692299 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:36.567338  4259 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:36.694705  1630 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:36.587376  3734 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:36.558341  4027 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.700681  2317 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:36.681953  4299 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:36.673884  4201 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:36.757560  4258 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.761198  4258 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.760812  4256 raft_consensus.cc:2955] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:36.764415  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2027) took 1261 ms. Trace:
I20260709 13:00:36.765012  1636 rpcz_store.cc:276] 0709 13:00:35.502757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.806958 (+304201us) service_pool.cc:224] Handling call
0709 13:00:36.764270 (+957312us) spinlock_profiling.cc:243] Waited 63.9 ms on lock 0x7b540017a7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:36.764363 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147013504}
I20260709 13:00:36.721089  4270 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:36.768467  4258 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:36.768707  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:36.771355  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:36.771972  2349 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:36.773649  2353 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:36.776944  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.758056  4298 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:00:36.690333  4267 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:36.694039  4089 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.717860  4311 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:36.702626  4259 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.727283  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:36.727815  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1657) took 2150 ms. Trace:
I20260709 13:00:36.728595  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1598) took 3320 ms. Trace:
I20260709 13:00:36.790468  1561 rpcz_store.cc:276] 0709 13:00:34.577569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.062307 (+484738us) service_pool.cc:224] Handling call
0709 13:00:35.894821 (+832514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:35.969114 (+ 74293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.727566 (+758452us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b5400206f50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:36.727739 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1914035456}
I20260709 13:00:36.791393  1708 rpcz_store.cc:276] 0709 13:00:33.408448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.313170 (+904722us) service_pool.cc:224] Handling call
0709 13:00:35.703659 (+1390489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.792156 (+ 88497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.728371 (+936215us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540036e2d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:36.728529 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3196900608}
I20260709 13:00:36.729377  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2107) took 2754 ms. Trace:
I20260709 13:00:36.731354  4268 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:36.732769  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:36.734779  1710 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 7. Ops: []
I20260709 13:00:36.737782  1526 leader_election.cc:400] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4.
I20260709 13:00:36.796617  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:36.797091  1526 leader_election.cc:403] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:36.798059  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 60490.7us
I20260709 13:00:36.799023  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:36.709812  4310 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:36.709407  4257 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:36.742004  4157 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:36.744652  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:36.696324  4113 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:36.779600  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 211793us
I20260709 13:00:36.690392  4252 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
W20260709 13:00:36.758064  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.707340  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:36.707281  2317 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:36.786295  1707 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.787603  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:36.792771  1554 rpcz_store.cc:276] 0709 13:00:33.974964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.543721 (+568757us) service_pool.cc:224] Handling call
0709 13:00:35.476360 (+932639us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:35.507148 (+ 30788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.729103 (+1221955us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54001ce150. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:36.729283 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2107970432}
W20260709 13:00:36.795531  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.825975  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 118797us
I20260709 13:00:36.792402  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:36.825994  2348 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:36.826995  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2142) took 1773 ms. Trace:
I20260709 13:00:36.827227  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:36.801442  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2144) took 1746 ms. Trace:
I20260709 13:00:36.827231  1561 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:36.827457  1707 rpcz_store.cc:276] 0709 13:00:35.053876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.698838 (+1644962us) service_pool.cc:224] Handling call
0709 13:00:36.786050 (+ 87212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.826752 (+ 40702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.826941 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:36.807444  1552 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:36.828088  2349 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:00:36.828774  1710 rpcz_store.cc:276] 0709 13:00:35.054777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.734315 (+1679538us) service_pool.cc:224] Handling call
0709 13:00:36.734607 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.801176 (+ 66569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.801377 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:36.809944  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:36.830515  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:00:36.816112  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 339) took 2128 ms (client timeout 2120 ms). Trace:
W20260709 13:00:36.831239  1557 rpcz_store.cc:269] 0709 13:00:34.687859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.857469 (+169610us) service_pool.cc:224] Handling call
0709 13:00:36.815941 (+1958472us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b54000bf254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:36.816046 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1737453824}
I20260709 13:00:36.832268  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:36.833019  1557 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:36.811517  1555 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
W20260709 13:00:36.832850  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 125668us
W20260709 13:00:36.820030  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.834712  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:36.835345  1555 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:00:36.835587  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:36.836221  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:36.836633  1561 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:36.836561  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:36.837826  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:36.841214  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:36.830542  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:36.806228  4123 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4.
I20260709 13:00:36.838419  2352 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:36.839550  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:36.812721  4122 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:36.842856  2348 raft_consensus.cc:2955] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:36.837324  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:36.847318  2352 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.844636  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:36.844821  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:36.854068  1557 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:36.785832  4267 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:36.853212  1555 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:36.848335  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:36.842880  4233 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:36.895726  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:36.896571  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48344.9us
I20260709 13:00:36.896644  1553 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
W20260709 13:00:36.897912  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.898840  2349 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.832123  4252 consensus_queue.cc:1243] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Peer 95fc6a4cf5874b7eb91280dffcd3b3fb log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:00:36.831787  4189 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:36.879521  4209 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.966s	user 0.006s	sys 0.032s
I20260709 13:00:36.902449  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:36.861920  4157 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:36.903584  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2037) took 1050 ms. Trace:
W20260709 13:00:36.906172  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 58890.4us
I20260709 13:00:36.847002  4310 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:36.894901  4233 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:36.839917  3976 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:36.842968  4192 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:36.846053  3967 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:36.845664  4161 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.849155  4322 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.916801  2352 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:36.901435  2349 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:36.895334  2345 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:36.919873  1555 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:00:36.919137  1628 rpcz_store.cc:276] 0709 13:00:35.853119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.451868 (+598749us) service_pool.cc:224] Handling call
0709 13:00:36.452212 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.452413 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:36.695599 (+243186us) log.cc:844] Serialized 14 byte log entry
0709 13:00:36.903264 (+207665us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:36.903310 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:36.903346 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.903408 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.903519 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":108164,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":3988,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":7605,"thread_start_us":449139,"threads_started":2}
I20260709 13:00:36.955749  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:36.954979  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:36.956373  2349 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:36.923308  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:36.914800  4209 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.003s	user 0.006s	sys 0.032s
I20260709 13:00:36.927819  4123 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:00:36.928910  4325 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:00:36.929042  1636 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:36.930033  1635 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.930446  4315 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.930699  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:36.964972  1627 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:36.932796  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:36.919260  4252 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.934669  4105 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.941941  4027 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:36.910355  4269 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.990329  3993 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:36.990928  4330 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:00:36.916283  4251 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.615s	user 0.015s	sys 0.000s
I20260709 13:00:36.909099  4122 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:36.954954  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2231) took 1278 ms. Trace:
I20260709 13:00:36.992053  1633 rpcz_store.cc:276] 0709 13:00:35.676634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.115256 (+438622us) service_pool.cc:224] Handling call
0709 13:00:36.115541 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.189952 (+ 74411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:36.189998 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:36.190055 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:36.190105 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.954756 (+764651us) spinlock_profiling.cc:243] Waited 74.2 ms on lock 0x7b5400349ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:36.954797 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.954904 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170569088}
I20260709 13:00:36.904214  4201 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:36.975206  4327 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:00:36.960430  1527 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:36.994243  1558 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:36.994619  1555 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.994710  4327 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:36.995182  4327 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:00:36.995263  1455 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 271) took 11802 ms (client timeout 15000 ms). Trace:
I20260709 13:00:36.996331  1558 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:36.996032  4327 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.998250  4267 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:36.998991  4081 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:00:36.999501  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:36.997399  4327 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.948729  4157 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:37.002425  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2038) took 1149 ms. Trace:
I20260709 13:00:37.002811  1630 rpcz_store.cc:276] 0709 13:00:35.853527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.936606 (+ 83079us) service_pool.cc:224] Handling call
0709 13:00:37.002238 (+1065632us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b540010b8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:37.002367 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1702650624}
I20260709 13:00:37.004369  4269 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:37.005553  1629 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:37.009738  4252 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.011893  4252 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:37.001022  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:36.946158  4268 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:37.015079  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 54764.1us
I20260709 13:00:37.002029  4157 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:36.990885  3993 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:37.017851  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 181404us
W20260709 13:00:37.029538  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2029) took 1525 ms (client timeout 1903 ms). Trace:
I20260709 13:00:37.030045  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:00:37.030202  1632 rpcz_store.cc:269] 0709 13:00:35.504435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.620478 (+116043us) service_pool.cc:224] Handling call
0709 13:00:37.029265 (+1408787us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54002dcdd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:37.029431 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741772160}
W20260709 13:00:36.990347  4323 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:00:36.970068  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 453) took 1735 ms. Trace:
I20260709 13:00:36.972751  3967 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:37.028422  1629 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:00:37.050200  1629 pending_rounds.cc:85] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 5
I20260709 13:00:37.050540  1629 pending_rounds.cc:107] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:00:37.050838  1629 raft_consensus.cc:2889] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:36.980484  2317 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:37.034934  4324 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.132s	user 0.003s	sys 0.000s
I20260709 13:00:37.045362  2352 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:37.031266  3734 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.039088  2344 rpcz_store.cc:276] 0709 13:00:35.234688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.236118 (+  1430us) service_pool.cc:224] Handling call
0709 13:00:35.609782 (+373664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.673538 (+ 63756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.946017 (+1272479us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b540054a450. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:36.969967 (+ 23950us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844963584}
W20260709 13:00:37.047350  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:37.027907  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 188402us
I20260709 13:00:37.140419  1601 leader_election.cc:400] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:37.140834  1601 leader_election.cc:403] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:00:36.995671  1455 rpcz_store.cc:269] 0709 13:00:25.192718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.194695 (+  1977us) service_pool.cc:224] Handling call
0709 13:00:36.995191 (+11800496us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:32.079388 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:32.086213 (+  6825us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:32.086417 (+   204us) write_op.cc:435] Acquired schema lock
0709 13:00:32.086444 (+    27us) tablet.cc:662] Decoding operations
0709 13:00:32.094154 (+  7710us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:32.094236 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:32.094266 (+    30us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:00:32.094884 (+   618us) tablet.cc:701] Row locks acquired
0709 13:00:32.094934 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:00:32.095121 (+   187us) write_op.cc:270] Start()
0709 13:00:32.095273 (+   152us) write_op.cc:276] Timestamp: P: 1783602032095054 usec, L: 0
0709 13:00:32.095297 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:00:32.227445 (+132148us) log.cc:844] Serialized 2530 byte log entry
0709 13:00:33.592605 (+1365160us) op_driver.cc:464] REPLICATION: finished
0709 13:00:34.123266 (+530661us) write_op.cc:301] APPLY: starting
0709 13:00:34.131756 (+  8490us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:34.180104 (+ 48348us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:34.202696 (+ 22592us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:34.323566 (+120870us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:34.349541 (+ 25975us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:00:34.349594 (+    53us) write_op.cc:312] APPLY: finished
0709 13:00:34.360168 (+ 10574us) log.cc:844] Serialized 53 byte log entry
0709 13:00:34.474417 (+114249us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:34.491404 (+ 16987us) write_op.cc:454] Released schema lock
0709 13:00:34.495244 (+  3840us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:34.556738 (+ 61494us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:34.559035 (+  2297us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":26640,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":413568,"apply.run_cpu_time_us":22936,"apply.run_wall_time_us":402509,"num_ops":6,"prepare.queue_time_us":253936,"prepare.run_cpu_time_us":127631,"prepare.run_wall_time_us":1006956,"raft.queue_time_us":395328,"raft.run_cpu_time_us":52080,"raft.run_wall_time_us":374819,"replication_time_us":1471467,"thread_start_us":1329785,"threads_started":5,"wal-append.queue_time_us":1008314,"wal-append.run_cpu_time_us":4153,"wal-append.run_wall_time_us":2042264}]]}
I20260709 13:00:37.118026  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:37.142316  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 422) took 3255 ms. Trace:
W20260709 13:00:37.064282  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 83897.8us
W20260709 13:00:37.148286  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.153322  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:37.153761  4105 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:37.153605  3993 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.138916  4251 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.056172  4189 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:36.971719  4161 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.158923  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:37.014954  4192 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:00:37.176065  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:37.191262  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:36.970855  4322 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:36.976488  4315 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.156087  1630 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:37.186940  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2128) took 2240 ms. Trace:
I20260709 13:00:37.139256  4327 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:37.031500  4268 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.170873  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:00:37.217417  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 46660.7us
I20260709 13:00:37.218394  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:37.142855  1705 rpcz_store.cc:276] 0709 13:00:33.886578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.307920 (+421342us) service_pool.cc:224] Handling call
0709 13:00:35.387179 (+1079259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:35.440824 (+ 53645us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:35.440855 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:35.443280 (+  2425us) log.cc:844] Serialized 26 byte log entry
0709 13:00:35.718086 (+274806us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:35.731354 (+ 13268us) log.cc:844] Serialized 14 byte log entry
0709 13:00:35.734099 (+  2745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.739226 (+  5127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:35.925858 (+186632us) raft_consensus.cc:1685] finished
0709 13:00:35.925878 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.735972 (+810094us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b18003d6c00. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:00:37.142061 (+406089us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540016ce50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:37.142239 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7259,"prepare.queue_time_us":12530,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":4405,"raft.queue_time_us":12781,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":11340,"spinlock_wait_cycles":2487117824,"thread_start_us":274447,"threads_started":1,"wal-append.queue_time_us":413983,"wal-append.run_cpu_time_us":1956,"wal-append.run_wall_time_us":1064200}
I20260709 13:00:37.220363  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.221215  1705 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:37.210175  1632 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:37.210506  1633 raft_consensus.cc:2368] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 3. Current term is 4.
I20260709 13:00:37.173605  4233 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:37.197283  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1607) took 2343 ms. Trace:
I20260709 13:00:37.214159  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:37.173230  4275 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.195839  4189 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.188598  4267 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
W20260709 13:00:37.236447  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.226692  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.213058  3993 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:37.241321  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.241443  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:37.154886  3734 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.164474  4027 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:00:37.244699  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1619) took 2376 ms. Trace:
I20260709 13:00:37.244870  1704 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:37.252606  1706 rpcz_store.cc:276] 0709 13:00:34.868500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.930517 (+1062017us) service_pool.cc:224] Handling call
0709 13:00:37.244465 (+1313948us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b540014dcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:37.244637 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276088192}
W20260709 13:00:37.257865  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.241271  1703 rpcz_store.cc:276] 0709 13:00:34.854187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.108159 (+1253972us) service_pool.cc:224] Handling call
0709 13:00:36.396320 (+288161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.397343 (+  1023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.197015 (+799672us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54000cb2d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:37.197208 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632310528}
I20260709 13:00:37.244195  1705 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:37.210114  2352 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:37.243980  4189 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.194044  4251 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.245738  4337 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.269018  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:37.276113  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 67900.3us
I20260709 13:00:37.276701  1706 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:37.277981  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.270226  4089 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:37.271924  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2145) took 2216 ms. Trace:
I20260709 13:00:37.246445  4233 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:37.253064  4175 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:37.242440  3967 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.280669  1708 rpcz_store.cc:276] 0709 13:00:35.055168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.802783 (+1747615us) service_pool.cc:224] Handling call
0709 13:00:36.803209 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:36.844137 (+ 40928us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:36.844181 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:36.878635 (+ 34454us) log.cc:844] Serialized 26 byte log entry
0709 13:00:37.022562 (+143927us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:37.048348 (+ 25786us) log.cc:844] Serialized 14 byte log entry
0709 13:00:37.052740 (+  4392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.052842 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:37.271144 (+218302us) raft_consensus.cc:1685] finished
0709 13:00:37.271723 (+   579us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.271868 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40447,"prepare.queue_time_us":33049,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":81209,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":5795,"spinlock_wait_cycles":141732096,"thread_start_us":97798,"threads_started":1,"wal-append.queue_time_us":301257}
I20260709 13:00:37.275502  4251 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:37.243002  4275 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:37.306233  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.286299  4337 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.319931  4337 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:37.296685  1707 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:37.267479  4338 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:37.264166  4123 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:37.331632  1634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 487) took 2014 ms (client timeout 1825 ms). Trace:
W20260709 13:00:37.332154  1634 rpcz_store.cc:269] 0709 13:00:35.316811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.470100 (+153289us) service_pool.cc:224] Handling call
0709 13:00:35.709132 (+239032us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b5800135698. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:00:37.331380 (+1622248us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54002db4d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:37.331534 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1198574720}
I20260709 13:00:37.334036  4338 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:37.334669  4161 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:37.335140  4338 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.339264  4275 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:37.301990  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:37.256475  4267 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:37.299096  4192 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:37.311655  4079 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.718s	user 0.005s	sys 0.008s
I20260709 13:00:37.343734  4123 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.351294  1706 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
W20260709 13:00:37.343966  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 102589us
I20260709 13:00:37.354897  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:37.346364  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:37.359393  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:37.361856  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:37.363422  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.348869  4267 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:37.367494  3734 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:37.371486  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2146) took 2315 ms. Trace:
I20260709 13:00:37.372025  1708 rpcz_store.cc:276] 0709 13:00:35.056037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.370209 (+2314172us) service_pool.cc:224] Handling call
0709 13:00:37.370780 (+   571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.371036 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.371070 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.371146 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:37.371198 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.371268 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.371411 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.373106  1708 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:37.364500  4337 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:37.364440  4175 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:37.367841  4252 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 13:00:37.378970  4257 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.000s
I20260709 13:00:37.383018  4175 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:37.389919  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 447) took 2681 ms. Trace:
W20260709 13:00:37.392537  4344 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:00:37.390116  4257 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:37.391417  4175 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.391862  4315 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:37.395794  4267 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:37.442925  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 282234us
I20260709 13:00:37.445505  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:37.450733  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:37.454954  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 424822us
I20260709 13:00:37.456743  1526 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:37.377440  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2151) took 2218 ms. Trace:
W20260709 13:00:37.470153  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.469772  1708 rpcz_store.cc:276] 0709 13:00:35.159124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.372779 (+2213655us) service_pool.cc:224] Handling call
0709 13:00:37.372983 (+   204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.377172 (+  4189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.377381 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.464660  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:37.475553  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:37.473440  1708 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:37.486042  4123 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:37.486388  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 432) took 2256 ms. Trace:
I20260709 13:00:37.487325  4123 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:37.485210  3734 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.390411  2347 rpcz_store.cc:276] 0709 13:00:34.708613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.708739 (+   126us) service_pool.cc:224] Handling call
0709 13:00:34.709038 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:35.963025 (+1253987us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b180041f9e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda276d2060 00007fda276d2bfb 00007fda276dd64b
0709 13:00:35.963142 (+   117us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:36.096160 (+133018us) log.cc:844] Serialized 14 byte log entry
0709 13:00:36.390628 (+294468us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:36.392183 (+  1555us) log.cc:844] Serialized 26 byte log entry
0709 13:00:36.392353 (+   170us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:37.013008 (+620655us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b180041f9e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda2b0d23c7 00007fda276e49a7 00007fda276f40bf 00007fda276f4038
0709 13:00:37.389032 (+376024us) log.cc:844] Serialized 14 byte log entry
0709 13:00:37.389395 (+   363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.389522 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:37.389719 (+   197us) raft_consensus.cc:1685] finished
0709 13:00:37.389733 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.389861 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3770,"prepare.queue_time_us":548202,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":22916,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":9389,"spinlock_wait_cycles":10016384,"thread_start_us":719372,"threads_started":3,"wal-append.queue_time_us":412165}
I20260709 13:00:37.488742  4123 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:37.487063  1708 rpcz_store.cc:276] 0709 13:00:35.230263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.471821 (+2241558us) service_pool.cc:224] Handling call
0709 13:00:37.472609 (+   788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.485949 (+ 13340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.486233 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.489758  3734 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.490427  4123 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:37.491991  1708 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:37.492729  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2152) took 1945 ms. Trace:
I20260709 13:00:37.492515  4123 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:37.493310  3734 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:37.484385  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:37.476126  2346 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.495462  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:37.497337  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:37.497889  4089 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:37.499336  4089 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:37.499817  4089 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:37.494261  4123 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:37.394132  4189 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.502269  1708 rpcz_store.cc:276] 0709 13:00:35.547388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.490896 (+1943508us) service_pool.cc:224] Handling call
0709 13:00:37.491511 (+   615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.492496 (+   985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.492661 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.501852  4123 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:37.485965  2345 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:37.506485  4252 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:37.507082  4252 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:37.501044  4089 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.515858  4358 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:37.516408  4358 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:37.522596  4363 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:37.523182  4363 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:37.513020  4123 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:37.524091  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:37.527951  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2133) took 1872 ms. Trace:
I20260709 13:00:37.528836  4233 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:37.528404  1556 rpcz_store.cc:276] 0709 13:00:35.655632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.118772 (+463140us) service_pool.cc:224] Handling call
0709 13:00:36.119850 (+  1078us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:36.126128 (+  6278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:36.126174 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:36.127981 (+  1807us) log.cc:844] Serialized 26 byte log entry
0709 13:00:37.157748 (+1029767us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b1800409420. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:00:37.359446 (+201698us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:37.359492 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.359576 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:37.527291 (+167715us) raft_consensus.cc:1685] finished
0709 13:00:37.527324 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.527714 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105288,"raft.queue_time_us":11181,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":8451840,"thread_start_us":201101,"threads_started":1,"wal-append.queue_time_us":1257862}
I20260709 13:00:37.530786  4337 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:37.531344  4337 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:37.531813  4337 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.531843  4345 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:37.532586  4345 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:37.507550  4252 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:37.534186  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 82019.9us
I20260709 13:00:37.558388  4359 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:37.559052  4359 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:37.560596  4359 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.550834  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:37.563947  4358 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.564632  4355 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:37.566054  4358 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:37.566924  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 67643.4us
I20260709 13:00:37.564073  4367 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.571497  4189 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:37.572165  4367 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.574501  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1605) took 2720 ms. Trace:
I20260709 13:00:37.574930  1701 rpcz_store.cc:276] 0709 13:00:34.853714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.087757 (+1234043us) service_pool.cc:224] Handling call
0709 13:00:36.671544 (+583787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.673796 (+  2252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.574014 (+900218us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b5400336b50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:37.574282 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1300418816}
I20260709 13:00:37.577107  1701 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:37.586311  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 435) took 1768 ms. Trace:
I20260709 13:00:37.524713  4363 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.592419  1701 rpcz_store.cc:276] 0709 13:00:35.817913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.576049 (+1758136us) service_pool.cc:224] Handling call
0709 13:00:37.576560 (+   511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.585935 (+  9375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.586231 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.594081  4363 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.595160  1701 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:37.580454  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 436) took 1339 ms. Trace:
I20260709 13:00:37.598282  1703 rpcz_store.cc:276] 0709 13:00:36.241004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.578714 (+1337710us) service_pool.cc:224] Handling call
0709 13:00:37.579238 (+   524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.579773 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.579808 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.579895 (+    87us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:37.579965 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.580041 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.580204 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.583993  4355 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:37.577155  4367 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:37.588773  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:37.595959  4233 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:37.578208  4345 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.618645  2345 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:37.638000  4089 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:37.638669  2317 leader_election.cc:400] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 3. Current term is 4.
I20260709 13:00:37.639063  2317 leader_election.cc:403] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:37.640002  3963 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 3. Current term is 4.
I20260709 13:00:37.640236  2317 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:37.641242  3963 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:37.641672  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:37.642688  1629 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:37.644536  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:37.643463  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:37.646059  3734 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.646881  3734 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.649843  3967 raft_consensus.cc:2764] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:37.653074  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:37.654915  4345 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:37.656711  1708 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:37.659929  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2143) took 1289 ms. Trace:
I20260709 13:00:37.660450  1560 rpcz_store.cc:276] 0709 13:00:36.369345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.574860 (+205515us) service_pool.cc:224] Handling call
0709 13:00:36.575226 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.575402 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:36.623452 (+ 48050us) log.cc:844] Serialized 14 byte log entry
0709 13:00:37.012376 (+388924us) spinlock_profiling.cc:243] Waited 8.21 ms on lock 0x7b18003d3a80. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda2b0d23c7 00007fda276e49a7 00007fda276f40bf 00007fda276f4038
0709 13:00:37.215412 (+203036us) log.cc:844] Serialized 14 byte log entry
0709 13:00:37.658485 (+443073us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.658583 (+    98us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:37.658694 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.658801 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.658981 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":661335,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":168858,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":7477,"spinlock_wait_cycles":29292800,"thread_start_us":621277,"threads_started":2}
I20260709 13:00:37.665704  4338 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:37.666585  4113 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:00:37.618108  4123 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:37.655145  3967 consensus_queue.cc:1175] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d62ce354d09a4be59c29094425d09bc1" responder_term: 4 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: [3.3-3.3]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:00:37.674517  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.679059  4338 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:37.215783  1709 rpcz_store.cc:276] 0709 13:00:34.946683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.566209 (+1619526us) service_pool.cc:224] Handling call
0709 13:00:36.566520 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.566764 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:36.777835 (+211071us) log.cc:844] Serialized 14 byte log entry
0709 13:00:37.186596 (+408761us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.186651 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:37.186694 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.186753 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.186880 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":169004,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":5455,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8207,"thread_start_us":516525,"threads_started":2}
I20260709 13:00:37.684782  3993 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:37.696622  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:37.697336  1628 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:37.702260  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.703660  1709 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:37.685000  3970 consensus_queue.cc:1059] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:00:37.685567  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:37.684790  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 228378us
I20260709 13:00:37.706146  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2154) took 2138 ms. Trace:
I20260709 13:00:37.719853  1708 rpcz_store.cc:276] 0709 13:00:35.547982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.509012 (+1961030us) service_pool.cc:224] Handling call
0709 13:00:37.562243 (+ 53231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.672129 (+109886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.686503 (+ 14374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.724535  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:37.668901  3734 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:37.734302  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:37.739996  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 97435.6us
I20260709 13:00:37.740705  4089 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.741076  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:37.741241  1602 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:37.742283  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:37.743024  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.734189  4175 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.755995  1628 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
W20260709 13:00:37.771018  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.785287  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.795029  4252 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.796233  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:37.742420  4233 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:00:37.797369  4123 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:37.800215  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2139) took 2747 ms. Trace:
I20260709 13:00:37.801622  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:37.803120  1636 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:37.812417  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:37.813024  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 45668.3us
I20260709 13:00:37.796707  4337 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:37.770246  4359 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:37.793282  4331 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.847s	user 0.002s	sys 0.004s
I20260709 13:00:37.785619  4338 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:37.818358  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:37.812489  1704 rpcz_store.cc:276] 0709 13:00:35.052628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.693645 (+1641017us) service_pool.cc:224] Handling call
0709 13:00:37.244631 (+550986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.252888 (+  8257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.799789 (+546901us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b540014dcd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:37.800014 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263214848}
I20260709 13:00:37.819658  2346 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:37.816620  1708 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
W20260709 13:00:37.824244  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 87621.8us
W20260709 13:00:37.830662  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 152183us
I20260709 13:00:37.822985  1602 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:37.818953  4338 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.839133  4370 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:37.839336  4338 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:37.839701  4370 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.840130  4370 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:37.835353  1627 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:37.842212  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:37.835824  1636 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:37.842625  1635 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:37.836391  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.843190  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:37.843935  2353 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.831092  3970 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.844741  3970 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:37.845911  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.846031  4267 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:37.846572  4267 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.847200  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:37.847389  3970 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:37.848953  3970 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.843798  1710 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:37.849654  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:37.850455  2350 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:37.851336  1631 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:37.852021  4233 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:37.853144  4369 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:00:37.854836  4175 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:37.852640  3970 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.861673  1633 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
W20260709 13:00:37.862015  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 119913us
I20260709 13:00:37.862915  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:37.863323  3970 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:37.866176  4368 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:00:37.876179  4252 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:37.876987  4252 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:37.880012  4259 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:37.877310  4355 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:00:37.881415  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1635) took 1057 ms. Trace:
I20260709 13:00:37.886089  1706 rpcz_store.cc:276] 0709 13:00:36.823420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.277640 (+454220us) service_pool.cc:224] Handling call
0709 13:00:37.880872 (+603232us) spinlock_profiling.cc:243] Waited 72.4 ms on lock 0x7b540016c454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:37.881118 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166524800}
I20260709 13:00:37.889017  4322 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:37.891309  4268 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:37.911103  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.912380  1705 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:37.892825  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 426) took 3205 ms. Trace:
I20260709 13:00:37.913589  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.913647  1702 rpcz_store.cc:276] 0709 13:00:34.686805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.049436 (+1362631us) service_pool.cc:224] Handling call
0709 13:00:36.049844 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:36.050188 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:36.050229 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:36.050740 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 13:00:36.571499 (+520759us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b1800476e60. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:00:36.781882 (+210383us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:36.781922 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.781991 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:37.792286 (+1010295us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:37.892380 (+100094us) raft_consensus.cc:1685] finished
0709 13:00:37.892406 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.892610 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10345,"raft.queue_time_us":2879,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":21127,"spinlock_wait_cycles":10094208,"thread_start_us":210186,"threads_started":1,"wal-append.queue_time_us":764057}
W20260709 13:00:37.920835  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:37.921209  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.922976  1706 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:37.924016  4113 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.924649  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:37.925369  1557 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:37.926614  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:37.926875  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:37.927424  1561 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:37.927641  1554 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:37.928779  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:37.930253  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:37.930976  1553 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:37.931174  4079 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:37.931759  4079 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:37.935992  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:37.937109  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45965.9us
I20260709 13:00:37.937506  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 723) took 1177 ms. Trace:
I20260709 13:00:37.937994  2346 rpcz_store.cc:276] 0709 13:00:36.760074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.764823 (+  4749us) service_pool.cc:224] Handling call
0709 13:00:37.937221 (+1172398us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b54004ecad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:37.937402 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1624690432}
I20260709 13:00:37.933107  4337 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:37.906267  1704 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:37.907437  4105 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.940004  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:37.940465  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:37.940332  4105 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.940824  1561 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:37.941277  1553 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:37.941980  4363 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:37.942653  4105 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:37.908298  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.942901  4189 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:37.943509  1709 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:37.943770  4189 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:37.909986  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:37.944149  4189 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:37.911603  1675 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:37.944767  1710 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:37.945197  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:37.945168  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:37.946249  1557 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:37.946457  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 81226.6us
I20260709 13:00:37.893221  4122 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:37.947273  4122 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:37.928807  4267 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:37.948544  1560 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:00:37.948947  1560 pending_rounds.cc:85] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 2
I20260709 13:00:37.947321  4337 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:37.949667  4337 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:37.949225  1560 pending_rounds.cc:107] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:37.950050  4337 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.942376  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:37.904359  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.950148  1560 raft_consensus.cc:2889] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:37.943349  4363 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:37.940055  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:37.957357  2353 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
W20260709 13:00:37.967139  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 117710us
I20260709 13:00:37.969345  1554 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
W20260709 13:00:37.969647  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 211732us
W20260709 13:00:37.971810  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:37.972988  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.933061  4079 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:37.982714  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:37.983415  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:37.990200  1557 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:37.997066  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:38.000725  1553 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
W20260709 13:00:38.012447  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 52432.8us
I20260709 13:00:37.986594  3967 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:37.979960  4105 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:00:38.043387  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:38.020197  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 69652.5us
I20260709 13:00:38.020699  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:38.021773  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:38.040657  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 129229us
I20260709 13:00:38.016113  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:38.021251  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:38.045832  1627 raft_consensus.cc:2410] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:38.019914  1557 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:38.015445  4252 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.020464  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:38.027554  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:38.031180  2346 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:38.034356  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 94463.2us
W20260709 13:00:37.993299  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 176440us
I20260709 13:00:38.066646  4252 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.066722  1554 raft_consensus.cc:2376] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4. Re-sending same reply.
I20260709 13:00:38.038517  4251 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.037905  4161 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.040877  3993 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:38.055346  4189 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.042915  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:38.072851  1553 raft_consensus.cc:2368] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 3. Current term is 4.
I20260709 13:00:38.020102  4363 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.024788  4079 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.040323  3970 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.985858  4338 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:38.067191  4252 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.046118  4209 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:38.048899  4370 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:38.107837  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 114864us
I20260709 13:00:38.100816  3993 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:38.117655  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 75036.9us
I20260709 13:00:38.101200  4363 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.088088  4089 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.034708  4256 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 13:00:38.041693  3734 consensus_queue.cc:1059] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:00:38.136433  1633 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:38.129794  1527 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:38.133821  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:38.134372  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:38.139639  4345 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.140982  4346 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:38.141536  4346 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:38.142094  4371 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:38.142576  4346 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.143164  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:38.145948  4122 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.146778  4380 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.147569  4122 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.147358  4380 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:38.149628  4268 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.150178  4381 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:38.150779  4381 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:38.150972  4338 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:38.152848  4381 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:38.153693  4345 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:38.154706  4372 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.116s	user 0.000s	sys 0.003s
I20260709 13:00:38.155879  4345 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:38.157043  4380 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:38.145393  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:38.139676  3993 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:38.159521  4359 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:38.159889  4359 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:38.160732  4359 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:38.118737  4322 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:38.161839  4268 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.162369  4322 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.165019  4322 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:38.168463  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:38.171020  2350 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:00:38.172123  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:38.172260  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:38.172955  2347 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:38.174005  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:38.175329  4380 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.176183  4310 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:38.088444  3967 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:38.182333  4327 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:38.182790  4359 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.182899  4327 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:38.183907  3967 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:38.180863  4268 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:38.180972  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:38.173873  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:38.188424  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 59708.7us
I20260709 13:00:38.184495  4327 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.169317  1602 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:38.067708  4251 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.195573  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:38.198385  1552 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:38.199563  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:38.104467  4337 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:38.202029  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 158726us
I20260709 13:00:38.202915  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:38.205412  1557 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:38.206686  1634 raft_consensus.cc:2410] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:38.130201  4363 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.208087  1556 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:38.208026  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:38.208868  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:38.105825  3970 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.210465  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:38.203897  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:38.089443  4161 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.217046  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:38.210016  4374 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:38.229463  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:38.267444  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.210564  4105 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.269361  1708 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:38.224822  4327 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:38.295253  1706 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:38.268936  4362 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.305368  1710 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:00:38.306408  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 119310us
W20260709 13:00:38.306453  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 167601us
I20260709 13:00:38.269822  4189 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:38.290745  4313 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:38.316497  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47510us
I20260709 13:00:38.304040  4089 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:38.329447  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 126593us
I20260709 13:00:38.302423  4379 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:38.307816  4362 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.291457  4268 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 13:00:38.334159  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 134279us
I20260709 13:00:38.335836  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:38.337463  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 163807us
I20260709 13:00:38.311390  4105 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.341362  4373 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:38.342749  4388 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:38.343231  4089 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:38.343684  4105 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:38.345052  4081 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:38.345630  4081 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.346112  4081 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:38.305915  4311 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.228s
I20260709 13:00:38.327034  4259 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.346422  4089 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.339118  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:38.347748  1709 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:38.353492  1559 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:38.355424  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.360921  1709 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:38.351883  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:38.363389  1633 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20260709 13:00:38.366101  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.335101  4362 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:38.420055  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 55419.5us
W20260709 13:00:38.371418  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44852.5us
I20260709 13:00:38.434378  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:38.435189  1559 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
W20260709 13:00:38.437031  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 73874.5us
I20260709 13:00:38.419621  4366 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.371577  3963 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 13:00:38.462848   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:00:38.501830  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 154831us
I20260709 13:00:38.451046  4367 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:38.371294  4388 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 13:00:38.512054  1675 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:38.466781  4337 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:38.466168  4370 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:38.457212  4259 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.506464  4256 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:38.466228  4359 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.477392  4389 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.456358  4313 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
W20260709 13:00:38.539644  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 188958us
I20260709 13:00:38.508783  4322 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 13:00:38.539893  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 169518us
W20260709 13:00:38.539076  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 106015us
I20260709 13:00:38.527123  3970 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:38.580147  4275 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.580425  4398 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.078s
I20260709 13:00:38.580914  4275 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:38.583693  4391 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 13:00:38.585662  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 116825us
W20260709 13:00:38.586079  4397 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:00:38.587311  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:38.588022  4161 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:38.590294  4395 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:00:38.594044  4390 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:00:38.600098  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:38.601014  1633 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:38.602878  4396 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:00:38.603223  4175 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:38.604542  4392 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:00:38.606097  4122 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.606664  4122 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:38.608675  4122 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:38.610169  2318 leader_election.cc:400] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 3 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 3. Current term is 4.
I20260709 13:00:38.609954  4105 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:38.610532  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:38.611438  1628 raft_consensus.cc:2410] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:38.612182  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:38.613807  2350 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:38.615170  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:38.616108  2348 raft_consensus.cc:2410] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:00:38.616777  4252 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.000s
I20260709 13:00:38.618856  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:38.622829  1633 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
W20260709 13:00:38.623955  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 121700us
I20260709 13:00:38.529964  4256 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.529421  4373 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:38.540012  4385 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:00:38.635262  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:38.617864  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:38.639531  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:38.651415  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:38.659581  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:38.563074  4337 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.681309  1629 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:38.617264  4252 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.530752  4377 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.290s
W20260709 13:00:38.688520  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 62523.7us
W20260709 13:00:38.730989  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 200129us
I20260709 13:00:38.697108  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:38.731314  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:38.563925  4367 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.618031  4081 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.752316  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:38.685525  4105 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.562935  4359 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.622227  4363 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:38.653679  4366 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:38.681010  4322 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.639496  3970 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:38.697173  4388 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.697098  4275 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:38.771384  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 106299us
I20260709 13:00:38.752558  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:38.776778  1631 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:38.776669  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:38.777141  1636 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:00:38.697293  3734 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:38.775486  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:38.743832  4175 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:38.778187  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 81140.9us
I20260709 13:00:38.786201  2349 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:38.788166  1628 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:38.796096  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:38.681821  3963 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.629915  4256 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:38.744398  2345 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:38.786404  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:38.686115  4370 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.800805  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:38.802685  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:38.803591  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:38.622697  4389 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.802912  4370 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.810741  1635 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:38.810981  4367 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:38.812041  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:38.812736  4405 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:38.813222  4405 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.813630  4405 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.806100  4370 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.818849  4081 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:38.802704  4275 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.822009  4275 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.822487  4275 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.770233  4373 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:38.812110  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:38.772500  3970 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.846170  1553 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:00:38.776162  4388 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.753304  4337 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:38.873039  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 61021.8us
I20260709 13:00:38.875346  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:38.877939  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.880100  1629 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:38.881407  1628 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:38.847268  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:38.770143  4105 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.827391  4362 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.828838  4379 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:38.927259  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:38.776181  4322 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:38.876067  4079 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.774s	user 0.004s	sys 0.007s
I20260709 13:00:38.834419  4381 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:00:38.803409  2344 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:38.828387  4407 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.852849  4388 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:38.934850  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 59632us
I20260709 13:00:38.939646  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:38.936071  3993 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:38.873895  4377 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.852612  3970 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.937963  4406 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:38.958139  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 110996us
I20260709 13:00:38.882900  4313 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:38.926204  4189 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.611s	user 0.007s	sys 0.004s
I20260709 13:00:38.850585  4367 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.850705  4389 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:38.926139  4310 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:38.945966  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:38.960433  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:38.960696  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:38.962105  4259 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:38.962666  4327 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.660s	user 0.010s	sys 0.001s
I20260709 13:00:38.963039  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:38.938956  4379 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.965574  1558 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:38.981528  1557 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:38.873656  4122 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:00:38.802837  3963 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.957993  4123 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.967921  4081 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:38.968724  4313 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.969575  4079 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:00:39.014011  4410 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.033665  4410 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:39.014139  4322 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:39.015264  3993 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.024434  4373 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:38.992633  1559 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:39.024700  4123 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.037547  1559 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:38.995258  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:38.979957  4310 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.002076  4415 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.003525  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:39.004958  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2248) took 1303 ms. Trace:
I20260709 13:00:39.025159  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:39.005641  1627 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:39.006916  4367 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:38.973273  4406 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.035807  3993 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:38.992087  4252 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:39.011704  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:39.029654  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:39.050534  4310 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:39.039515  1560 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:39.014729  4359 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:39.027834  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:39.027094  2345 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:39.025768  4081 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:00:39.051658  2352 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:39.038892  4123 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:39.028110  4105 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:39.025941  4259 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:00:39.030401  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:39.026032  4313 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.027102  4189 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:39.028362  4414 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:39.016566  4362 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:39.032502  4175 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:39.010427  4405 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:39.061956  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 304679us
I20260709 13:00:39.064397  4366 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:39.067355  2347 raft_consensus.cc:2410] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:00:39.068558  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 43522.3us
W20260709 13:00:39.075083  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 298455us
I20260709 13:00:39.078193  1558 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:39.021994  3963 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:38.947275  4407 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.069646  4327 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:39.088624  1557 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:38.990839  4377 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.058995  1634 rpcz_store.cc:276] 0709 13:00:37.701806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.737164 (+ 35358us) service_pool.cc:224] Handling call
0709 13:00:39.004656 (+1267492us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b5400344254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:39.004860 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":870837632}
I20260709 13:00:39.113942  2344 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:39.139745  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 262494us
I20260709 13:00:39.151314  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:39.114879  4359 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.114283  4189 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:39.151314  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:39.066857  4415 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.173008  1631 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:39.140432  4414 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:39.151856  4327 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:39.181313  4381 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:39.278352  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 127145us
I20260709 13:00:39.136713  4370 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.151513  4275 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:39.286700  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 135497us
I20260709 13:00:39.220717  3734 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.215574  4337 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:39.283020  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:39.267398  4327 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:39.227494  4401 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:39.319726  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:39.309968  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.335985  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:39.336530  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:39.221220  4359 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.289732  3967 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:39.347424  1636 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:39.347875  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:39.349294  1560 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:39.330338  4403 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.288043  4381 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:39.377002  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 94140.3us
I20260709 13:00:39.377758  4428 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.348644  3967 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:39.378552  4421 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:00:39.367807  4113 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.355444  3963 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:39.381763  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:39.342571  4408 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:39.355593  4424 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:39.350191  4379 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.362057  4311 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.392012  4259 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.384271  4405 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:39.401075  3967 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.329707  4189 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.409628  3993 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:39.406426  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:39.409278  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 449532us
I20260709 13:00:39.383342  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:39.378456  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:39.399148  4367 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:39.330011  3734 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.372660  4373 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:00:39.433745  1630 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:00:39.421396  4322 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.413697  3963 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.400679  4428 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.431391  4259 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.427887  4420 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.501677  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:39.424361  4424 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.534650  4373 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:39.519622  4175 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:39.398556  4366 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.433538  4408 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.500936  4189 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:39.510813  4406 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:39.398236  4403 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.482957  4105 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.499660  4362 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.491770  3963 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.487205  4113 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:39.488636  4346 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:39.562011  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:39.494822  4367 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:39.418694  4414 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.555668  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:39.427109  4311 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.544714  4081 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:39.593781  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:39.596168  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:39.660149  1705 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:39.579851  4238 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.588886  4370 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:39.590098  4407 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:39.595305  4373 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:39.624402  2344 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:39.625903  4259 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:39.627461  4113 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.628387  4346 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.598353  4322 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.008s
W20260709 13:00:39.631822  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 133902us
I20260709 13:00:39.717527  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:39.718088  4427 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.632701  1630 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:00:39.641285  4409 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.314s	user 0.000s	sys 0.002s
W20260709 13:00:39.642190  4422 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.163s	user 0.001s	sys 0.001s
I20260709 13:00:39.641798  4367 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.642946  4432 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:39.643188  4426 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:00:39.616875  4105 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.689849  4081 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:39.644510  4434 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:00:39.644826  4430 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:00:39.645804  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:39.595853  4420 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.647271  4157 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.649793  3734 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:39.605634  4415 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:39.652709  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:39.654592  4251 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:39.602658  4418 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:39.655341  4403 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:39.655620  4412 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.345s	user 0.000s	sys 0.002s
I20260709 13:00:39.656620  3963 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.657778  4423 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:39.534687  3993 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:00:39.612130  4377 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:39.600456  4175 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.685385  4436 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:39.730237  4386 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.407s	user 0.001s	sys 0.010s
I20260709 13:00:39.630576  4362 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.688140  4238 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:39.734714  1561 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:40.013038  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 740) took 1799 ms. Trace:
I20260709 13:00:40.096266  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:40.015000  1703 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:00:39.893244  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1641) took 1785 ms. Trace:
I20260709 13:00:39.999598  1634 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:40.010977  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:40.088210  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:39.899632  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.080164  1526 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.0.107.254:43857, user_credentials={real_user=slave}} blocked reactor thread for 80419.7us
I20260709 13:00:40.059410  1705 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:40.110092  1710 rpcz_store.cc:276] 0709 13:00:38.108262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.108394 (+   132us) service_pool.cc:224] Handling call
0709 13:00:38.129317 (+ 20923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:38.305812 (+176495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.892863 (+1587051us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54000cbcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:39.893151 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350519680}
W20260709 13:00:39.766812  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.112267  1557 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:40.114427  1555 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:40.110963  1560 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.708280  4123 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.109932  2349 rpcz_store.cc:276] 0709 13:00:38.213547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.213682 (+   135us) service_pool.cc:224] Handling call
0709 13:00:38.242511 (+ 28829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:38.798821 (+556310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.012630 (+1213809us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54004f4a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:40.012914 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1234706688}
W20260709 13:00:39.951063  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.139308  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:40.088403  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1646) took 1893 ms (client timeout 2094 ms). Trace:
W20260709 13:00:40.159278  1706 rpcz_store.cc:269] 0709 13:00:38.194601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.194730 (+   129us) service_pool.cc:224] Handling call
0709 13:00:40.088151 (+1893421us) spinlock_profiling.cc:243] Waited 94.8 ms on lock 0x7b5400077954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:40.088309 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218120448}
I20260709 13:00:39.794690  4432 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:39.707078  4381 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:40.011823  4209 raft_consensus.cc:3055] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:40.141438  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:40.008870  4408 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:39.736002  4081 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:40.074308  4113 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:40.141009  1633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.008s	sys 0.004s
W20260709 13:00:39.849422  4422 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:00:40.010394  4443 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
W20260709 13:00:40.013711  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1643) took 1826 ms (client timeout 1850 ms). Trace:
I20260709 13:00:39.734030  4105 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:40.094159  4398 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:39.886848  4384 raft_consensus.cc:1027] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: attempt to promote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:00:39.955421  4268 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:39.848752  4445 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.109447  4414 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:39.736636  4256 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.856s	user 0.005s	sys 0.008s
I20260709 13:00:39.728758  4427 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:39.737155  4373 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:40.163537  1705 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:40.441430  4209 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:40.441784  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 454) took 1025 ms. Trace:
I20260709 13:00:40.444114  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2044) took 1843 ms. Trace:
I20260709 13:00:40.444067  1705 rpcz_store.cc:276] 0709 13:00:39.416418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.138931 (+722513us) service_pool.cc:224] Handling call
0709 13:00:40.441674 (+302743us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:40.444864  1627 rpcz_store.cc:276] 0709 13:00:38.600364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.605592 (+  5228us) service_pool.cc:224] Handling call
0709 13:00:39.005387 (+399795us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:39.061746 (+ 56359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.443803 (+1382057us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b5400344250. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:40.443993 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":918667008}
I20260709 13:00:40.443470  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 452) took 1074 ms. Trace:
I20260709 13:00:40.447642  1702 rpcz_store.cc:276] 0709 13:00:39.372613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.134823 (+762210us) service_pool.cc:224] Handling call
0709 13:00:40.443391 (+308568us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:40.450827  4209 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:40.453934  4209 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:40.453934  4345 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:40.407114  4424 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:40.464545  4362 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:40.465277  4458 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:00:40.429378  4310 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.338s	user 0.017s	sys 0.001s
W20260709 13:00:40.332499  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44312.6us
W20260709 13:00:40.503364  1709 rpcz_store.cc:269] 0709 13:00:38.187705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.193243 (+  5538us) service_pool.cc:224] Handling call
0709 13:00:40.013437 (+1820194us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54001865d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:40.013612 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313978752}
W20260709 13:00:40.504601  1633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.015s	sys 0.010s
I20260709 13:00:40.505724  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2271) took 1629 ms. Trace:
I20260709 13:00:40.506364  1633 rpcz_store.cc:276] 0709 13:00:38.876613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.399434 (+522821us) service_pool.cc:224] Handling call
0709 13:00:39.416618 (+ 17184us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.432069 (+ 15451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:39.432117 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:39.475759 (+ 43642us) log.cc:844] Serialized 26 byte log entry
0709 13:00:40.505177 (+1029418us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:40.505235 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.505355 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:40.505417 (+    62us) raft_consensus.cc:1685] finished
0709 13:00:40.505438 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:40.505644 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8994,"raft.queue_time_us":9803,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":45563,"thread_start_us":969642,"threads_started":1,"wal-append.queue_time_us":836955}
I20260709 13:00:40.152746  4436 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:40.516021  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 450) took 1156 ms. Trace:
W20260709 13:00:40.229413  1676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.006s	sys 0.026s
W20260709 13:00:40.595314  1676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.016s	sys 0.033s
W20260709 13:00:40.596133  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 878676us
I20260709 13:00:40.597131  1676 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:40.537232  1560 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:40.599635  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1693) took 1164 ms. Trace:
I20260709 13:00:40.129343  4311 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:40.518765  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:40.467854  3970 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.491s	user 0.008s	sys 0.008s
W20260709 13:00:40.324165  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.000s	sys 0.022s
W20260709 13:00:40.604386  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.002s	sys 0.024s
W20260709 13:00:40.604528  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.013s	sys 0.053s
W20260709 13:00:40.605089  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.013s	sys 0.053s
I20260709 13:00:40.262009  4322 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:40.387390  3967 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.907s	user 0.010s	sys 0.005s
W20260709 13:00:40.164875  4313 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.029s	user 0.009s	sys 0.006s
I20260709 13:00:40.518078  4362 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:40.307343  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:40.409211  4449 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:40.586376  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.515730  4401 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.181s	user 0.009s	sys 0.004s
I20260709 13:00:40.330437  4452 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.151440  4275 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:40.141436  4446 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:40.465868  4458 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:40.142791  4367 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:40.590940  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 104413us
I20260709 13:00:40.514835  4420 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:40.514029  4407 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.891027  4359 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:40.601325  1560 rpcz_store.cc:276] 0709 13:00:39.434998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.073825 (+638827us) service_pool.cc:224] Handling call
0709 13:00:40.599542 (+525717us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:40.503090  3963 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:40.513540  4459 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:40.605011  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 1.01122e+06us
I20260709 13:00:40.167724  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:40.574121  1527 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.004s	sys 0.007s
I20260709 13:00:40.154512  4123 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:40.519173  4337 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:40.570470  4432 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:40.608326  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.608629  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:40.457554  4345 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:40.465713  4256 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:40.167925  4428 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:40.584920  4381 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.517102  4454 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:40.501149  4414 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:40.593822  1707 rpcz_store.cc:276] 0709 13:00:39.359751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.359900 (+   149us) service_pool.cc:224] Handling call
0709 13:00:39.379900 (+ 20000us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.380310 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:39.380477 (+   167us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:39.381111 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 13:00:39.442394 (+ 61283us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:39.894265 (+451871us) log.cc:844] Serialized 14 byte log entry
0709 13:00:39.894563 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.894663 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:40.515662 (+620999us) raft_consensus.cc:1685] finished
0709 13:00:40.515687 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:40.515834 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":719186,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":84,"raft.queue_time_us":272493,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":26058,"spinlock_wait_cycles":36988672,"thread_start_us":461366,"threads_started":2,"wal-append.queue_time_us":215589}
I20260709 13:00:40.501843  4268 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:40.627264  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:40.504379  4310 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:40.165037  4157 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:40.159193  4423 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:40.158542  4418 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:40.520162  4445 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:41.406009  1527 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.493s	user 0.012s	sys 0.066s
I20260709 13:00:41.147614  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:41.407436  4367 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:41.408042  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.849622  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2267) took 2063 ms (client timeout 1790 ms). Trace:
W20260709 13:00:41.414000  1631 rpcz_store.cc:269] 0709 13:00:38.786583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.798273 (+ 11690us) service_pool.cc:224] Handling call
0709 13:00:39.003331 (+205058us) spinlock_profiling.cc:243] Waited 8.48 ms on lock 0x7b5800135698. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:00:40.849333 (+1846002us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b540010dbd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:40.849502 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282158720}
I20260709 13:00:41.415454  4423 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.253947  4252 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.416196  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:40.625213  4449 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.419682  4310 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.420673  4310 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:41.457588  4310 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.368436  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 448) took 2011 ms. Trace:
I20260709 13:00:41.464195  4337 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:40.633143  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1689) took 1225 ms. Trace:
I20260709 13:00:41.388723  4238 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:41.167315  4327 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.818s	user 0.015s	sys 0.002s
I20260709 13:00:41.411626  4381 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.409422  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:40.897854  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 297292us
I20260709 13:00:41.466145  4327 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:41.466651  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1688) took 2059 ms (client timeout 2230 ms). Trace:
W20260709 13:00:41.467097  1560 rpcz_store.cc:269] 0709 13:00:39.407092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.409088 (+2001996us) service_pool.cc:224] Handling call
0709 13:00:41.466572 (+ 57484us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:41.467860  4381 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.468200  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:41.468367  4327 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.468474  1631 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:41.412722  1633 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:40.611142  3967 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:00:41.468962  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1700) took 1306 ms (client timeout 1582 ms). Trace:
I20260709 13:00:40.611016  4313 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:41.469401  1560 rpcz_store.cc:269] 0709 13:00:40.162595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.467866 (+1305271us) service_pool.cc:224] Handling call
0709 13:00:41.468907 (+  1041us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:41.470122  1701 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:41.470049  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:41.470358  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:41.006176  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.470798  3967 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.470836  1633 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:41.471942  4313 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.416795  4252 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.408125  4367 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.417866  3963 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:40.633001  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:40.702402  4113 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.421640  1627 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:41.423202  4449 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:41.410301  4447 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.002s	user 0.012s	sys 0.000s
I20260709 13:00:41.423472  4268 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.423879  4359 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:41.424928  4444 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.823s	user 0.009s	sys 0.000s
I20260709 13:00:41.425843  4435 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:41.485179  4439 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.799s	user 0.000s	sys 0.011s
I20260709 13:00:41.426846  4454 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:40.647051  1630 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.909s	user 0.010s	sys 0.003s
I20260709 13:00:41.427999  4407 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:41.428619  4363 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.429044  1557 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:41.431001  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2266) took 2660 ms. Trace:
W20260709 13:00:41.431957  1561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 374) took 2007 ms (client timeout 1685 ms). Trace:
W20260709 13:00:41.433531  4393 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 2.965s	user 0.000s	sys 0.014s
I20260709 13:00:41.435212  4123 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:41.435470  4404 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 2.671s	user 0.004s	sys 0.035s
I20260709 13:00:41.436525  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2262) took 2682 ms. Trace:
I20260709 13:00:41.437624  4122 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:41.033852  4448 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.439185  1558 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:41.439508  4459 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.441110  4418 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.442996  4346 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.314081  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1679) took 3203 ms. Trace:
I20260709 13:00:41.445176  4157 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:40.747957  4457 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:41.032733  4433 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:41.445961  4432 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.609644  3970 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:40.610841  4322 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:41.448606  4345 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.449674  4458 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:40.622080  4175 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.450627  4414 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.033200  4466 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:41.326632  4394 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 2.849s	user 0.007s	sys 0.006s
W20260709 13:00:41.035967  4453 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.629s	user 0.001s	sys 0.000s
I20260709 13:00:40.631265  4455 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.223799  4251 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:41.453037  4445 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:40.629972  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:41.456260  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.623220  1709 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:41.033349  4462 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.459115  4161 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:41.637990  4161 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.638475  4161 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.630784  4457 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.641563  4238 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:41.642036  4457 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.644378  4457 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:41.459309  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 850692us
I20260709 13:00:41.650126  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 732) took 3483 ms. Trace:
I20260709 13:00:41.622382  4455 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.650579  2346 rpcz_store.cc:276] 0709 13:00:38.166585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.166733 (+   148us) service_pool.cc:224] Handling call
0709 13:00:38.167095 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:38.167525 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:38.167557 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:38.167998 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:00:38.330666 (+162668us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:38.330704 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.330756 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.313635 (+2982879us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:41.649725 (+336090us) raft_consensus.cc:1685] finished
0709 13:00:41.649751 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.650008 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2160,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"spinlock_wait_cycles":3083008,"thread_start_us":162289,"threads_started":1,"wal-append.queue_time_us":308744}
I20260709 13:00:41.652854  4455 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:41.653628  1708 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:41.654066  4337 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:40.623873  4362 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.655978  4362 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:40.623668  4401 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:40.629657  4452 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.403909  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2166) took 2442 ms. Trace:
I20260709 13:00:40.703379  4446 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.464488  1703 rpcz_store.cc:276] 0709 13:00:39.357256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.357595 (+   339us) service_pool.cc:224] Handling call
0709 13:00:40.014616 (+657021us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:40.129128 (+114512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.368023 (+1238895us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54001865d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:41.368311 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455056256}
I20260709 13:00:41.014189  4366 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.465579  1555 rpcz_store.cc:276] 0709 13:00:39.407439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.162251 (+754812us) service_pool.cc:224] Handling call
0709 13:00:40.633070 (+470819us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:41.471984  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2275) took 1997 ms. Trace:
I20260709 13:00:41.671284  1633 rpcz_store.cc:276] 0709 13:00:39.474371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.469699 (+1995328us) service_pool.cc:224] Handling call
0709 13:00:41.471920 (+  2221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:41.475193  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:41.468802  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:41.672556  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:41.483364  1601 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.005s	sys 0.014s
W20260709 13:00:41.497674  1710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.907s	user 0.037s	sys 0.107s
W20260709 13:00:41.673365  1601 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.955s	user 0.005s	sys 0.015s
W20260709 13:00:41.673578  1710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.037s	sys 0.107s
I20260709 13:00:41.513566  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2278) took 1059 ms. Trace:
W20260709 13:00:41.674789  4404 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:00:41.669521  4462 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.676038  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1644) took 3482 ms. Trace:
I20260709 13:00:41.386193  4379 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.147102  4311 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:41.677846  4465 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:00:41.676473  1704 rpcz_store.cc:276] 0709 13:00:38.193807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.193974 (+   167us) service_pool.cc:224] Handling call
0709 13:00:38.211287 (+ 17313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:38.268992 (+ 57705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:38.269023 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:38.302156 (+ 33133us) log.cc:844] Serialized 26 byte log entry
0709 13:00:38.466146 (+163990us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:38.516415 (+ 50269us) log.cc:844] Serialized 14 byte log entry
0709 13:00:38.685627 (+169212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.685725 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.370160 (+2684435us) spinlock_profiling.cc:243] Waited 4.69 ms on lock 0x7b180040dec0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:00:41.434378 (+ 64218us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:41.675669 (+241291us) raft_consensus.cc:1685] finished
0709 13:00:41.675696 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.675904 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22927,"prepare.queue_time_us":53066,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":85856,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":16309,"spinlock_wait_cycles":15762560,"thread_start_us":264871,"threads_started":2,"wal-append.queue_time_us":233057}
I20260709 13:00:41.678133  4311 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:41.521135  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 446) took 2159 ms (client timeout 2550 ms). Trace:
I20260709 13:00:41.679153  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:41.679105  1701 rpcz_store.cc:269] 0709 13:00:39.356554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.368622 (+ 12068us) service_pool.cc:224] Handling call
0709 13:00:41.512476 (+2143854us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b5400246654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:41.515782 (+  3306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4728450304}
I20260709 13:00:41.679958  1704 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:41.523922  1630 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:41.681089  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:41.681802  1630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2274) took 2279 ms (client timeout 2190 ms). Trace:
I20260709 13:00:41.527482  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:41.682260  1630 rpcz_store.cc:269] 0709 13:00:39.402873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.521305 (+118432us) service_pool.cc:224] Handling call
0709 13:00:41.681717 (+2160412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:41.528820  1634 rpcz_store.cc:276] 0709 13:00:38.770813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.285017 (+514204us) service_pool.cc:224] Handling call
0709 13:00:39.289645 (+  4628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:40.110300 (+820655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.430708 (+1320408us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54003b88d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:41.430904 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632010624}
I20260709 13:00:41.682974  1702 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:41.491772  4423 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:41.683415  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.684063  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:41.683879  4423 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.684779  1702 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
W20260709 13:00:41.529687  1561 rpcz_store.cc:269] 0709 13:00:39.424660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.492856 (+ 68196us) service_pool.cc:224] Handling call
0709 13:00:41.431743 (+1938887us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b5400389ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:41.431852 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340803456}
I20260709 13:00:41.686275  4459 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:41.686412  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:41.687402  4423 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:41.687511  1561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1699) took 1527 ms (client timeout 1989 ms). Trace:
I20260709 13:00:41.533035  1628 rpcz_store.cc:276] 0709 13:00:38.754544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.825997 (+ 71453us) service_pool.cc:224] Handling call
0709 13:00:38.881020 (+ 55023us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:38.881985 (+   965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.436252 (+2554267us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b5400197650. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:41.436455 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114484608}
I20260709 13:00:41.516152  4449 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:41.688011  1561 rpcz_store.cc:269] 0709 13:00:40.159926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.686090 (+1526164us) service_pool.cc:224] Handling call
0709 13:00:41.687439 (+  1349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:41.512569  4268 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.688673  4449 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.689039  4268 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.489401  3963 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.566399  1553 rpcz_store.cc:276] 0709 13:00:38.110883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.111035 (+   152us) service_pool.cc:224] Handling call
0709 13:00:38.129127 (+ 18092us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:38.846661 (+717534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.313635 (+2466974us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b54002706d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:41.313945 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1622281600}
I20260709 13:00:41.691359  4268 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.517194  3967 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:41.586973  3976 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:41.692296  3967 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.693427  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:41.694921  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:41.695080  3967 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.536904  4381 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.695636  1703 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:41.546602  4123 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.703011  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 453) took 2305 ms. Trace:
I20260709 13:00:41.679198  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 455) took 2243 ms. Trace:
W20260709 13:00:41.568619  2317 connection.cc:256] RPC call timeout handler was delayed by -1.0736039720000008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
I20260709 13:00:41.517633  4359 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.681063  4311 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:41.578472  4189 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.976s	user 0.016s	sys 0.000s
I20260709 13:00:41.576305  4418 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.615772  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 366) took 2670 ms. Trace:
I20260709 13:00:41.613960  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2269) took 2778 ms. Trace:
I20260709 13:00:41.707731  1556 rpcz_store.cc:276] 0709 13:00:38.951388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.961131 (+  9743us) service_pool.cc:224] Handling call
0709 13:00:38.961449 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.089179 (+127730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:39.089227 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:39.114217 (+ 24990us) log.cc:844] Serialized 26 byte log entry
0709 13:00:39.369239 (+255022us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:39.369288 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.309804 (+1940516us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54000bf254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:41.309889 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.400503 (+ 90614us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:41.615436 (+214933us) raft_consensus.cc:1685] finished
0709 13:00:41.615468 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.615691 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25214,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":8760,"spinlock_wait_cycles":277602944,"thread_start_us":97098,"threads_started":1,"wal-append.queue_time_us":333763}
I20260709 13:00:41.563289  4448 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.630380  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1648) took 1008 ms. Trace:
I20260709 13:00:41.633395  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 507) took 3526 ms. Trace:
I20260709 13:00:41.630425  4313 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.618775  4466 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.716372  1709 rpcz_store.cc:276] 0709 13:00:40.621868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.629691 (+  7823us) service_pool.cc:224] Handling call
0709 13:00:41.630305 (+1000614us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:41.462178  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:41.657835  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2265) took 2895 ms. Trace:
I20260709 13:00:41.565454  4435 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.576100  4420 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:00:41.715701  1552 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:00:41.721719  1707 raft_consensus.cc:2368] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:41.657232  4401 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:41.567055  4267 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:00:41.670336  4415 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.735317  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1655) took 1019 ms. Trace:
I20260709 13:00:41.735932  1707 rpcz_store.cc:276] 0709 13:00:40.716340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.461763 (+745423us) service_pool.cc:224] Handling call
0709 13:00:41.735236 (+273473us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:41.673774  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 1.04759e+06us
I20260709 13:00:41.674295  1627 rpcz_store.cc:276] 0709 13:00:40.454024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.518387 (+ 64363us) service_pool.cc:224] Handling call
0709 13:00:41.513197 (+994810us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b54003b9cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:41.513359 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1886760192}
W20260709 13:00:41.395867  1636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2260) took 2641 ms (client timeout 1587 ms). Trace:
I20260709 13:00:41.692375  3976 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.589151  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2280) took 1134 ms. Trace:
I20260709 13:00:41.695824  4381 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.551752  4327 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.498783  4113 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.696301  4424 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:41.681016  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:41.697295  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:41.633507  4310 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.699400  4407 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:41.619014  4251 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:41.703573  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 228510us
I20260709 13:00:41.621675  4256 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:41.587711  4367 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.702755  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:41.723070  4374 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.562317  4436 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:41.619966  1705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.173s	user 0.015s	sys 0.018s
I20260709 13:00:41.704655  1710 rpcz_store.cc:276] 0709 13:00:39.397825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.156375 (+758550us) service_pool.cc:224] Handling call
0709 13:00:40.156788 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:40.157065 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:40.589493 (+432428us) log.cc:844] Serialized 14 byte log entry
0709 13:00:41.702615 (+1113122us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:41.702688 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:41.702744 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.702830 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.702950 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32845,"prepare.queue_time_us":180506,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":24524,"thread_start_us":1442753,"threads_started":2,"wal-append.queue_time_us":1050936}
I20260709 13:00:41.715704  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:41.659566  4458 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:41.900920  1705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.454s	user 0.015s	sys 0.019s
I20260709 13:00:41.715828  1557 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:41.704691  1706 rpcz_store.cc:276] 0709 13:00:39.435596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.160766 (+725170us) service_pool.cc:224] Handling call
0709 13:00:40.289466 (+128700us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:40.315344 (+ 25878us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:40.315394 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:40.332584 (+ 17190us) log.cc:844] Serialized 26 byte log entry
0709 13:00:40.627806 (+295222us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:40.627850 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.627931 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.678985 (+1051054us) raft_consensus.cc:1685] finished
0709 13:00:41.679008 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.679135 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35501,"raft.queue_time_us":143999,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":51882,"thread_start_us":286186,"threads_started":1,"wal-append.queue_time_us":1171018}
I20260709 13:00:41.788128  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:41.722450  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:41.717594  1629 rpcz_store.cc:276] 0709 13:00:38.762168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.975863 (+213695us) service_pool.cc:224] Handling call
0709 13:00:38.976305 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.313758 (+337453us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:00:39.313789 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:39.368420 (+ 54631us) log.cc:844] Serialized 26 byte log entry
0709 13:00:39.599858 (+231438us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:39.599927 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.603394 (+  3467us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.424200 (+1820806us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:41.626948 (+202748us) raft_consensus.cc:1685] finished
0709 13:00:41.626969 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.657753 (+ 30784us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8482,"raft.queue_time_us":70241,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":109,"spinlock_wait_cycles":2508544,"thread_start_us":436907,"threads_started":2,"wal-append.queue_time_us":205062}
I20260709 13:00:41.665982  1559 rpcz_store.cc:276] 0709 13:00:38.961880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.038389 (+ 76509us) service_pool.cc:224] Handling call
0709 13:00:39.038719 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.286277 (+247558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:39.286424 (+   147us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:39.286990 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 13:00:39.478639 (+191649us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:39.479190 (+   551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.403207 (+1924017us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54002cbd54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:41.403291 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.403376 (+    85us) raft_consensus.cc:1685] finished
0709 13:00:41.403400 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.403808 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16276,"raft.queue_time_us":7877,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":30336,"spinlock_wait_cycles":527189888,"thread_start_us":83116,"threads_started":1,"wal-append.queue_time_us":227063,"wal-append.run_cpu_time_us":2424,"wal-append.run_wall_time_us":1121331}
W20260709 13:00:41.710314  1526 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.934s (SENT) [suppressed 14 similar messages]
I20260709 13:00:41.709223  1635 rpcz_store.cc:276] 0709 13:00:38.835621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.351463 (+515842us) service_pool.cc:224] Handling call
0709 13:00:39.372130 (+ 20667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.398884 (+ 26754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:39.398915 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:39.424614 (+ 25699us) log.cc:844] Serialized 26 byte log entry
0709 13:00:39.682970 (+258356us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:39.683038 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.683272 (+   234us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.409419 (+1726147us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:41.605208 (+195789us) raft_consensus.cc:1685] finished
0709 13:00:41.605253 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.613851 (+  8598us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10107,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":9187,"spinlock_wait_cycles":24019712,"thread_start_us":195213,"threads_started":1,"wal-append.queue_time_us":712164}
I20260709 13:00:41.706502  4337 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.740962  1631 rpcz_store.cc:276] 0709 13:00:40.454734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.415696 (+960962us) service_pool.cc:224] Handling call
0709 13:00:41.588903 (+173207us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b540010e0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:41.589067 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112503680}
I20260709 13:00:41.703354  4430 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:41.739950  1636 rpcz_store.cc:269] 0709 13:00:38.753985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.786050 (+ 32065us) service_pool.cc:224] Handling call
0709 13:00:41.395471 (+2609421us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54002db9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:41.395704 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229755136}
I20260709 13:00:41.743199  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:41.701107  4462 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.743921  4322 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:41.743842  4401 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.745148  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:41.745815  4420 consensus_queue.cc:1243] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer 95fc6a4cf5874b7eb91280dffcd3b3fb log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:00:41.746624  3963 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.748725  4452 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.749411  1560 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:41.708621  4366 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.750559  4252 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.928817  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:41.725636  4313 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.929553  2352 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:41.751317  4448 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.707599  4445 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:41.752418  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:41.752893  4466 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.604497  4433 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.691402  4449 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.716488  1632 rpcz_store.cc:276] 0709 13:00:38.103341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.103492 (+   151us) service_pool.cc:224] Handling call
0709 13:00:38.133057 (+ 29565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:38.133659 (+   602us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:38.133693 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:38.138488 (+  4795us) log.cc:844] Serialized 26 byte log entry
0709 13:00:38.280501 (+142013us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:38.280573 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.280683 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.432546 (+3151863us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:41.621261 (+188715us) raft_consensus.cc:1685] finished
0709 13:00:41.621301 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.630014 (+  8713us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6978,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"spinlock_wait_cycles":7461120,"thread_start_us":141565,"threads_started":1,"wal-append.queue_time_us":202268}
I20260709 13:00:41.943709  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:41.945015  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:41.755951  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:41.824038  4327 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.803983  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:41.850032  1708 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:41.843715  2346 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:41.897246  1633 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
W20260709 13:00:41.649322  4209 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.182s	user 0.009s	sys 0.001s
I20260709 13:00:41.794349  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:41.711694  4123 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.957401  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:41.958817  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:41.724707  4311 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:41.710752  4423 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:41.959255  4123 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:41.901527  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1698) took 2322 ms (client timeout 1842 ms). Trace:
I20260709 13:00:41.959594  4311 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:41.960857  1557 rpcz_store.cc:269] 0709 13:00:39.583967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.130576 (+546609us) service_pool.cc:224] Handling call
0709 13:00:41.901421 (+1770845us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:41.964097  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:41.965101  1557 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
W20260709 13:00:41.903151  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 192880us
I20260709 13:00:41.902539  1555 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
W20260709 13:00:41.904760  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 202079us
I20260709 13:00:41.953954  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2171) took 2331 ms. Trace:
W20260709 13:00:41.966180  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1696) took 2419 ms (client timeout 3168 ms). Trace:
I20260709 13:00:41.905601  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:41.900169  1701 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
W20260709 13:00:41.966663  1555 rpcz_store.cc:269] 0709 13:00:39.546950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.722101 (+2175151us) service_pool.cc:224] Handling call
0709 13:00:41.966114 (+244013us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:41.967481  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1651) took 1344 ms. Trace:
I20260709 13:00:41.908761  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:41.968037  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:41.915719  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:41.856737  4388 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 2.898s	user 0.014s	sys 0.006s
I20260709 13:00:41.916006  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:41.916343  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:41.916488  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:41.972384  1710 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:41.972741  1631 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
W20260709 13:00:41.917666  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.918438  1556 raft_consensus.cc:2410] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:41.745792  3976 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.751348  3970 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:41.920781  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 368) took 2968 ms. Trace:
I20260709 13:00:41.738821  4189 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.176s
I20260709 13:00:41.986682  4435 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.986720  1554 rpcz_store.cc:276] 0709 13:00:38.952122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.960359 (+  8237us) service_pool.cc:224] Handling call
0709 13:00:41.343913 (+2383554us) spinlock_profiling.cc:243] Waited 8.67 ms on lock 0x7b5800002798. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5eee 0000000000375939 00007fda2c6291e0 00007fda2c6f20c4 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:00:41.344118 (+   205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:41.344418 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:41.618903 (+274485us) log.cc:844] Serialized 14 byte log entry
0709 13:00:41.908619 (+289716us) log.cc:844] Serialized 14 byte log entry
0709 13:00:41.920381 (+ 11762us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:41.920442 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:41.920480 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.920566 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.920719 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17104,"prepare.queue_time_us":370139,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":87,"raft.queue_time_us":21015,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":5885,"spinlock_wait_cycles":30819328,"thread_start_us":539084,"threads_started":2,"wal-append.queue_time_us":332862}
I20260709 13:00:41.903508  4310 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.924835  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:41.931727  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:41.934289  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:41.726419  4418 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.942258  4381 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.944629  4161 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.950114  4359 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:41.945907  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 190097us
W20260709 13:00:41.865439  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 101709us
I20260709 13:00:41.938830  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:41.917335  4337 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.985162  4454 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.959648  4423 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.949936  4445 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:41.843549  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:41.997440  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.043213  1553 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:41.804054  4415 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.957065  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:41.961404  4366 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:41.961601  1703 raft_consensus.cc:2376] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4. Re-sending same reply.
I20260709 13:00:41.962486  2349 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.059827  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:41.902534  1709 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:42.063294  1559 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:41.968432  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.064116  2352 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:41.915503  4430 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:41.950821  1627 raft_consensus.cc:2376] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4. Re-sending same reply.
I20260709 13:00:42.045392  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:42.054708  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:41.903319  4424 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:42.069701  4410 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.070155  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:42.070358  4410 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:42.071444  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.071971  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.069752  4161 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:41.907986  4374 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:42.074532  4410 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:42.076141  4424 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:42.076722  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:41.966682  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:42.078485  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 111952us
I20260709 13:00:41.968019  1701 rpcz_store.cc:276] 0709 13:00:40.623080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.680669 (+1057589us) service_pool.cc:224] Handling call
0709 13:00:41.967279 (+286610us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:41.969259  1629 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:41.978140  1560 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:41.972644  1635 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:41.982241  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:41.984601  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.073400  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.083201  1631 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
W20260709 13:00:41.985821  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2169) took 2387 ms (client timeout 2630 ms). Trace:
I20260709 13:00:42.038897  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.004012  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:41.959266  4327 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:41.993276  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:41.793396  4432 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:41.989804  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.043534  1710 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:42.017041  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.046231  2353 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:41.989797  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:41.952975  4449 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:42.042160  4427 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:42.058835  4310 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:42.067682  1558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 372) took 2752 ms (client timeout 1602 ms). Trace:
I20260709 13:00:42.068455  4418 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:42.072086  1559 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:41.966519  1561 rpcz_store.cc:276] 0709 13:00:39.615630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.751941 (+2136311us) service_pool.cc:224] Handling call
0709 13:00:41.946228 (+194287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.044888  2347 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.970391  4251 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:42.278254  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.327986  1557 raft_consensus.cc:2410] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:00:42.127102  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.328236  1633 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:00:42.055359  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:42.214864  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.330307  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2281) took 1862 ms. Trace:
W20260709 13:00:42.153049  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 107740us
I20260709 13:00:42.130270  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.330858  1633 rpcz_store.cc:276] 0709 13:00:40.468514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.962665 (+1494151us) service_pool.cc:224] Handling call
0709 13:00:42.330241 (+367576us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.154642  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.332528  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:42.333155  1628 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:42.214038  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.334008  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:42.147508  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.290798  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:42.291993  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1656) took 1575 ms (client timeout 1540 ms). Trace:
W20260709 13:00:42.305994  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2172) took 2631 ms (client timeout 2190 ms). Trace:
I20260709 13:00:42.297623  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:42.335922  1708 rpcz_store.cc:269] 0709 13:00:40.716647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.014492 (+297845us) service_pool.cc:224] Handling call
0709 13:00:42.291692 (+1277200us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54002d87d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:42.291880 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":566745344}
W20260709 13:00:42.329651  1556 rpcz_store.cc:269] 0709 13:00:39.597977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.722318 (+2124341us) service_pool.cc:224] Handling call
0709 13:00:41.985755 (+263437us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:42.337055  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 46346.5us
I20260709 13:00:41.973301  4398 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.246263  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.329685  1554 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:42.336122  1560 rpcz_store.cc:269] 0709 13:00:39.674974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.470105 (+1795131us) service_pool.cc:224] Handling call
0709 13:00:42.305669 (+835564us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400230854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:42.305861 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640193280}
I20260709 13:00:42.289541  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:42.342070  1634 raft_consensus.cc:2410] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:42.124619  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.124593  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.124601  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.343272  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2169) took 1041 ms. Trace:
I20260709 13:00:42.343493  1629 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:42.343856  1709 rpcz_store.cc:276] 0709 13:00:41.301916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.063802 (+761886us) service_pool.cc:224] Handling call
0709 13:00:42.343194 (+279392us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.146319  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:42.336728  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46741.9us
I20260709 13:00:42.295615  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.342435  1703 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.331247  1635 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:42.294145  4423 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:42.071300  4113 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:42.331278  2351 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
W20260709 13:00:42.328478  1558 rpcz_store.cc:269] 0709 13:00:39.314951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.383050 (+ 68099us) service_pool.cc:224] Handling call
0709 13:00:42.067447 (+2684397us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b5400313654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:42.067577 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4520657792}
I20260709 13:00:42.100562  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:42.330701  1553 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.328701  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1685) took 2922 ms. Trace:
I20260709 13:00:42.044700  4433 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:42.055994  4337 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:42.355827  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.042675  4446 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:42.345402  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:42.202930  4381 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:42.299836  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:42.381829  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.382823  1629 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:42.054105  4313 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:42.391324  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.392139  2351 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:42.363673  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.363788  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.008186  4401 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:42.348685  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.393996  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.394454  1633 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:42.348443  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.001937  4311 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:42.395081  1561 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:42.395478  4430 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:42.395800  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.396263  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:42.396562  1703 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:42.397248  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2168) took 1095 ms. Trace:
I20260709 13:00:42.397621  1703 rpcz_store.cc:276] 0709 13:00:41.301595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.062765 (+761170us) service_pool.cc:224] Handling call
0709 13:00:42.397183 (+334418us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.350148  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:42.354099  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:42.399049  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:42.399298  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 49368.6us
I20260709 13:00:42.400041  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.400492  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:42.400708  1631 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:42.401338  4458 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.345671  2349 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:42.346911  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.401810  4430 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.402359  4432 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:42.402518  4430 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.402671  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2174) took 1099 ms. Trace:
I20260709 13:00:42.402978  4430 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.403224  1706 rpcz_store.cc:276] 0709 13:00:41.304056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.346599 (+1042543us) service_pool.cc:224] Handling call
0709 13:00:42.402611 (+ 56012us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.404227  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.404774  1706 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:42.345772  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:42.405941  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 60258.9us
I20260709 13:00:42.407706  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:42.366983  1556 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:42.409139  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2174) took 2263 ms. Trace:
I20260709 13:00:42.409570  1556 rpcz_store.cc:276] 0709 13:00:40.146245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.337922 (+2191677us) service_pool.cc:224] Handling call
0709 13:00:42.409071 (+ 71149us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:42.410120  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.368845  4113 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.350159  1632 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:42.348299  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.345674  1555 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:00:42.414450  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 376) took 1007 ms. Trace:
I20260709 13:00:42.414506  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2277) took 2821 ms. Trace:
I20260709 13:00:42.414808  1555 rpcz_store.cc:276] 0709 13:00:41.407550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.127287 (+719737us) service_pool.cc:224] Handling call
0709 13:00:42.414391 (+287104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.414937  1630 rpcz_store.cc:276] 0709 13:00:39.593101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.348036 (+2754935us) service_pool.cc:224] Handling call
0709 13:00:42.414436 (+ 66400us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.368937  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.144701  4410 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.348779  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:41.991997  4388 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:42.354324  1557 rpcz_store.cc:276] 0709 13:00:39.406049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.033835 (+2627786us) service_pool.cc:224] Handling call
0709 13:00:42.328632 (+294797us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.365803  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2176) took 1060 ms. Trace:
I20260709 13:00:42.370198  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:42.417068  1701 rpcz_store.cc:276] 0709 13:00:41.305139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.081989 (+776850us) service_pool.cc:224] Handling call
0709 13:00:42.365733 (+283744us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:42.417407  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47305.1us
I20260709 13:00:42.417742  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.418092  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.418756  1628 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:42.418855  1701 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
W20260709 13:00:42.362247  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 261830us
I20260709 13:00:42.294306  4415 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:42.350149  3976 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:42.126197  4445 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:42.346328  4452 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:42.337692  4366 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.371950  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.411046  4367 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:42.372741  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:42.373270  4462 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.438481  4462 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.438529  4310 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:42.439147  4462 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.433269  1455 catalog_manager.cc:5697] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:42.441912  1455 catalog_manager.cc:5697] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 2 to 3, term changed from 1 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:42.443953  1455 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 4 to 6, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 6 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:42.374449  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:42.447064  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 72705.6us
I20260709 13:00:42.377795  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.370543  4251 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.383903  1554 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:42.420063  4401 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:42.419891  4410 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.384502  2347 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:00:42.385416  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.353570  4449 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:42.124742  4267 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.256s
I20260709 13:00:42.463898  4267 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:42.464341  4267 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:42.387571  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.389994  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.465920  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.426013  4381 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:42.398013  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:42.401899  4458 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.361205  1704 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:42.348130  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.468844  1636 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:42.469977  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.354470  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.401708  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.412060  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:42.412911  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:42.471745  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59767.3us
I20260709 13:00:42.472015  1556 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:42.473069  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:42.472280  1455 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 3 to 4, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 4 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:42.473774  1556 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.420420  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.421388  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:42.422144  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:42.426321  1634 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:42.423015  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.427407  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:42.479612  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.480325  1628 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.428181  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.428428  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:42.482606  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 55121.5us
W20260709 13:00:42.483161  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.474706  1455 catalog_manager.cc:5697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 4, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:00:42.483566  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61529.6us
I20260709 13:00:42.346495  4420 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:00:42.484380  1455 catalog_manager.cc:5697] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 4, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:42.485558  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:42.486084  4476 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:42.486742  4476 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:42.486340  4423 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:42.487361  4476 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.486340  1455 catalog_manager.cc:5697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 2, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 2 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:42.431442  4366 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.433214  4452 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:42.496151  1706 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:42.478850  2348 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:42.433949  1702 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:42.434614  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.435038  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.428953  3976 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.499274  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.437950  2350 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:42.434063  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.500468  1632 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 2.
I20260709 13:00:42.451794  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.416325  4113 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.454654  4310 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:42.455314  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.448910  1710 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
W20260709 13:00:42.509086  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.456161  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.383921  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.457806  1602 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:42.476601  4458 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.432826  4373 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.477826  1555 raft_consensus.cc:2376] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6. Re-sending same reply.
I20260709 13:00:42.497809  1630 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:42.503594  3976 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.516360  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.488902  1455 catalog_manager.cc:5697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:42.503780  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.503894  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:42.519238  1629 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:42.504439  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.520061  2353 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:42.504828  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.521167  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.521732  1561 raft_consensus.cc:2410] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:42.462323  4401 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.462596  4410 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.497699  4313 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.526145  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:42.507608  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.508045  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.510181  1557 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:42.527169  1559 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:42.466871  4367 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:42.468561  4381 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:42.511130  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 53478.1us
I20260709 13:00:42.531816  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.532401  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:42.530742  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.487546  4435 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:42.507458  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.513684  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2177) took 1056 ms. Trace:
I20260709 13:00:42.537433  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:42.537686  1560 rpcz_store.cc:276] 0709 13:00:41.457726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.350307 (+892581us) service_pool.cc:224] Handling call
0709 13:00:42.513611 (+163304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.514426  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.521426  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.517184  4337 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:42.504439  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.552261  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2175) took 1095 ms. Trace:
I20260709 13:00:42.552497  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.552701  1554 rpcz_store.cc:276] 0709 13:00:41.456905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.481521 (+1024616us) service_pool.cc:224] Handling call
0709 13:00:42.552183 (+ 70662us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.553627  1628 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:42.554340  4337 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:42.526291  4401 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.465291  4267 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:42.524860  1601 connection.cc:256] RPC call timeout handler was delayed by -1.6296125429999506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 13:00:42.526145  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:42.507910  4113 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.533573  2350 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:42.534015  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:42.533111  1602 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:42.534761  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.538282  1559 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:42.562150  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2176) took 1104 ms. Trace:
I20260709 13:00:42.562616  1559 rpcz_store.cc:276] 0709 13:00:41.457414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.531593 (+1074179us) service_pool.cc:224] Handling call
0709 13:00:42.562047 (+ 30454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.539597  1555 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:00:42.521870  1632 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 2.
I20260709 13:00:42.535756  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:42.536648  1710 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:42.538792  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:42.530061  1455 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:42.563805  1629 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:42.564687  1455 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 5, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 5 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:42.566380  1455 catalog_manager.cc:5697] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 4, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 4 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:42.539461  4363 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:42.538712  1707 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:42.569000  1554 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:42.549639  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.549926  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.549646  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.567798  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.501978  4366 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.526683  4313 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.574540  4415 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.574872  4313 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.575825  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.576610  1710 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:42.577414  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.577927  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.557132  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.578501  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.557165  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.579632  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1692) took 3140 ms. Trace:
I20260709 13:00:42.580044  1554 rpcz_store.cc:276] 0709 13:00:39.430137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.556048 (+3125911us) service_pool.cc:224] Handling call
0709 13:00:42.568645 (+ 12597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:42.570254 (+  1609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.570486 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.558791  2348 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:00:42.551308  4483 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 13:00:42.558002  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4712125429999787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 13:00:42.583153  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4712125429999787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
I20260709 13:00:42.539848  4435 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:42.535244  4381 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.507550  4449 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:42.586270  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4082125429999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 13:00:42.586853  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0761125429999652s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
W20260709 13:00:42.587250  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0430679199999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
I20260709 13:00:42.570106  1628 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:42.570200  1602 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:42.571341  1634 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:42.571488  1455 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 3, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:42.555261  4089 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:42.578748  1561 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:42.596917  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1697) took 3030 ms. Trace:
I20260709 13:00:42.597290  1561 rpcz_store.cc:276] 0709 13:00:39.566904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.578087 (+3011183us) service_pool.cc:224] Handling call
0709 13:00:42.578427 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:42.596596 (+ 18169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.596840 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.578862  1630 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:42.579257  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2173) took 1275 ms. Trace:
I20260709 13:00:42.598531  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.556748  4337 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.583853  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:42.568728  3963 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.626881  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.627024  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:42.632879  4313 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:42.635442  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2165) took 1334 ms (client timeout 1569 ms). Trace:
W20260709 13:00:42.635941  1704 rpcz_store.cc:269] 0709 13:00:41.300565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.680809 (+380244us) service_pool.cc:224] Handling call
0709 13:00:42.635247 (+954438us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b54002a76d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:42.635375 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1047805440}
I20260709 13:00:42.637778  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.638623  1704 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:42.639775  4459 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.640443  4459 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:42.641646  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:42.642565  4423 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.642951  4423 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:42.643383  4423 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.639411  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:42.644404  1704 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.645102  4425 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:42.645586  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.646679  4425 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.647037  4425 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.647253  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.648494  1709 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:42.598832  1710 rpcz_store.cc:276] 0709 13:00:41.303358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.578268 (+1274910us) service_pool.cc:224] Handling call
0709 13:00:42.579200 (+   932us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.642992  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:42.649956  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.650305  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:42.650560  4418 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.650741  2347 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:42.650906  1710 raft_consensus.cc:2376] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5. Re-sending same reply.
I20260709 13:00:42.651978  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.636327  4313 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:42.652776  2347 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:42.589720  1455 catalog_manager.cc:5697] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:42.657747  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.572455  4420 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:42.569062  4363 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.604578  1555 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:42.684973  1704 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:42.685602  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1695) took 3191 ms. Trace:
I20260709 13:00:42.686101  1555 rpcz_store.cc:276] 0709 13:00:39.494558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.569447 (+3074889us) service_pool.cc:224] Handling call
0709 13:00:42.569779 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:42.685290 (+115511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.685531 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.588186  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.687515  1632 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:42.687891  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:42.688565  1704 raft_consensus.cc:2376] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3. Re-sending same reply.
I20260709 13:00:42.688531  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:42.606833  1552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1687) took 3200 ms (client timeout 1716 ms). Trace:
I20260709 13:00:42.689808  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:42.690030  1552 rpcz_store.cc:269] 0709 13:00:39.406762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.652361 (+245599us) service_pool.cc:224] Handling call
0709 13:00:42.606653 (+2954292us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b540038a4d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:42.606762 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3675323264}
I20260709 13:00:42.691390  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.692153  1552 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:42.608289  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.596294  4089 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:42.693646  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.694515  2348 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:42.694267  4089 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:42.608945  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2173) took 2891 ms. Trace:
I20260709 13:00:42.610116  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.705134  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.613915  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:42.706796  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 93008.2us
I20260709 13:00:42.707857  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:42.708971  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:42.615767  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.613965  4427 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.608325  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.618152  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.618561  4374 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:42.590131  4435 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.719951  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.621021  4398 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:42.607991  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.728838  1706 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:42.647377  4425 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.651929  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.652040  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:42.591838  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:42.603626  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1662) took 1305 ms (client timeout 1672 ms). Trace:
I20260709 13:00:42.649310  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2178) took 1160 ms. Trace:
I20260709 13:00:42.644304  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.684419  1629 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:42.690541  1704 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:42.691852  3976 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:42.607950  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:42.693233  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.597873  4267 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.695300  1561 rpcz_store.cc:276] 0709 13:00:39.726090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.608084 (+2881994us) service_pool.cc:224] Handling call
0709 13:00:42.608406 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:42.608609 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:42.608638 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:42.608688 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:42.608721 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.608775 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.608887 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.601055  4457 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:42.696755  1455 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 3, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:42.697981  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:42.597029  4452 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:42.766374  2348 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:00:42.788252  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 180382us
I20260709 13:00:42.712860  4427 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:42.763079  4487 log.cc:927] Time spent T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:00:42.806627  4427 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.710043  4420 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:42.808318  4423 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:00:42.814139  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 331725us
W20260709 13:00:42.833372  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 223133us
I20260709 13:00:42.834430  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:42.651085  4418 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.713255  1634 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:42.713277  1630 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:42.714713  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.624821  3963 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.731200  4398 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.719058  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.643409  4459 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:42.790066  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:42.790947  1702 rpcz_store.cc:269] 0709 13:00:41.298542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.703359 (+404817us) service_pool.cc:224] Handling call
0709 13:00:42.603412 (+900053us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b540016ba54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:42.603558 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224738176}
I20260709 13:00:42.698854  4363 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.791857  1709 rpcz_store.cc:276] 0709 13:00:41.489427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.646923 (+1157496us) service_pool.cc:224] Handling call
0709 13:00:42.649173 (+  2250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.794077  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 461) took 1169 ms. Trace:
I20260709 13:00:42.868773  1704 rpcz_store.cc:276] 0709 13:00:41.624368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.689569 (+1065201us) service_pool.cc:224] Handling call
0709 13:00:42.794014 (+104445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.794677  2353 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:42.796411  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:42.797398  2351 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:42.762693  4089 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.797621  4252 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.800043  3976 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:42.768235  4476 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:42.801555  4374 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:42.880841  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:42.802253  4479 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.800s	user 0.009s	sys 0.000s
I20260709 13:00:42.801945  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.882294  1706 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 2.
I20260709 13:00:42.765056  4462 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:42.765920  4367 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:42.718441  2344 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:42.808383  4420 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:42.710722  2349 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:42.703047  1553 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:42.896185  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2188) took 1155 ms. Trace:
I20260709 13:00:42.896590  1553 rpcz_store.cc:276] 0709 13:00:41.740964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.055051 (+314087us) service_pool.cc:224] Handling call
0709 13:00:42.896046 (+840995us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.838230  1708 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:42.897872  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2163) took 1598 ms. Trace:
I20260709 13:00:42.898313  1708 rpcz_store.cc:276] 0709 13:00:41.299814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.668082 (+1368268us) service_pool.cc:224] Handling call
0709 13:00:42.897792 (+229710us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.717724  1558 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:00:42.900012  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1701) took 1475 ms. Trace:
I20260709 13:00:42.900490  1558 rpcz_store.cc:276] 0709 13:00:41.425079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.664001 (+1238922us) service_pool.cc:224] Handling call
0709 13:00:42.713015 (+ 49014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:42.899611 (+186596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.899923 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:42.715546  4432 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.846421  2350 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:42.849952  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.854012  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.835407  4457 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:42.741379  4424 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:42.872761  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 456) took 3399 ms. Trace:
I20260709 13:00:42.879287  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.805711  4313 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:42.788153  1455 catalog_manager.cc:5697] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 1 to 3, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:42.844125  4418 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.884799  1556 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:42.885883  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.918337  1627 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:42.919433  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:42.925583  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.944566  1709 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:42.927446  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.590121  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:42.888342  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:42.844487  4446 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:42.856899  3963 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:42.889508  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55249.6us
I20260709 13:00:42.887562  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.887593  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:42.907655  4476 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:42.883996  4374 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:42.909781  3976 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:42.878103  4252 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:42.949311  1628 raft_consensus.cc:2410] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:42.930716  1705 rpcz_store.cc:276] 0709 13:00:39.473592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.445194 (+971602us) service_pool.cc:224] Handling call
0709 13:00:40.445683 (+   489us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:40.446154 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:40.446192 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:40.446970 (+   778us) log.cc:844] Serialized 52 byte log entry
0709 13:00:41.443025 (+996055us) spinlock_profiling.cc:243] Waited 61.9 ms on lock 0x7b48001e1a58. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:00:42.872133 (+1429108us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b54001c5aa0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27644090 00007fda276516b8 00007fda276da92b 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c
0709 13:00:42.872231 (+    98us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:42.872296 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.872408 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:42.872476 (+    68us) raft_consensus.cc:1685] finished
0709 13:00:42.872494 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.872685 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33830,"raft.run_cpu_time_us":8503,"raft.run_wall_time_us":942587,"spinlock_wait_cycles":165480576,"thread_start_us":1453932,"threads_started":1,"wal-append.queue_time_us":1354550,"wal-append.run_cpu_time_us":8028,"wal-append.run_wall_time_us":1115577}
I20260709 13:00:42.931635  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.931942  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:42.932348  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:42.880827  4488 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.004s
I20260709 13:00:42.938657  1455 catalog_manager.cc:5697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:43.042970  1455 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:42.940651  1560 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:42.938972  1704 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:43.046451  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:43.047968  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:43.049440  1704 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:42.940711  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:43.050344  1556 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:42.956538  1702 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:42.956696  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:42.969894  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:43.052448  1627 raft_consensus.cc:2376] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6. Re-sending same reply.
I20260709 13:00:42.925998  2345 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:42.978660  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:42.991789  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.998946  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:43.062268  1709 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20260709 13:00:43.002761  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.926739  2344 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:43.070586  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2192) took 1086 ms. Trace:
I20260709 13:00:43.071223  1709 rpcz_store.cc:276] 0709 13:00:41.983780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.945295 (+961515us) service_pool.cc:224] Handling call
0709 13:00:43.070486 (+125191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.052807  1630 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:43.028715  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:42.931521  4457 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:43.029301  1558 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:00:42.927505  4388 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:43.028115  4252 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.038496  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.090827  1634 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:42.919071  4462 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:42.928033  1629 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
W20260709 13:00:43.093926  1559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.000s	sys 0.028s
I20260709 13:00:42.927591  4432 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:42.913457  4366 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:42.930583  4424 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:43.056190  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 99582.5us
I20260709 13:00:42.973560  4446 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:43.114287  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:43.057691  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2166) took 1756 ms. Trace:
I20260709 13:00:43.114835  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:43.059762  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.115744  1708 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:43.116048  1627 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:00:43.074050  1710 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:43.079030  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:43.117878  1702 raft_consensus.cc:2410] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:42.985689  4374 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:43.080978  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.118899  1628 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:42.909797  4423 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:43.095083  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:43.120754  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2075) took 1063 ms. Trace:
I20260709 13:00:43.121320  1705 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
W20260709 13:00:43.061250  4251 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.595s	user 0.012s	sys 0.017s
I20260709 13:00:43.039422  3976 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:43.145182  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:43.145807  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.146335  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2279) took 2692 ms. Trace:
I20260709 13:00:43.146996  1628 raft_consensus.cc:2410] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:43.148175  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.144918  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:43.097085  1559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.000s	sys 0.028s
I20260709 13:00:43.103082  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1686) took 3696 ms. Trace:
I20260709 13:00:43.103052  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:43.115440  1704 rpcz_store.cc:276] 0709 13:00:41.300895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.047686 (+1746791us) service_pool.cc:224] Handling call
0709 13:00:43.057600 (+  9914us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:43.073971  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 447026us
I20260709 13:00:43.121199  1627 rpcz_store.cc:276] 0709 13:00:42.057350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.059424 (+1002074us) service_pool.cc:224] Handling call
0709 13:00:43.120681 (+ 61257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.124675  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:43.126274  1630 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:43.150830  1560 rpcz_store.cc:276] 0709 13:00:39.406418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.549058 (+3142640us) service_pool.cc:224] Handling call
0709 13:00:42.940383 (+391325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.045229 (+104846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.102775 (+ 57546us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b54003c0fd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:43.102996 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":898875648}
W20260709 13:00:43.154803  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.061677  4476 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:43.146768  1629 rpcz_store.cc:276] 0709 13:00:40.454378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.096986 (+2642608us) service_pool.cc:224] Handling call
0709 13:00:43.146264 (+ 49278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.162895  1704 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:43.153684  1703 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:43.163812  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1652) took 2448 ms. Trace:
I20260709 13:00:43.164414  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:43.164347  1704 rpcz_store.cc:276] 0709 13:00:40.715210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.162024 (+2446814us) service_pool.cc:224] Handling call
0709 13:00:43.162494 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.163426 (+   932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.163732 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.165241  1708 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:43.151607  1709 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:43.156409  1601 leader_election.cc:400] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:43.166141  1601 leader_election.cc:403] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:00:43.155416  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.156883  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:43.167356  1704 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:43.167415  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.168074  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1653) took 2452 ms. Trace:
I20260709 13:00:43.168692  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2072) took 1112 ms. Trace:
I20260709 13:00:43.168489  1704 rpcz_store.cc:276] 0709 13:00:40.715664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.166239 (+2450575us) service_pool.cc:224] Handling call
0709 13:00:43.166583 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.167812 (+  1229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.168018 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.169083  1627 rpcz_store.cc:276] 0709 13:00:42.056311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.167147 (+1110836us) service_pool.cc:224] Handling call
0709 13:00:43.168632 (+  1485us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:43.170374  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56229.4us
I20260709 13:00:43.170848  1704 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:43.173087  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 521096us
I20260709 13:00:43.174552  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:43.176420  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:43.113663  4432 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.190382  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:43.163259  4449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.018s	sys 0.040s
I20260709 13:00:43.185940  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.188762  1705 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:43.190711  1552 raft_consensus.cc:2368] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:43.192327  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2194) took 1208 ms. Trace:
I20260709 13:00:43.172777  1675 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:43.192672  1627 raft_consensus.cc:2410] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:43.193346  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:43.192731  1703 rpcz_store.cc:276] 0709 13:00:42.066816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.398446 (+331630us) service_pool.cc:224] Handling call
0709 13:00:43.192149 (+793703us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b540016bf54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:43.192266 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1733834624}
I20260709 13:00:43.185050  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1660) took 2337 ms. Trace:
I20260709 13:00:43.197705  1704 rpcz_store.cc:276] 0709 13:00:40.847475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.169966 (+2322491us) service_pool.cc:224] Handling call
0709 13:00:43.170593 (+   627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.171755 (+  1162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.184949 (+ 13194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.200062  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2175) took 1895 ms. Trace:
I20260709 13:00:43.200556  1704 rpcz_store.cc:276] 0709 13:00:41.304548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.198863 (+1894315us) service_pool.cc:224] Handling call
0709 13:00:43.199317 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.199596 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.199637 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:43.199698 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:43.199744 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.199819 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.199979 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.202541  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:43.205258  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:43.206169  1558 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
W20260709 13:00:43.207263  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 428763us
I20260709 13:00:43.187775  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:43.210618  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.211508  1710 raft_consensus.cc:2410] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:43.213285  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:43.213869  1708 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:43.153501  4462 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:43.147287  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 482) took 1017 ms. Trace:
W20260709 13:00:43.212054  4415 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.617s	user 0.014s	sys 0.000s
I20260709 13:00:43.242468  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:43.221105  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1650) took 2569 ms. Trace:
I20260709 13:00:43.259088  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 460) took 1635 ms. Trace:
I20260709 13:00:43.264897  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:43.202006  4449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.018s	sys 0.040s
I20260709 13:00:43.262060  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.275389  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2172) took 1972 ms. Trace:
I20260709 13:00:43.276001  1709 rpcz_store.cc:276] 0709 13:00:41.303035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.175763 (+1872728us) service_pool.cc:224] Handling call
0709 13:00:43.235601 (+ 59838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.262881 (+ 27280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.262951 (+    70us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:43.274989 (+ 12038us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:43.275053 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.275144 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.275311 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.238451  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1654) took 2522 ms. Trace:
I20260709 13:00:43.256599  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:43.278136  1708 rpcz_store.cc:276] 0709 13:00:40.716019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.166259 (+2450240us) service_pool.cc:224] Handling call
0709 13:00:43.213614 (+ 47355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.238054 (+ 24440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.238366 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.266575  1705 rpcz_store.cc:276] 0709 13:00:40.622763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.153080 (+2530317us) service_pool.cc:224] Handling call
0709 13:00:43.188572 (+ 35492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.192203 (+  3631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.192423 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.267877  2348 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:43.260833  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:43.270349  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 95948.1us
I20260709 13:00:43.179517  4423 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:43.267285  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:43.306217  1553 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:43.265508  1710 rpcz_store.cc:276] 0709 13:00:41.624093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.143823 (+1519730us) service_pool.cc:224] Handling call
0709 13:00:43.259017 (+115194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.297533  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:43.307840  1558 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.301934  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 523) took 1603 ms. Trace:
I20260709 13:00:43.308764  1634 rpcz_store.cc:276] 0709 13:00:41.698211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.202228 (+1504017us) service_pool.cc:224] Handling call
0709 13:00:43.301845 (+ 99617us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:43.304322  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 514279us
I20260709 13:00:43.305083  1703 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:43.310784  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2179) took 1793 ms. Trace:
I20260709 13:00:43.311214  1703 rpcz_store.cc:276] 0709 13:00:41.517681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.266364 (+1748683us) service_pool.cc:224] Handling call
0709 13:00:43.304786 (+ 38422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.310343 (+  5557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.310705 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.301947  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:43.314765  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:43.321429  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:43.343560  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.344405  1630 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:43.275097  4415 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:43.339630  4267 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.007s
W20260709 13:00:43.293895  2345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 744) took 1551 ms (client timeout 1989 ms). Trace:
I20260709 13:00:43.364529  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2184) took 1386 ms. Trace:
I20260709 13:00:43.357555  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.365785  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:43.365145  1710 rpcz_store.cc:276] 0709 13:00:41.978391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.353855 (+1375464us) service_pool.cc:224] Handling call
0709 13:00:43.354416 (+   561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.364021 (+  9605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.364052 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:43.364124 (+    72us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:43.364184 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.364280 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.364448 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.366732  1709 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:00:43.218567  4493 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.125s
I20260709 13:00:43.367818  1710 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:43.369047  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2195) took 1228 ms. Trace:
I20260709 13:00:43.369637  1710 rpcz_store.cc:276] 0709 13:00:42.140606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.367002 (+1226396us) service_pool.cc:224] Handling call
0709 13:00:43.367400 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.368579 (+  1179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.368977 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.334045  4419 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
W20260709 13:00:43.346482  4430 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.943s	user 0.014s	sys 0.005s
I20260709 13:00:43.341197  4445 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:43.373580  2346 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.007s	sys 0.024s
W20260709 13:00:43.415063  2346 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.008s	sys 0.025s
I20260709 13:00:43.346426  1526 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:43.388228  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 499708us
I20260709 13:00:43.273772  4462 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:43.388579  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 527) took 1329 ms. Trace:
I20260709 13:00:43.419283  1634 rpcz_store.cc:276] 0709 13:00:42.058870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.339100 (+1280230us) service_pool.cc:224] Handling call
0709 13:00:43.388475 (+ 49375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.421650  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:43.422271  1709 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:43.347556  4449 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:43.422928  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:43.426059  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 79742.3us
I20260709 13:00:43.427013  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:43.426921  1602 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:43.410931  1553 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:43.386930  1561 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:43.434129  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:43.364753  4423 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:43.438457  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.430801  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:43.261149  2344 rpcz_store.cc:276] 0709 13:00:42.130274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.926014 (+795740us) service_pool.cc:224] Handling call
0709 13:00:42.926389 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.085986 (+159597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.120494 (+ 34508us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.356298  4313 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:43.368563  4433 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:43.439953  1634 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
W20260709 13:00:43.439193  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 551625us
I20260709 13:00:43.349093  1627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:48418
I20260709 13:00:43.464255  1627 raft_consensus.cc:606] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:43.464924  1627 raft_consensus.cc:614] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:43.422284  4267 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:43.421103  4445 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:43.461558  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:43.467559  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:43.468272  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:43.426962  4449 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:43.488641  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:43.438881  4415 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:43.343089  3976 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:43.473783  1628 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:43.475032  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 230282us
I20260709 13:00:43.486447  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:43.438915  4423 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:43.432977  4500 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:43.471390  4452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.019s	sys 0.029s
W20260709 13:00:43.371841  2345 rpcz_store.cc:269] 0709 13:00:41.742266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.842307 (+100041us) service_pool.cc:224] Handling call
0709 13:00:42.645432 (+803125us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b5800003c98. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5eee 0000000000375939 00007fda2c6291e0 00007fda2c6f20c4 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:00:43.256295 (+610863us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400643a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:43.276028 (+ 19733us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563120768}
I20260709 13:00:43.451573  4419 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:43.514040  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 889) took 1470 ms. Trace:
I20260709 13:00:43.459403  4388 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:43.510681  1560 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:43.518340  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:43.460394  4497 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:43.426700  4430 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:43.356213  4424 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:43.507368  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:43.538864  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.543794  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:43.495208  4449 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:43.490331  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 247963us
I20260709 13:00:43.523707  2348 rpcz_store.cc:276] 0709 13:00:42.043810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.802743 (+758933us) service_pool.cc:224] Handling call
0709 13:00:43.261815 (+459072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.295592 (+ 33777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.513747 (+218155us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b5400643a50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:43.513954 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1054794112}
I20260709 13:00:43.486532  4445 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:43.509294  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 530) took 1409 ms. Trace:
I20260709 13:00:43.522955  4423 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:43.519356  4496 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:43.522653  4452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.019s	sys 0.030s
I20260709 13:00:43.518785  1634 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:43.492997  4415 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.541824  4419 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:43.566195  1674 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:43.495115  4504 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:43.567550  1633 rpcz_store.cc:276] 0709 13:00:42.099642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.395542 (+295900us) service_pool.cc:224] Handling call
0709 13:00:43.459410 (+1063868us) spinlock_profiling.cc:243] Waited 7.96 ms on lock 0x7b540005c920. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27651dc0 00007fda276dde18 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6
0709 13:00:43.508997 (+ 49587us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b54001b6f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:43.509202 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2192849408}
W20260709 13:00:43.506028  4492 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:00:43.502876  4457 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:43.503834  4506 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:43.519346  4267 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:43.573160  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:43.522509  4500 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:43.541985  4388 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.541432  4366 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:43.556886  1628 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:43.491796  4433 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:43.592973  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2273) took 4409 ms. Trace:
I20260709 13:00:43.602926  4415 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:43.569504  4497 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:43.591001  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 485) took 1273 ms. Trace:
I20260709 13:00:43.616875  4327 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:43.590121  4345 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.007s	sys 0.006s
W20260709 13:00:43.600281  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 415518us
W20260709 13:00:43.614516  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 147717us
I20260709 13:00:43.562836  4446 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:43.562911  4509 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:43.623639  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 189601us
I20260709 13:00:43.642041  4493 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:43.586890  4311 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:43.599300  4494 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:43.643325  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 99634.8us
I20260709 13:00:43.644538  1634 rpcz_store.cc:276] 0709 13:00:39.183419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.472172 (+4288753us) service_pool.cc:224] Handling call
0709 13:00:43.518567 (+ 46395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.592449 (+ 73882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.592885 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:43.671422  4345 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.007s	sys 0.007s
I20260709 13:00:43.699150  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 757) took 1600 ms. Trace:
W20260709 13:00:43.662119  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:43.603235  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 176407us
I20260709 13:00:43.626722  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 487) took 1252 ms. Trace:
I20260709 13:00:43.637980  4388 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:00:43.623781  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 57683us
I20260709 13:00:43.718024  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:43.632161  4457 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:43.682050  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 360686us
I20260709 13:00:43.686988  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:43.636955  4500 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.645776  4516 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:43.658649  4504 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:43.599993  1706 raft_consensus.cc:2368] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:43.696162  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:43.591199  4508 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:43.706082  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:43.676118  4366 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:43.707144  2353 rpcz_store.cc:276] 0709 13:00:42.098668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.908601 (+809933us) service_pool.cc:224] Handling call
0709 13:00:42.908916 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:42.910437 (+  1521us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:43.338066 (+427629us) log.cc:844] Serialized 14 byte log entry
0709 13:00:43.698727 (+360661us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:43.698795 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:43.698845 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.698928 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.699071 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18833,"prepare.queue_time_us":460219,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"thread_start_us":762683,"threads_started":3,"wal-append.queue_time_us":348615}
I20260709 13:00:43.712579  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2170) took 4102 ms. Trace:
I20260709 13:00:43.783622  1554 rpcz_store.cc:276] 0709 13:00:39.610071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.581025 (+2970954us) service_pool.cc:224] Handling call
0709 13:00:42.596668 (+ 15643us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:42.597065 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:42.597105 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:42.597833 (+   728us) log.cc:844] Serialized 26 byte log entry
0709 13:00:43.078084 (+480251us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:43.078149 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.078228 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:43.712254 (+634026us) raft_consensus.cc:1685] finished
0709 13:00:43.712288 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.712477 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1755,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":16353,"thread_start_us":468071,"threads_started":1,"wal-append.queue_time_us":739106}
W20260709 13:00:43.681555  4427 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.874s	user 0.017s	sys 0.004s
W20260709 13:00:43.715427  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.719755  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:43.638391  4506 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:43.687235  4493 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:43.730465  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 43633.3us
I20260709 13:00:43.730897  1634 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:43.691875  4327 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:43.616900  4313 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:43.740857  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 140571us
W20260709 13:00:43.717085  4401 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.138s	user 0.009s	sys 0.005s
I20260709 13:00:43.700018  4494 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:43.731413  4418 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.798s	user 0.012s	sys 0.000s
I20260709 13:00:43.769647  4267 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:43.729641  4510 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:43.796586  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 100507us
W20260709 13:00:43.801512  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 81818.4us
I20260709 13:00:43.872095  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:43.764331  4410 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.214s	user 0.009s	sys 0.004s
I20260709 13:00:43.802335  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:43.732928  4462 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:43.806546  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2051) took 2265 ms. Trace:
I20260709 13:00:43.765317  4508 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:43.806355  4459 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:43.887619  4446 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:43.888132  4459 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:43.888347  4446 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:43.888828  4459 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:43.889583  4446 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:43.891621  4508 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:43.892876  4425 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.083s	user 0.009s	sys 0.002s
I20260709 13:00:43.858170  4494 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:43.917187  4515 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:00:43.869699  4510 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:43.937032  4510 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:43.943212  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 761) took 1645 ms. Trace:
I20260709 13:00:43.943714  2351 rpcz_store.cc:276] 0709 13:00:42.298011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.930547 (+632536us) service_pool.cc:224] Handling call
0709 13:00:42.930917 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.178249 (+247332us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:43.570574 (+392325us) log.cc:844] Serialized 14 byte log entry
0709 13:00:43.846533 (+275959us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:43.846627 (+    94us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:43.846696 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.942920 (+ 96224us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54006185d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:43.942973 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.943136 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":324771,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":141260,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":11329,"spinlock_wait_cycles":577100928,"thread_start_us":635937,"threads_started":3,"wal-append.queue_time_us":322892}
W20260709 13:00:43.945039  3963 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.915s	user 0.009s	sys 0.001s
I20260709 13:00:43.810766  4493 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:43.834304  4433 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:43.689975  4419 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:43.879662  1634 rpcz_store.cc:276] 0709 13:00:41.541071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.696185 (+2155114us) service_pool.cc:224] Handling call
0709 13:00:43.730640 (+ 34455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.801507 (+ 70867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.806462 (+  4955us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:43.803040  4509 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 13:00:43.881714  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:43.713798  4499 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 13:00:43.822451  1558 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
W20260709 13:00:43.885834  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 83586.3us
I20260709 13:00:43.927336  4452 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:43.711537  2344 rpcz_store.cc:276] 0709 13:00:42.365219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.507670 (+1142451us) service_pool.cc:224] Handling call
0709 13:00:43.519898 (+ 12228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.550293 (+ 30395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.558945 (+  8652us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:43.569890 (+ 10945us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:43.584793 (+ 14903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.591823 (+  7030us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.617910 (+ 26087us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:43.879560  4398 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.997s	user 0.004s	sys 0.009s
I20260709 13:00:43.914753  4366 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:43.915210  4381 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.285s	user 0.009s	sys 0.003s
I20260709 13:00:43.999848  4381 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:44.001185  1636 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:43.894196  4520 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:43.888829  4507 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.217s	user 0.001s	sys 0.001s
W20260709 13:00:44.002871  4435 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.216s	user 0.013s	sys 0.010s
W20260709 13:00:43.916141  4518 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 13:00:43.988847  4374 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.810s	user 0.008s	sys 0.005s
I20260709 13:00:43.800737  4427 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:43.928315  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 57456us
I20260709 13:00:44.004073  4374 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.004941  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:44.005895  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:43.951834  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:44.006490  4374 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:43.663018  2347 rpcz_store.cc:276] 0709 13:00:42.317610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.958788 (+641178us) service_pool.cc:224] Handling call
0709 13:00:43.028422 (+ 69634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.028696 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:43.424485 (+395789us) log.cc:844] Serialized 14 byte log entry
0709 13:00:43.585327 (+160842us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:43.585612 (+   285us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:43.585713 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.585884 (+   171us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.590818 (+  4934us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21265,"prepare.queue_time_us":425645,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":88023,"thread_start_us":491603,"threads_started":2}
W20260709 13:00:43.887154  4505 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.325s	user 0.001s	sys 0.001s
I20260709 13:00:43.883864  4462 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:44.008483  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 758) took 1720 ms. Trace:
I20260709 13:00:44.008934  2350 rpcz_store.cc:276] 0709 13:00:42.288113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.928346 (+640233us) service_pool.cc:224] Handling call
0709 13:00:42.928975 (+   629us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:42.929394 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:42.929419 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:42.930898 (+  1479us) log.cc:844] Serialized 564 byte log entry
0709 13:00:43.297136 (+366238us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:43.297183 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.297255 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.008218 (+710963us) raft_consensus.cc:1685] finished
0709 13:00:44.008252 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.008412 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":365898,"threads_started":1,"wal-append.queue_time_us":536844}
I20260709 13:00:43.859118  4531 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 13:00:43.735031  4516 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.011065  4516 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:44.011425  4495 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.505s	user 0.004s	sys 0.004s
I20260709 13:00:44.008510  4462 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:44.012331  4388 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:43.893549  4430 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:43.870265  4418 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:43.973389  2351 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:43.973824  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:43.893388  4425 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:43.877310  4401 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:43.818843  4504 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.139s
I20260709 13:00:43.884259  4410 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:43.994304  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:43.997704  4459 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:43.998651  4398 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:44.006064  1552 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:44.008610  4113 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.437s	user 0.008s	sys 0.008s
I20260709 13:00:44.012874  4462 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:44.013813  4430 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:43.991094  4506 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:44.015065  1556 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:43.819186  1554 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:44.003573  4435 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:43.832276  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 161696us
I20260709 13:00:43.979151  1526 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:43.953272  4337 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.287s	user 0.013s	sys 0.008s
W20260709 13:00:44.018893  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.038030  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 362337us
I20260709 13:00:44.039418  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:00:43.986027  4363 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.105s	user 0.011s	sys 0.004s
W20260709 13:00:43.993383  4252 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.863s	user 0.011s	sys 0.002s
I20260709 13:00:43.989297  4419 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:44.083570  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:43.961256  3963 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:43.857870  4497 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:43.917263  4327 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.219393  2352 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:00:44.182811  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 49189.9us
I20260709 13:00:44.229182  1634 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.230376  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2181) took 2601 ms. Trace:
I20260709 13:00:44.230795  1554 rpcz_store.cc:276] 0709 13:00:41.629272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.785390 (+2156118us) service_pool.cc:224] Handling call
0709 13:00:43.818988 (+ 33598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.230041 (+411053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.230307 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:44.191567  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 898) took 2117 ms. Trace:
W20260709 13:00:44.106293  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 99484.8us
I20260709 13:00:44.235646  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:44.236719  1601 leader_election.cc:400] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:44.237116  1601 leader_election.cc:403] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:00:44.253938  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 372306us
I20260709 13:00:44.233393  2349 rpcz_store.cc:276] 0709 13:00:42.074025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.896386 (+822361us) service_pool.cc:224] Handling call
0709 13:00:42.896877 (+   491us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:43.372215 (+475338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.372281 (+    66us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:43.449864 (+ 77583us) log.cc:844] Serialized 52 byte log entry
0709 13:00:43.664956 (+215092us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:43.681816 (+ 16860us) log.cc:844] Serialized 14 byte log entry
0709 13:00:43.878394 (+196578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.878500 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.191119 (+312619us) raft_consensus.cc:1685] finished
0709 13:00:44.191158 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.191443 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":230084,"prepare.queue_time_us":18874,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":251438,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":8952,"thread_start_us":673719,"threads_started":2,"wal-append.queue_time_us":738669}
W20260709 13:00:44.273806  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 234696us
I20260709 13:00:43.814476  4519 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.357s
I20260709 13:00:44.234292  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:44.342010  4427 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:44.343273  4423 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.744s	user 0.012s	sys 0.000s
I20260709 13:00:44.343833  4457 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:43.997313  4449 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:44.346704  3963 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:44.133662  4373 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.613s	user 0.015s	sys 0.000s
I20260709 13:00:44.017846  4345 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.323s
I20260709 13:00:44.191799  4522 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:44.540647  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1669) took 1862 ms (client timeout 2199 ms). Trace:
W20260709 13:00:44.474110  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 495033us
I20260709 13:00:44.377820  1702 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:44.560537  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:44.508198  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2203) took 1471 ms. Trace:
I20260709 13:00:44.528184  1634 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:44.561904  1552 rpcz_store.cc:276] 0709 13:00:43.036631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.470725 (+434094us) service_pool.cc:224] Handling call
0709 13:00:44.507921 (+1037196us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b5400143054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.508118 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026244864}
I20260709 13:00:44.280006  1701 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:00:44.562912  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 328702us
I20260709 13:00:44.563971  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:44.348009  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 68366.6us
W20260709 13:00:44.554250  1706 rpcz_store.cc:269] 0709 13:00:42.678652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.882936 (+204284us) service_pool.cc:224] Handling call
0709 13:00:44.540329 (+1657393us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b540016c454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.540556 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495822208}
I20260709 13:00:44.565074  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:44.565693  1557 raft_consensus.cc:2410] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:00:44.519893  2352 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 760) took 2230 ms (client timeout 1873 ms). Trace:
I20260709 13:00:44.378494  1555 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:44.461937  2317 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.018s	sys 0.036s
I20260709 13:00:44.393719  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2283) took 3880 ms. Trace:
I20260709 13:00:44.570864  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:44.571843  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 335256us
I20260709 13:00:44.574721  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:44.280509  4504 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:44.575735  4504 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:44.236789  4458 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.693s	user 0.016s	sys 0.006s
W20260709 13:00:44.567771  2317 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.019s	sys 0.037s
W20260709 13:00:44.591436  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 885371us
W20260709 13:00:44.485868  1602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.002s	sys 0.020s
W20260709 13:00:44.593830  1602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.003s	sys 0.020s
W20260709 13:00:44.194936  4537 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.227s	user 0.000s	sys 0.000s
I20260709 13:00:44.188328  4418 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:44.253435  4430 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:44.375458  4415 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.697s	user 0.010s	sys 0.000s
W20260709 13:00:44.539729  4510 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.002s
I20260709 13:00:44.518787  4435 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:44.564731  4522 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.566742  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:44.583868  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:44.137113  4502 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:44.098743  4509 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:44.581871  4504 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:44.568406  4501 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:44.235460  4252 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:44.548525  4554 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.193894  4388 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:44.566334  2352 rpcz_store.cc:269] 0709 13:00:42.289034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.451262 (+162228us) service_pool.cc:224] Handling call
0709 13:00:44.519582 (+2068320us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54007bff54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.519760 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3950039808}
I20260709 13:00:44.618913  4531 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:44.687904  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2284) took 3215 ms. Trace:
I20260709 13:00:44.688257  4531 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.688946  4531 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:44.597064  4458 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:44.594276  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 639906us
I20260709 13:00:44.691496  4458 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:44.691819  1602 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:44.692436  1707 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:00:44.279886  4529 log.cc:927] Time spent T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.087s	user 0.002s	sys 0.000s
I20260709 13:00:44.599313  3963 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:44.270412  4113 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.236120  4251 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.347693  4527 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:44.513352  4432 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.282s	user 0.010s	sys 0.000s
W20260709 13:00:44.407191  4498 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.901s	user 0.003s	sys 0.006s
I20260709 13:00:44.270711  4337 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.599004  1633 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:44.617901  4509 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:44.610803  4522 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.403255  4401 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:44.608533  2349 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:44.587695  4462 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.559513  4425 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:44.614634  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:44.616276  4519 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:44.617125  4550 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:44.576727  4524 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:44.582146  4416 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.270509  4363 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.580700  1674 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:44.617879  4415 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:44.685760  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 101971us
I20260709 13:00:44.618095  4510 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.687515  1632 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:44.623405  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 56834.6us
W20260709 13:00:44.639788  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.704124  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2202) took 1667 ms (client timeout 1978 ms). Trace:
I20260709 13:00:44.506616  4367 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:44.766513  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:44.571262  1627 rpcz_store.cc:276] 0709 13:00:40.513859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.569143 (+3055284us) service_pool.cc:224] Handling call
0709 13:00:43.602503 (+ 33360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.621202 (+ 18699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.621236 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:43.635020 (+ 13784us) log.cc:844] Serialized 26 byte log entry
0709 13:00:43.883519 (+248499us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:43.920354 (+ 36835us) log.cc:844] Serialized 14 byte log entry
0709 13:00:43.921124 (+   770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.921242 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.393408 (+472166us) raft_consensus.cc:1685] finished
0709 13:00:44.393432 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.393639 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2447,"prepare.queue_time_us":34709,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":79,"raft.queue_time_us":126113,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":38647,"thread_start_us":192521,"threads_started":1,"wal-append.queue_time_us":675332}
I20260709 13:00:44.608206  4430 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:44.596728  4549 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:44.619387  4420 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.992s	user 0.013s	sys 0.009s
W20260709 13:00:44.619809  4445 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.028s	user 0.013s	sys 0.000s
W20260709 13:00:44.774181  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 199513us
I20260709 13:00:44.610428  4502 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:44.743424  4550 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.778923  1553 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:44.745755  4345 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:44.599223  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:44.688218  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:44.686089  4435 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:44.686946  4252 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:44.736600  1552 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:44.730993  1631 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:44.729913  4251 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:44.728531  4432 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.740628  1635 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:44.689173  4493 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.717s	user 0.008s	sys 0.003s
I20260709 13:00:44.721550  4527 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.630476  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:44.779611  4420 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.014s	sys 0.010s
I20260709 13:00:44.759891  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:44.688366  1628 rpcz_store.cc:276] 0709 13:00:41.472508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.658319 (+2185811us) service_pool.cc:224] Handling call
0709 13:00:43.658709 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.659127 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.659176 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:43.659790 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 13:00:43.857979 (+198189us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:43.858026 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.858125 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.687609 (+829484us) raft_consensus.cc:1685] finished
0709 13:00:44.687653 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.687841 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15421,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":9868,"thread_start_us":160862,"threads_started":1,"wal-append.queue_time_us":192673}
W20260709 13:00:44.764430  1556 rpcz_store.cc:269] 0709 13:00:43.036341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.156547 (+120206us) service_pool.cc:224] Handling call
0709 13:00:44.703727 (+1547180us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b5400337cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.704007 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1947985280}
I20260709 13:00:44.802331  1556 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:00:44.803071  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1718) took 1566 ms. Trace:
I20260709 13:00:44.803468  1556 rpcz_store.cc:276] 0709 13:00:43.237210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.801679 (+1564469us) service_pool.cc:224] Handling call
0709 13:00:44.802054 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:44.802782 (+   728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.803010 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:44.805740  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45977.7us
I20260709 13:00:44.781131  4502 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:44.771046  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 77927.4us
I20260709 13:00:44.685333  4554 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.391862  4548 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.229s
I20260709 13:00:44.775218  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:44.394469  4494 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.739338  4427 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 13:00:44.761224  4500 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.966s	user 0.009s	sys 0.004s
I20260709 13:00:44.716120  4418 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:44.829190  4337 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:44.829586  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2300) took 2227 ms (client timeout 2491 ms). Trace:
W20260709 13:00:44.830170  1632 rpcz_store.cc:269] 0709 13:00:42.601848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.695676 (+ 93828us) service_pool.cc:224] Handling call
0709 13:00:44.829161 (+2133485us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54003b92d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.829404 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4206982912}
I20260709 13:00:44.831768  1632 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:44.832278  4548 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:44.832419  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2061) took 3233 ms. Trace:
I20260709 13:00:44.832780  4548 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:44.583551  4423 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.761106  4388 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:44.786954  4509 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.789409  1706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:00:44.791772  1552 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:00:44.794541  4491 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.307s	user 0.002s	sys 0.002s
I20260709 13:00:44.798022  2351 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:44.758488  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 177864us
W20260709 13:00:44.799425  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 200422us
I20260709 13:00:44.766345  4367 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:44.834542  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2193) took 2507 ms. Trace:
I20260709 13:00:44.781672  4445 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:44.778923  4524 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:44.807616  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:44.912436  4496 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.298s	user 0.009s	sys 0.006s
W20260709 13:00:44.912572  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 105029us
I20260709 13:00:44.913122  4496 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:44.913740  4432 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:44.914572  1554 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:44.914947  4373 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:44.784593  4435 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:44.915628  4516 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.904s	user 0.007s	sys 0.006s
I20260709 13:00:44.817063  1630 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:44.748147  4416 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:44.782634  4550 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:44.816363  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:44.916891  2352 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:44.823659  4420 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.900825  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1717) took 1867 ms. Trace:
W20260709 13:00:44.839380  4552 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:00:44.802693  4527 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:44.939244  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2184) took 3257 ms. Trace:
I20260709 13:00:44.901937  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:44.832868  1632 rpcz_store.cc:276] 0709 13:00:41.598971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.831210 (+3232239us) service_pool.cc:224] Handling call
0709 13:00:44.831592 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.832147 (+   555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.832329 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:44.902344  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:44.941529  1556 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:44.902774  1706 raft_consensus.cc:606] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:44.833861  4462 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:44.844853  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 480) took 2799 ms. Trace:
I20260709 13:00:44.945578  4509 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.946170  1632 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:44.946867  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2067) took 3312 ms. Trace:
I20260709 13:00:44.947160  1632 rpcz_store.cc:276] 0709 13:00:41.634913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.940954 (+3306041us) service_pool.cc:224] Handling call
0709 13:00:44.945982 (+  5028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.946570 (+   588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.946781 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:44.818816  4252 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
W20260709 13:00:44.968909  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.900840  1634 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:44.950649  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2288) took 2704 ms (client timeout 1760 ms). Trace:
W20260709 13:00:44.972615  1635 rpcz_store.cc:269] 0709 13:00:42.246685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.348336 (+101651us) service_pool.cc:224] Handling call
0709 13:00:44.950415 (+2602079us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b5400010ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.950567 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5397373696}
I20260709 13:00:44.904798  1557 rpcz_store.cc:276] 0709 13:00:42.326844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.534856 (+208012us) service_pool.cc:224] Handling call
0709 13:00:44.834293 (+2299437us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b54001c0054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.834479 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4627202816}
I20260709 13:00:44.905747  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:44.903849  4549 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:44.948776  4530 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.836s	user 0.001s	sys 0.001s
W20260709 13:00:44.837715  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 62598us
I20260709 13:00:44.907675  1627 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:44.910673  4367 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.915012  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:44.832516  4419 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.597s	user 0.008s	sys 0.003s
I20260709 13:00:44.846001  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2054) took 3302 ms. Trace:
I20260709 13:00:44.590529  4457 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:44.848301  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.921847  1552 rpcz_store.cc:276] 0709 13:00:43.033288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.785757 (+1752469us) service_pool.cc:224] Handling call
0709 13:00:44.786222 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.900429 (+114207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.900750 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:45.036526  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.985293  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 79691us
I20260709 13:00:45.038559  4367 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:45.039027  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:44.848948  1636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2078) took 2423 ms (client timeout 1729 ms). Trace:
I20260709 13:00:44.923316  4420 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:45.039914  1636 rpcz_store.cc:269] 0709 13:00:42.425984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.498980 (+ 72996us) service_pool.cc:224] Handling call
0709 13:00:44.848611 (+2349631us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54001383d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.848854 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3451840384}
I20260709 13:00:44.904563  4510 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.041913  1708 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:44.805611  4493 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.043203  3963 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.043845  3963 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.915938  4435 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:45.045650  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.045893  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.046478  1704 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:45.047338  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.048647  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:44.930300  4556 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:00:44.905450  4500 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:44.931007  4548 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:44.932197  4445 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:44.931185  1558 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:45.049947  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.050366  4548 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:45.051538  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.052177  2344 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:45.052887  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:44.934231  4251 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:44.936571  4524 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:44.935968  4418 consensus_queue.cc:1175] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" responder_term: 7 status { last_received { term: 5 index: 8 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:44.936411  4554 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:44.848021  4410 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:44.938740  1561 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:45.056723  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:44.938668  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1710) took 2264 ms (client timeout 1672 ms). Trace:
I20260709 13:00:44.943316  1706 raft_consensus.cc:614] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:44.849776  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2177) took 3544 ms. Trace:
I20260709 13:00:45.068269  1702 rpcz_store.cc:276] 0709 13:00:41.487128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.203946 (+1716818us) service_pool.cc:224] Handling call
0709 13:00:43.204329 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.572060 (+1367731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.849443 (+277383us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002a67d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:44.849699 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2697598720}
I20260709 13:00:44.852638  4452 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:45.037165  4508 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.145s	user 0.009s	sys 0.005s
I20260709 13:00:45.075646  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:44.939700  1560 rpcz_store.cc:276] 0709 13:00:41.681946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.938224 (+2256278us) service_pool.cc:224] Handling call
0709 13:00:43.938669 (+   445us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:43.939113 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.939146 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:43.939856 (+   710us) log.cc:844] Serialized 52 byte log entry
0709 13:00:44.099830 (+159974us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:44.099902 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.100035 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.838612 (+738577us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:44.938879 (+100267us) raft_consensus.cc:1685] finished
0709 13:00:44.938904 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.939171 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85193,"raft.run_cpu_time_us":527,"raft.run_wall_time_us":82835,"spinlock_wait_cycles":27117952,"thread_start_us":159426,"threads_started":1,"wal-append.queue_time_us":600011}
I20260709 13:00:44.854907  4363 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:44.846509  4113 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:44.840919  4415 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:44.948850  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 515) took 3312 ms. Trace:
I20260709 13:00:45.079782  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1694) took 5595 ms. Trace:
W20260709 13:00:44.964977  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2170) took 3662 ms (client timeout 2656 ms). Trace:
I20260709 13:00:44.950103  4423 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:44.962819  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1657) took 4245 ms (client timeout 3168 ms). Trace:
I20260709 13:00:45.056502  4251 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:45.086201  1701 rpcz_store.cc:269] 0709 13:00:40.716966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.507344 (+1790378us) service_pool.cc:224] Handling call
0709 13:00:44.962553 (+2455209us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54002e81d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.962738 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3930469760}
I20260709 13:00:44.951561  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2198) took 2542 ms. Trace:
I20260709 13:00:45.056342  4521 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.087342  1554 rpcz_store.cc:276] 0709 13:00:42.409230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.380536 (+1971306us) service_pool.cc:224] Handling call
0709 13:00:44.403915 (+ 23379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.914973 (+511058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.951276 (+ 36303us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b54002cbd54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:44.951488 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173856128}
I20260709 13:00:44.807224  4311 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:45.089103  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:44.952610  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2272) took 5769 ms. Trace:
I20260709 13:00:45.091853  1630 rpcz_store.cc:276] 0709 13:00:39.183022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.351023 (+4168001us) service_pool.cc:224] Handling call
0709 13:00:43.418706 (+ 67683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:44.916197 (+1497491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.952327 (+ 36130us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400138dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:44.952540 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3215146368}
I20260709 13:00:45.030457  1633 rpcz_store.cc:276] 0709 13:00:41.543444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.040516 (+2497072us) service_pool.cc:224] Handling call
0709 13:00:44.040875 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.732055 (+691180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.845723 (+113668us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b54003b83d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:44.845927 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1283052416}
I20260709 13:00:45.004659  1556 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:00:45.030934  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 116003us
I20260709 13:00:45.034567  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.048710  1704 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:45.093572  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.093904  1704 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:44.851899  4381 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:45.054697  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.069268  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.049341  2348 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
W20260709 13:00:45.057441  1555 rpcz_store.cc:269] 0709 13:00:42.674258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.704735 (+ 30477us) service_pool.cc:224] Handling call
0709 13:00:44.938383 (+2233648us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b5400320ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.938572 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3845399808}
I20260709 13:00:45.053311  2344 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:44.854429  4398 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:45.072412  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.074105  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:44.962888  4327 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:45.054898  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 288465us
I20260709 13:00:45.237003  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:45.075579  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.077562  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:45.077824  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 262988us
I20260709 13:00:45.237426  4502 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.238114  1705 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:45.239234  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.075820  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.239705  4502 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:44.945326  2346 rpcz_store.cc:276] 0709 13:00:42.049028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.851827 (+802799us) service_pool.cc:224] Handling call
0709 13:00:42.870095 (+ 18268us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:43.415559 (+545464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.415603 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:43.461054 (+ 45451us) log.cc:844] Serialized 52 byte log entry
0709 13:00:43.622529 (+161475us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:43.669062 (+ 46533us) log.cc:844] Serialized 14 byte log entry
0709 13:00:43.782244 (+113182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.782358 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.737403 (+955045us) raft_consensus.cc:1685] finished
0709 13:00:44.737432 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.844586 (+107154us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b540055ffd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.844759 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8619,"prepare.queue_time_us":45015,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":229711,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":26254,"spinlock_wait_cycles":40445696,"thread_start_us":722848,"threads_started":3,"wal-append.queue_time_us":691483}
I20260709 13:00:45.240417  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.240304  4435 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.241081  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:45.080600  1707 rpcz_store.cc:269] 0709 13:00:41.302323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.580335 (+1278012us) service_pool.cc:224] Handling call
0709 13:00:44.964719 (+2384384us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b5400078354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:44.964896 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4813336448}
I20260709 13:00:45.080250  1559 rpcz_store.cc:276] 0709 13:00:39.484779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.563193 (+3078414us) service_pool.cc:224] Handling call
0709 13:00:42.573547 (+ 10354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:42.573915 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:42.573945 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:42.574570 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 13:00:43.181212 (+606642us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:43.181333 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.181420 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.793742 (+1612322us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:45.076159 (+282417us) raft_consensus.cc:1685] finished
0709 13:00:45.076185 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.079712 (+  3527us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1272,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":113,"spinlock_wait_cycles":56032512,"thread_start_us":606212,"threads_started":1,"wal-append.queue_time_us":813051}
I20260709 13:00:45.242134  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.242288  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.242352  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.243249  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.243803  1707 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:45.243727  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.244673  2346 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:45.029670  4416 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:45.078312  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.245808  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.083961  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.080049  1632 rpcz_store.cc:276] 0709 13:00:41.636030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.947910 (+3311880us) service_pool.cc:224] Handling call
0709 13:00:44.948320 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.948533 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:44.948557 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:44.948597 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:44.948627 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.948676 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.948779 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:45.246995  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.247133  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.247783  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 518) took 3611 ms. Trace:
I20260709 13:00:45.248267  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.248170  1632 rpcz_store.cc:276] 0709 13:00:41.636818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.246964 (+3610146us) service_pool.cc:224] Handling call
0709 13:00:45.247278 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.247459 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:45.247485 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.247527 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:45.247570 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.247622 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.247725 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:45.248813  1709 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:45.059923  4494 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.090018  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.090811  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:44.935171  4516 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.093724  2352 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:45.250875  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.251641  1703 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:45.252403  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.036890  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.256587  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.072320  4501 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.257542  4501 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.258674  2350 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:45.259864  4501 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:45.099391  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.261830  4435 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:45.262346  1555 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:45.261843  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2057) took 3700 ms. Trace:
I20260709 13:00:45.263283  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.263969  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2079) took 2837 ms. Trace:
I20260709 13:00:45.264329  1632 rpcz_store.cc:276] 0709 13:00:42.426326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.249210 (+2822884us) service_pool.cc:224] Handling call
0709 13:00:45.263406 (+ 14196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.263641 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:45.263670 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.263717 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:45.263756 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.263807 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.263912 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:45.253386  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.100338  1633 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:45.101776  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.266695  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 516) took 3630 ms. Trace:
I20260709 13:00:45.266960  1701 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:45.247789  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 524) took 3479 ms. Trace:
I20260709 13:00:45.267621  1636 rpcz_store.cc:276] 0709 13:00:41.768575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.247006 (+3478431us) service_pool.cc:224] Handling call
0709 13:00:45.247334 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.247499 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:45.247524 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.247564 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:45.247594 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.247643 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.247738 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20224}
I20260709 13:00:45.268261  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.101691  4363 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:45.084828  4452 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:45.102533  4445 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:45.102880  4500 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.103098  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.300607  1706 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.104482  1629 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:45.104557  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2056) took 3560 ms. Trace:
I20260709 13:00:45.301667  1634 rpcz_store.cc:276] 0709 13:00:41.544046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.562445 (+3018399us) service_pool.cc:224] Handling call
0709 13:00:44.900636 (+338191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.972403 (+ 71767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.104344 (+131941us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54003b92d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:45.104488 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624473472}
I20260709 13:00:45.105265  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:45.104373  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 533) took 2674 ms (client timeout 2225 ms). Trace:
I20260709 13:00:45.303071  1560 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
W20260709 13:00:45.302937  1631 rpcz_store.cc:269] 0709 13:00:42.429844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.571797 (+141953us) service_pool.cc:224] Handling call
0709 13:00:45.104119 (+2532322us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b54001b6a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:45.104298 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4942421248}
W20260709 13:00:45.105891  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.106313  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.107861  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2180) took 3649 ms. Trace:
I20260709 13:00:45.304519  1553 rpcz_store.cc:276] 0709 13:00:41.458597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.503798 (+2045201us) service_pool.cc:224] Handling call
0709 13:00:43.504294 (+   496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.779387 (+1275093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.107485 (+328098us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540026b1d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:45.107769 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2930818816}
I20260709 13:00:44.952920  4531 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.111160  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.306366  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.307004  2347 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:45.111393  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.105640  1704 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.115345  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.116353  2345 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:00:45.116737  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.091473  4311 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:45.315438  1704 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.116977  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.117117  4419 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:45.116479  4327 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:45.120172  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.093377  4459 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.298244  4418 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:45.121524   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:45.072568  4524 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.125242  4522 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.127399  1557 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:45.096236  4373 raft_consensus.cc:2764] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:00:45.139413  1675 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:45.323354  4574 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:00:45.370510  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.319837  4311 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:45.117026  4425 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.147604  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.151408  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2194) took 2812 ms. Trace:
W20260709 13:00:45.160954  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.171739  4527 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.174351  4420 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.111078  4251 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:45.119616  4410 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:45.182128  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 93194.3us
I20260709 13:00:45.146013  4398 consensus_queue.cc:1059] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:00:45.184165  4493 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.185721  4504 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.188293  4430 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.180559  4550 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.207594  2351 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
W20260709 13:00:45.211274  4573 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:00:45.216138  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2190) took 2891 ms. Trace:
I20260709 13:00:45.390385  1602 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:45.391745  1561 rpcz_store.cc:276] 0709 13:00:42.325110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.940978 (+1615868us) service_pool.cc:224] Handling call
0709 13:00:44.938534 (+997556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.068513 (+129979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.215790 (+147277us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b5400320ad0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:45.216025 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2293268864}
I20260709 13:00:45.393873  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.235057  4553 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.396602  4553 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.398798  4430 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.361534  4459 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.090940  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.400909  4553 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.266709  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.401852  1703 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 3.
I20260709 13:00:45.402009  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.270202  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:45.402884  4459 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:45.403561  1632 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:45.350888  4419 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.248162  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.406234  1554 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:45.407344  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.407648  1710 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:45.274739  1707 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:45.270773  1676 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:45.409030  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.263029  1627 rpcz_store.cc:276] 0709 13:00:41.561099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.791209 (+3230110us) service_pool.cc:224] Handling call
0709 13:00:44.814757 (+ 23548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.031660 (+216903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.261594 (+229934us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b5400197b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:45.261779 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101832960}
I20260709 13:00:45.409984  1707 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:45.408972  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.277915  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.410874  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.264436  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.411279  1554 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:45.411938  1555 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:45.290869  1525 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:45.312563  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 197274us
I20260709 13:00:45.412885  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.308909  1630 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:45.413460  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.348976  1635 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:45.328305  4494 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:45.344712   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:45.347918  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.362186  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.365593  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:45.366050  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 226723us
I20260709 13:00:45.103514  3963 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:45.358790  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.361673  1526 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:45.359061  1709 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:45.372287  4500 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.359009  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:45.360132  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.371418  1560 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:45.080340  4508 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.322582  4506 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.316730  4435 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:45.377075  4509 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.381891  1552 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:45.381575  1559 raft_consensus.cc:2368] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11.
I20260709 13:00:45.395836  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.460958  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2276) took 5986 ms. Trace:
I20260709 13:00:45.461501  1629 rpcz_store.cc:276] 0709 13:00:39.474666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.489787 (+4015121us) service_pool.cc:224] Handling call
0709 13:00:45.104272 (+1614485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.301300 (+197028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.460655 (+159355us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54001b6a50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:45.460878 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3629360896}
I20260709 13:00:45.374961  1558 rpcz_store.cc:276] 0709 13:00:42.339467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.977416 (+1637949us) service_pool.cc:224] Handling call
0709 13:00:43.977893 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.050359 (+1072466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.151062 (+100703us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b54001fd454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:45.151322 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2191808384}
I20260709 13:00:45.388759  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:45.369568  4452 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.400627  4522 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:45.267100  1633 rpcz_store.cc:276] 0709 13:00:41.636293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.099938 (+3463645us) service_pool.cc:224] Handling call
0709 13:00:45.100197 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.266430 (+166233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.266636 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:45.237139  4549 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.402964  2351 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:45.405453  4419 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.378175  4457 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:00:45.490823  4457 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:45.491595  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.491847  4457 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.492813  1633 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:45.479918  4459 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.494072  2351 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:45.494782  4459 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.489463  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:45.496951  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.409806  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2297) took 2895 ms. Trace:
W20260709 13:00:45.319464  1601 connection.cc:256] RPC call timeout handler was delayed by -1.1149820580000096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 13:00:45.498451  1601 connection.cc:256] RPC call timeout handler was delayed by -1.1149820580000096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
I20260709 13:00:45.498304  1632 rpcz_store.cc:276] 0709 13:00:42.510613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.404464 (+2893851us) service_pool.cc:224] Handling call
0709 13:00:45.404841 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.405089 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:45.405121 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.405177 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:45.405228 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.405296 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.405413 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:45.305246  4521 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:45.415524  1554 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:45.512291  1561 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:45.428514  1556 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 4: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:45.448697  1553 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:45.513127  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0708820579999951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
I20260709 13:00:45.427724  1705 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:45.418345  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.514271  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:45.514945  1557 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:45.389808  2318 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.416986  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.449169  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:45.530161  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 140454us
I20260709 13:00:45.530452  1709 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:45.410897  1627 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:45.454524  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 183833us
I20260709 13:00:45.467165  4435 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:45.412900  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 122185us
I20260709 13:00:45.387034  4251 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:45.456979  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 66675.5us
I20260709 13:00:45.457824  1701 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:45.457901  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:45.459120  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 97280.5us
I20260709 13:00:45.411509  1707 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:45.467960  2352 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
W20260709 13:00:45.467972  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 79317.3us
I20260709 13:00:45.414919  4449 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:45.483891  4549 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:45.550486  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:45.551208  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.551137  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:45.544631  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.484500  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:45.416460  4524 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.489406  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.490135  4527 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.493973  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.524739  4493 raft_consensus.cc:2764] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
W20260709 13:00:45.511785  1674 connection.cc:256] RPC call timeout handler was delayed by -1.7500629390000313s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
I20260709 13:00:45.464720  4494 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.516192  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.420856  4504 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.523020  4566 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:45.535007  4571 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:45.422623  4313 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:45.453845  4427 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:45.398821  4550 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.535630  4424 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:00:45.599396  4424 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:45.599838  4424 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:45.600569  4418 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:45.601018  4418 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:45.467005  4410 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:45.527974  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:45.602483  4496 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:45.603103  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.604394  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:45.605171  4508 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:45.605319  4516 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.459054  4416 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.357625  2347 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:45.606241  4416 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:45.607230  4579 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.113s	user 0.002s	sys 0.000s
I20260709 13:00:45.607457  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.479823  4542 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:45.456238  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.608100  4251 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.608764  1704 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:45.610217  4572 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.610773  4572 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.517865  4419 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.612175  4459 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:45.612499  4419 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.589452  4313 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:45.614748  4419 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:45.536072  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.614662  4313 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.540643  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.615967  4313 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:45.616425  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.617659  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.618098  4531 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.618355  2352 raft_consensus.cc:2368] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5.
I20260709 13:00:45.537614  4430 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:00:45.619171  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.600823  4424 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.619526  4430 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:45.619834  2352 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:45.619990  4430 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:45.620282  4424 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:45.620663  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.541163  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:45.557915  4501 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:00:45.621843  4572 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:45.622110  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 81053.8us
I20260709 13:00:45.622539  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.622990  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:45.559630  4433 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:00:45.624543  4433 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:45.625042  4433 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:45.625274  4416 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:45.547078  1706 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:45.625941  4416 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.626670  4416 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:45.544642  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:45.628062  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 83518.4us
I20260709 13:00:45.628710  4542 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:45.535105  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.629669  4542 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:45.630244  4572 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:45.630689  4572 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:45.630978  1561 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:45.631588  4572 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.541184  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.536387  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.633121  4572 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:45.633443  1632 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:45.633848  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 92736.3us
I20260709 13:00:45.634786  1632 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:45.555357  4327 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:45.636512  4327 consensus_queue.cc:1243] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:00:45.637009  4571 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:45.637533  4571 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:45.558583  4363 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:00:45.637974  4571 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:45.558408  4445 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:00:45.639312  4445 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:45.639652  4445 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:45.640470  4445 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.626452  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.643571  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.644301  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.645606  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.626406  4433 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.565603  1553 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:45.563318  1635 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:45.567134  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.568781  1703 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:45.649634  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.650868  1703 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:45.414842  1630 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:45.654313  1630 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:45.656111  1630 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:45.573338  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:45.659035  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:45.659808  2353 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:45.664471  1703 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:45.667140  4524 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:45.668234  4524 raft_consensus.cc:3055] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:45.668627  4524 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:45.562072  1558 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:45.670564  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:45.671056  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:45.575435  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:45.671407  1558 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:45.672189  1709 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:45.674145  1709 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:45.675886  1709 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:45.666683  4539 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:00:45.678402  4539 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:45.678821  4539 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:45.680099  1559 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:45.679867  4539 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.550125  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.579830  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.449870  1634 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:45.524032  4497 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.581283  1702 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
W20260709 13:00:45.582666  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 66603.9us
I20260709 13:00:45.381644  4425 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.532195  3963 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:45.591238  4566 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:45.762418  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:45.593912  4577 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:00:45.597929  4427 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:45.623706  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.764286  1629 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:00:45.635757  2318 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:45.643667  4398 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:45.646307  1701 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:45.647526  1553 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:45.563086  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.768916  1633 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:45.567358  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.469882  4500 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.645080  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:45.501230  4509 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.621212  2352 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:45.698443  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:45.740073  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:00:45.788045  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 48122.8us
I20260709 13:00:45.739195  1555 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:45.789940  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:45.790972  1633 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:45.794126  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:45.745616  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:45.795759  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50307.1us
W20260709 13:00:45.801759  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.803939  1631 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:45.736927  4493 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.826347  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:45.830978  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.660660  4435 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.621054  4430 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:45.839444  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45471.1us
I20260709 13:00:45.842346  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:45.779709  4251 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:45.756475  1558 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:45.758798  4510 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:45.759828  4459 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:45.762197  1559 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:00:45.779373  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 143721us
I20260709 13:00:45.756526  4327 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:00:45.875617  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:45.748766  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 198695us
I20260709 13:00:45.877753  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:45.777185  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 197563us
I20260709 13:00:45.681414  4539 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.809185  1710 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:45.755893  4497 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.885462  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.811524  1527 leader_election.cc:400] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5.
I20260709 13:00:45.811690  1629 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:00:45.754108  4252 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:45.757036  4445 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:45.844578  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.752839  1707 raft_consensus.cc:2410] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:45.779440  4398 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:45.777963  3963 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:45.851397  4423 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.780973  4508 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.853236  1558 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:45.953873  1558 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:45.954162  1706 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:45.955718  1706 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:45.879845  4435 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:45.872335  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.872927  1555 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:45.754396  4501 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 13:00:45.813289  4531 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:45.813730  4527 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:45.888674  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:45.976191  2347 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:45.976593  1629 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 3: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:45.889564  1527 leader_election.cc:403] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:45.886883  1710 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:45.895857  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.916891  1633 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:45.979332  1630 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:45.919803  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.867218  4089 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:45.860335  4419 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:45.991627  1675 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:45.753775  1704 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:45.904737  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2295) took 3474 ms. Trace:
I20260709 13:00:45.943347  1701 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:45.792001  4310 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:45.966328  1634 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:45.968883  1703 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:45.895803  4430 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:45.839305  4493 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:45.972635  1560 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:45.957835  4363 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:45.967482  4427 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:00:45.980859  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 61332.4us
I20260709 13:00:45.993716  1636 rpcz_store.cc:276] 0709 13:00:42.430029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.268690 (+2838661us) service_pool.cc:224] Handling call
0709 13:00:45.280503 (+ 11813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.280768 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:45.525272 (+244504us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.904041 (+378769us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.904119 (+    78us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:45.904297 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.904402 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.904579 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":328768,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":45,"raft.queue_time_us":1201,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":30155,"thread_start_us":592216,"threads_started":2}
I20260709 13:00:46.021050  1676 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:45.916165  4449 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:45.985524  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 89746.7us
I20260709 13:00:46.022871  1674 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:45.988279  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:46.024708  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:46.025568  1629 raft_consensus.cc:2368] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5.
I20260709 13:00:46.026497  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:46.030725  1629 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:45.989745  1706 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:45.945672  4459 raft_consensus.cc:2764] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:00:45.776865  4524 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:46.075686  1633 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:00:45.838567  4418 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:46.084793  1633 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:46.010618  2352 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:46.033749  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:46.093046  1627 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:46.037523  4398 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:46.012310  4424 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:45.963487  4423 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:46.045817  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:46.046384  4416 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.001144  4089 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:45.984122  4550 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:45.986159  4313 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:46.050033  4494 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:46.053278  4509 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:46.054812  4493 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:45.960804  4566 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:46.118259  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 81894.4us
I20260709 13:00:45.972466  4531 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:46.120489  4531 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:46.120577  4432 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:46.122501  4089 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.060482  4425 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:46.065689  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.020373  4310 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:46.127369  4311 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.748s	user 0.009s	sys 0.002s
I20260709 13:00:46.127913  4311 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:46.066704  4500 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:46.003530  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 118184us
W20260709 13:00:46.072863  4506 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.606s	user 0.015s	sys 0.001s
I20260709 13:00:46.078173  1629 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:00:46.078936  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 57967.2us
I20260709 13:00:46.131361  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:46.079867  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 57113.4us
I20260709 13:00:46.132504  1674 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.041828  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:46.104842  4388 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:46.108430  4549 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.527s	user 0.002s	sys 0.007s
I20260709 13:00:46.108628  4423 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:46.108758  4521 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.109674  4398 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.113811  4418 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.102366  4327 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:46.114459  4550 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.000869  4419 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:46.067339  3963 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:46.077314  4559 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:00:46.165982  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.054831  4497 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:46.139343  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:46.185004  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45783.6us
I20260709 13:00:46.141764  4506 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:46.145648  4313 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.146328  4424 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:46.171576  2349 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:46.135843  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:46.196060  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 60340us
I20260709 13:00:46.188097  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:46.173422  4550 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.171811  4419 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.104197  4457 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:46.212006  4311 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:46.162220  4549 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:46.189857  4388 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:46.196245  4571 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.558s	user 0.001s	sys 0.012s
W20260709 13:00:46.260918  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.253962  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.220149  4494 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:46.222913  4457 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.207515  4310 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:46.243326  4506 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:46.296307  1552 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:46.299774  4549 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:46.295599  4494 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.296458  4423 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.308346  4252 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.333889  4494 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.334077  4425 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:46.309659  4419 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:46.337450  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 461856us
I20260709 13:00:46.338569  2318 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:46.339506  1632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.010s	sys 0.033s
W20260709 13:00:46.339982  1632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.011s	sys 0.033s
I20260709 13:00:46.346652  1708 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:00:46.257905  4493 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:00:46.380671  1555 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:00:46.381351  1555 pending_rounds.cc:85] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 4
I20260709 13:00:46.381922  1555 pending_rounds.cc:107] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:00:46.382261  1555 raft_consensus.cc:2889] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:46.338636  4457 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:46.376294  1705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.006s	sys 0.025s
I20260709 13:00:46.381259  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 392) took 1351 ms. Trace:
I20260709 13:00:46.377092  4310 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:46.339260  4252 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.261927  4522 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.110s
W20260709 13:00:46.387018  1705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.006s	sys 0.025s
I20260709 13:00:46.280161  4571 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:46.396180  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 479) took 1277 ms. Trace:
W20260709 13:00:46.398604  1561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.066s	sys 0.150s
W20260709 13:00:46.399084  1561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.066s	sys 0.150s
I20260709 13:00:46.387295  1554 rpcz_store.cc:276] 0709 13:00:45.029917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.556585 (+526668us) service_pool.cc:224] Handling call
0709 13:00:45.588645 (+ 32060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.588934 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:45.617850 (+ 28916us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.374901 (+757051us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:46.374974 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:46.375042 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.380906 (+  5864us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:46.381165 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29300,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":84478,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":52,"thread_start_us":755293,"threads_started":2,"wal-append.queue_time_us":599698}
W20260709 13:00:46.401615  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.015s	sys 0.021s
I20260709 13:00:46.395538  4432 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:46.343950  4423 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:46.396725  1705 rpcz_store.cc:276] 0709 13:00:45.119067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.682867 (+563800us) service_pool.cc:224] Handling call
0709 13:00:45.683212 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.683457 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:45.683826 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.393409 (+709583us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.393681 (+   272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:46.395229 (+  1548us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:46.395297 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.395962 (+   665us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:46.396117 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":345,"prepare.queue_time_us":52369,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":78,"thread_start_us":706780,"threads_started":1}
W20260709 13:00:46.411469  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.015s	sys 0.022s
I20260709 13:00:46.405154  2353 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:46.418548  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 720192us
I20260709 13:00:46.390239  4425 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
W20260709 13:00:46.424218  4586 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:00:46.375725  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:00:46.429260  4539 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.000s
W20260709 13:00:46.435195  1557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.012s	sys 0.014s
W20260709 13:00:46.441682  1557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.013s	sys 0.014s
I20260709 13:00:46.416299  4416 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5.
I20260709 13:00:46.403760  4531 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:46.445007  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:46.458556  4599 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.459298  4599 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:46.436630  4363 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:46.468315  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 46872.1us
W20260709 13:00:46.443166  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 67517.9us
I20260709 13:00:46.467113  4493 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:46.448351  4310 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:46.436296  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 639286us
I20260709 13:00:46.438926  4527 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:46.472263  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:46.476517  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:46.477103  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.479787  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:46.454985  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2214) took 1003 ms. Trace:
I20260709 13:00:46.480590  2348 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:46.484228  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:46.496865  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:46.501821  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.503139  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:46.472371  4493 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:46.505151  2351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.021s	sys 0.004s
W20260709 13:00:46.508980  2351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.021s	sys 0.004s
I20260709 13:00:46.486027  1556 rpcz_store.cc:276] 0709 13:00:45.451804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.951160 (+499356us) service_pool.cc:224] Handling call
0709 13:00:45.951520 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.951774 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:46.013270 (+ 61496us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.448325 (+435055us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.454567 (+  6242us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:46.454630 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:46.454689 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.454763 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:46.454915 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3426,"prepare.queue_time_us":67367,"prepare.run_cpu_time_us":145,"prepare.run_wall_time_us":149,"raft.queue_time_us":7920,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":10866,"spinlock_wait_cycles":25698816,"thread_start_us":434209,"threads_started":1}
I20260709 13:00:46.513149  4311 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:46.513960  4596 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:46.514472  4596 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:46.514875  4596 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.480773  4430 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:00:46.518087  2345 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:46.519342  3963 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:46.519786  4433 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.472985  4416 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:46.478461  4310 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:46.531910  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 175359us
I20260709 13:00:46.532968  1601 leader_election.cc:400] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6.
I20260709 13:00:46.503562  4599 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:46.508190  4419 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:46.500634  4549 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:00:46.511484  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 780) took 1145 ms. Trace:
I20260709 13:00:46.526657  4416 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.543406  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:46.545540  1704 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.542856  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:46.539289  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:46.540558  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.540689  2351 rpcz_store.cc:276] 0709 13:00:45.366270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.495131 (+128861us) service_pool.cc:224] Handling call
0709 13:00:45.495565 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.495843 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:45.847927 (+352084us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.511018 (+663091us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:46.511095 (+    77us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:46.511166 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.511255 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:46.511407 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":680592,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":1361,"raft.queue_time_us":182008,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":131,"thread_start_us":963450,"threads_started":2,"wal-append.queue_time_us":88151}
I20260709 13:00:46.574277  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:46.576751  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.580185  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 478) took 1461 ms. Trace:
I20260709 13:00:46.581869  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2215) took 1129 ms. Trace:
I20260709 13:00:46.583776  1559 rpcz_store.cc:276] 0709 13:00:45.452131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.951351 (+499220us) service_pool.cc:224] Handling call
0709 13:00:45.951629 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.951826 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:46.413305 (+461479us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.581504 (+168199us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:46.581563 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:46.581625 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.581693 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:46.581809 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25620,"prepare.queue_time_us":70430,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"thread_start_us":559882,"threads_started":2}
I20260709 13:00:46.534454  4310 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.579907  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:46.580461  1704 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:46.573652  4416 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.606467  1632 pending_rounds.cc:85] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 4
I20260709 13:00:46.607228  1632 pending_rounds.cc:107] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:00:46.564715  4571 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:46.607648  1632 raft_consensus.cc:2889] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:46.538165  4425 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:00:46.580600  1709 rpcz_store.cc:276] 0709 13:00:45.118721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.676909 (+558188us) service_pool.cc:224] Handling call
0709 13:00:45.677274 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.685566 (+  8292us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:46.210455 (+524889us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.579809 (+369354us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:46.579864 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:46.579926 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.580001 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:46.580124 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52791,"prepare.queue_time_us":152433,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":6503,"raft.queue_time_us":92,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":3143,"thread_start_us":787962,"threads_started":2}
I20260709 13:00:46.535037  4252 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:46.616541  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.616730  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:46.579095  4572 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:46.624456  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:46.581423  4521 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:46.631269  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:46.633667  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:46.640214  1706 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:46.640394  1702 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:00:46.634565  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:46.635823  1630 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:46.546687  4539 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:00:46.636744  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:46.645720  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.646591  1525 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:46.649586  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:46.588606  4605 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.651157  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:46.653111  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:46.654619  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51766.6us
I20260709 13:00:46.539541  4608 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:00:46.659756  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 535) took 4229 ms. Trace:
I20260709 13:00:46.659921  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:46.587961  4508 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:00:46.656323  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:46.659130  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:46.666496  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:46.657361  2352 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:46.660214  1628 rpcz_store.cc:276] 0709 13:00:42.430690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.347813 (+2917123us) service_pool.cc:224] Handling call
0709 13:00:45.348125 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.348354 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:45.590156 (+241802us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.999729 (+409573us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.412534 (+412805us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b18003948a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:00:46.513847 (+101313us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b18003948a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:00:46.659253 (+145406us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:46.659322 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:46.659375 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.659555 (+   180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:46.659690 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":261766,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":9436,"raft.queue_time_us":61355,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":13663,"spinlock_wait_cycles":60595328,"thread_start_us":746955,"threads_started":3,"wal-append.queue_time_us":705614}
I20260709 13:00:46.655889  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:46.671504  1705 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:46.663295  2318 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.674242  2347 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:46.594615  4363 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:46.678490  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:46.678124  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 87221.2us
I20260709 13:00:46.638105  4572 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:46.674832  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:46.636861  4614 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.655411  4459 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:46.628609  3963 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:00:46.685989  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:46.698137  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.681363  2318 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:46.760089  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 78894.3us
I20260709 13:00:46.761534  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.682979  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.655397  4430 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.682684  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:46.595476  4527 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:46.655400  4506 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:46.712006  4597 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:46.667336  4606 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.599318  4493 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:46.782914  4608 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:46.777070  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 91159.8us
I20260709 13:00:46.682663  4572 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:46.684332  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:46.785431  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:46.795536  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 112633us
I20260709 13:00:46.796494  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:46.796784  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:46.800534  1525 leader_election.cc:400] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5.
W20260709 13:00:46.783396  4601 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:00:46.802742  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:46.680791  4612 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:46.765785  4509 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.688647  4423 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:46.769577  4311 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:46.789413  4424 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.766367  4623 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:46.666608  4605 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:46.684315  4613 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.143s
I20260709 13:00:46.842970  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1729) took 1733 ms. Trace:
W20260709 13:00:46.844298  4604 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:00:46.843406  1557 rpcz_store.cc:276] 0709 13:00:45.109278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.804223 (+694945us) service_pool.cc:224] Handling call
0709 13:00:45.804749 (+   526us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:45.805251 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:45.805305 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:45.826864 (+ 21559us) log.cc:844] Serialized 52 byte log entry
0709 13:00:46.450837 (+623973us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:00:46.509790 (+ 58953us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.532796 (+ 23006us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.554149 (+ 21353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.554261 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:46.842720 (+288459us) raft_consensus.cc:1685] finished
0709 13:00:46.842745 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:46.842903 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85996,"prepare.queue_time_us":26288,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":1997,"raft.queue_time_us":82331,"raft.run_cpu_time_us":454,"raft.run_wall_time_us":47361,"spinlock_wait_cycles":49378432,"thread_start_us":556809,"threads_started":1,"wal-append.queue_time_us":671204}
I20260709 13:00:46.846202  4531 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:46.841917  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2220) took 1181 ms. Trace:
I20260709 13:00:46.802814  4427 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:00:46.846728  4609 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:00:46.846985  4531 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:46.847738  4610 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.244s	user 0.001s	sys 0.001s
I20260709 13:00:46.847936  4531 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:46.846913  1555 rpcz_store.cc:276] 0709 13:00:45.660813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.965542 (+304729us) service_pool.cc:224] Handling call
0709 13:00:46.015381 (+ 49839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:46.382829 (+367448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.841597 (+458768us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54003377d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:46.841834 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838840576}
I20260709 13:00:46.848426  4531 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:46.848898  4531 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:46.849140  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.849159  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:46.846230  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:46.850980  1555 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:46.804018  4603 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:46.814548  4571 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.847107  2345 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:46.818075  4539 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:46.825572  4506 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:46.864940  4398 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.701s	user 0.005s	sys 0.010s
I20260709 13:00:46.883133  1559 raft_consensus.cc:2410] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 8 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:00:46.857847  4459 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:46.818861  4509 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:46.897221  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 100801us
I20260709 13:00:46.900422  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.873687  4602 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:46.909049  1601 leader_election.cc:400] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11.
I20260709 13:00:46.852588  4424 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:46.909690  1601 leader_election.cc:403] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:46.911039  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:46.912369  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 150991us
I20260709 13:00:46.918215  2318 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.919152  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:46.877264  4430 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:00:46.929739  2318 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.889022  4506 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:46.924713  4089 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.799s	user 0.013s	sys 0.002s
I20260709 13:00:46.935516  4571 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.916772  4622 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11.
I20260709 13:00:46.913249  4624 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:46.944594  4494 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.610s	user 0.013s	sys 0.001s
I20260709 13:00:46.937419  4631 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.907572  4459 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:46.957490  1674 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.543s (SENT) [suppressed 45 similar messages]
I20260709 13:00:46.937505  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:46.928882  4252 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:46.928323  4621 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:46.926728  4419 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:00:46.963364  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 480) took 1844 ms. Trace:
I20260709 13:00:46.964974  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:46.964633  1703 rpcz_store.cc:276] 0709 13:00:45.119392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.688620 (+569228us) service_pool.cc:224] Handling call
0709 13:00:45.689009 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.994100 (+305091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.946498 (+952398us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400222c54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:46.963289 (+ 16791us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631548800}
W20260709 13:00:46.964000  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.947402  4089 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:46.971215  4617 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:00:46.971352  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:46.972213  1705 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:46.972406  4508 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:00:46.973568  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:46.973776  4611 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.261s	user 0.002s	sys 0.000s
W20260709 13:00:46.976378  4619 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:00:46.979066  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 766) took 2166 ms. Trace:
I20260709 13:00:46.979517  2349 rpcz_store.cc:276] 0709 13:00:44.812555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.812929 (+   374us) service_pool.cc:224] Handling call
0709 13:00:44.813372 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:46.193389 (+1380017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.978795 (+785406us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540002b2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:46.979002 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3122873600}
I20260709 13:00:46.962931  4501 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:46.983304  1629 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:46.988198  1556 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:47.006880  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:47.007218  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:47.008047  1556 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:46.993077  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:47.009053  1703 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:46.995105  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:47.011330  1709 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:47.011405  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:47.012549  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:47.015005  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:47.015965  1710 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:47.009945  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:47.022327  1602 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:47.024533  1560 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
W20260709 13:00:47.025339  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 223131us
I20260709 13:00:47.027621  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:00:47.025470  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 175538us
I20260709 13:00:47.038349  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:47.038841  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:47.027216  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:47.031759  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:47.033766  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:47.033766  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:47.034785  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 919) took 1290 ms. Trace:
I20260709 13:00:47.034086  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:47.045756  2353 rpcz_store.cc:276] 0709 13:00:45.744656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.747072 (+  2416us) service_pool.cc:224] Handling call
0709 13:00:45.781865 (+ 34793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:46.418096 (+636231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.034509 (+616413us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b54006310d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:47.034712 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1429681920}
I20260709 13:00:47.046731  1633 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:00:47.051826  4584 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.170s	user 0.008s	sys 0.002s
I20260709 13:00:47.033420  4597 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:47.053539  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 910) took 1813 ms. Trace:
I20260709 13:00:47.053969  1631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:54730
I20260709 13:00:47.054648  1631 raft_consensus.cc:606] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Received request to transfer leadership
I20260709 13:00:47.055719  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:47.017344  4616 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:00:47.057050  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:47.058159  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:47.058506  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:46.957929  4494 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:47.034832  4620 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:00:46.961134  4627 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:47.068101  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:47.100687  1560 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:47.053917  2346 rpcz_store.cc:276] 0709 13:00:45.239993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.411676 (+171683us) service_pool.cc:224] Handling call
0709 13:00:45.412111 (+   435us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:45.459633 (+ 47522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:45.459686 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:45.460539 (+   853us) log.cc:844] Serialized 52 byte log entry
0709 13:00:45.691991 (+231452us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:45.692057 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.692162 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.050919 (+1358757us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:47.053122 (+  2203us) raft_consensus.cc:1685] finished
0709 13:00:47.053149 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.053431 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29904,"raft.queue_time_us":24110,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":25560,"spinlock_wait_cycles":102452096,"thread_start_us":196402,"threads_started":1,"wal-append.queue_time_us":386341}
I20260709 13:00:47.073078  1709 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 3: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:47.053990  4501 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:47.110123  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:47.111476  1633 raft_consensus.cc:2410] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:47.005381  4631 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:00:47.060904  4597 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:46.983791  4625 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.111s
I20260709 13:00:47.077383  4459 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:47.078279  4605 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:47.081331  4595 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.656s	user 0.002s	sys 0.000s
I20260709 13:00:46.960878  4629 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 13:00:47.069831  4624 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.145298  1636 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:00:47.032613  4615 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:00:47.090121  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:47.061681  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:47.096720  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:47.096776  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:47.099413  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.116699  4501 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:47.117419  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:47.157644  1709 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:47.158668  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:47.033275  4571 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:47.122280  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:47.030714  4632 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:47.091959  4313 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.890s	user 0.012s	sys 0.004s
I20260709 13:00:47.161006  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:47.161993  1560 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:47.108853  1705 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:00:47.162081  1634 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:47.133384  4627 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:47.145043  4457 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.753s	user 0.015s	sys 0.000s
I20260709 13:00:47.110291  4622 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:47.132598  4550 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.916s	user 0.009s	sys 0.007s
W20260709 13:00:47.137831  4418 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.973s	user 0.005s	sys 0.010s
I20260709 13:00:47.144120  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:47.146780  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2217) took 1680 ms. Trace:
I20260709 13:00:47.032393  4398 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 13:00:47.096371  4608 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:47.151202  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 111666us
I20260709 13:00:47.151371  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:47.142789  4416 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.001s
I20260709 13:00:47.062616  4606 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:47.138145  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:47.141388  4635 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.165719  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:47.166944  1703 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:47.143631  4506 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:47.143893  4424 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:47.169870  1558 rpcz_store.cc:276] 0709 13:00:45.503883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.954784 (+450901us) service_pool.cc:224] Handling call
0709 13:00:45.955072 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:46.109296 (+154224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.109328 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:46.109372 (+    44us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:46.109408 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.146549 (+1037141us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54003c0fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:47.146592 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.146715 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6605,"raft.run_cpu_time_us":43,"raft.run_wall_time_us":43,"spinlock_wait_cycles":353771264}
I20260709 13:00:47.112998  4419 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 13:00:47.171725  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 214276us
W20260709 13:00:47.171434  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 160087us
I20260709 13:00:47.177151  1633 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:47.179535  1628 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:00:47.200599  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:47.165433  4313 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:47.180713  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:47.162899  4501 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:47.186040  1627 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:47.186774  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 164479us
I20260709 13:00:47.194716  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:47.210554  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:47.167886  4509 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.220907  1633 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:47.182730  4642 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:47.215217  1705 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:47.150923  4641 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:00:47.212172  1602 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:47.157900  4624 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:00:47.232491  1552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2219) took 1727 ms (client timeout 2171 ms). Trace:
I20260709 13:00:47.178864  4550 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:47.256435  1552 rpcz_store.cc:269] 0709 13:00:45.504740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.535728 (+ 30988us) service_pool.cc:224] Handling call
0709 13:00:47.232107 (+1696379us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54002526d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:47.232387 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1564879744}
I20260709 13:00:47.241676  4644 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:47.246074  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2308) took 1811 ms. Trace:
I20260709 13:00:47.249209  1702 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 6. Current term is 7. Ops: [6.6-6.7]
I20260709 13:00:47.203946  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:47.260694  4644 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.167819  4457 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.252753  1602 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:47.207299  4571 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:47.216997  1558 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:00:47.249058  4310 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.646s	user 0.015s	sys 0.000s
W20260709 13:00:47.272403  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 68587.8us
I20260709 13:00:47.282275  1558 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:47.190018  4416 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.288940  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:47.277572  1705 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:47.295478  1706 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:47.214393  4631 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:00:47.258594  1635 rpcz_store.cc:276] 0709 13:00:45.434245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.647277 (+213032us) service_pool.cc:224] Handling call
0709 13:00:45.647630 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.753582 (+105952us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:45.761167 (+  7585us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.183944 (+422777us) log.cc:844] Serialized 14 byte log entry
0709 13:00:46.184407 (+   463us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:46.184477 (+    70us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:46.184568 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.333520 (+148952us) spinlock_profiling.cc:243] Waited 9.72 ms on lock 0x7b180039d240. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:00:47.080679 (+747159us) spinlock_profiling.cc:243] Waited 5.05 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:47.146191 (+ 65512us) spinlock_profiling.cc:243] Waited 4.98 ms on lock 0x7b180039d240. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419
0709 13:00:47.245766 (+ 99575us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54002da5d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:47.245829 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.245998 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24424,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":102,"raft.queue_time_us":1103,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":10287,"spinlock_wait_cycles":288381568,"thread_start_us":421590,"threads_started":1,"wal-append.queue_time_us":663628}
I20260709 13:00:47.292086  4500 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:47.311532  1709 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:47.261288  4644 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:47.303685  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:47.304971  4638 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:47.306114  4416 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:47.308353  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.309540  1705 pending_rounds.cc:85] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 6
I20260709 13:00:47.254957  4494 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.261757  4449 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:47.312659  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 128681us
I20260709 13:00:47.288136  3963 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:47.312527  2346 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:47.313494  1710 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:47.202421  4635 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.287606  4522 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:47.315495  1705 pending_rounds.cc:107] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:00:47.335276  1705 raft_consensus.cc:2889] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:47.332739  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 58558.5us
I20260709 13:00:47.329844  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:47.345811  1710 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.346825  2317 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:47.351409  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:47.355214  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:47.334863  4522 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:47.311614  4500 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:47.371464  4522 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:47.372704  4635 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.314370  4638 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.372359  4500 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.358222  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:47.376297  1710 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:47.320123  4641 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:47.337551  4596 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.822s	user 0.012s	sys 0.005s
I20260709 13:00:47.395409  1633 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:47.391327  4416 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.392618  1635 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:47.390523  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:47.301820  4310 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:47.385555  4572 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.580s	user 0.007s	sys 0.007s
I20260709 13:00:47.296808  4571 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:47.432245  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 225381us
I20260709 13:00:47.342415  4449 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:47.436075  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:47.386348  4638 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.385299  4631 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:47.393976  4605 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.393760  4542 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:47.226444  4597 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.428967  4500 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:47.259054  4642 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:47.451692  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61306.5us
I20260709 13:00:47.430241  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:47.431476  4608 consensus_queue.cc:1175] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba0548b7d2a94e8598475a46c279fe0e" responder_term: 3 status { last_received { term: 3 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 3 index: 5 } }
I20260709 13:00:47.388861  4494 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:00:47.429730  4596 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.463666  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:47.439195  4651 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.301332  4627 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:47.465096  4433 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.945s	user 0.016s	sys 0.002s
I20260709 13:00:47.462795  1633 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:47.531304  4650 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:47.531920  4457 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:47.437216  4416 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.533943  4642 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:47.428571  4641 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.537569  4598 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.537919  4646 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:47.549091  2353 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:47.466135  3963 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:47.564239  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 100720us
I20260709 13:00:47.567070  4524 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.462102  4572 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.545683  4542 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:47.545650  4494 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:47.547184  4606 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.580683  4416 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.480705  4605 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.606173  1707 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:47.607165  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:47.606026  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1687) took 2526 ms (client timeout 1570 ms). Trace:
W20260709 13:00:47.610360  1708 rpcz_store.cc:269] 0709 13:00:45.079497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.090693 (+ 11196us) service_pool.cc:224] Handling call
0709 13:00:47.605701 (+2515008us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b5400224554. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:47.605912 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1821240192}
I20260709 13:00:47.611994  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:47.613003  1708 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:47.550036  4373 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:00:47.608146  1636 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:47.619660  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:47.620791  1708 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:47.583350  4550 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.624296  4658 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.594063  4494 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:47.629163  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:47.581892  4449 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.615382  4624 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.575309  3963 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.641618  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:47.643435  1526 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:47.646507  1704 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:47.647775  1704 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:47.648186  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:47.627388  4571 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.649468  4650 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.551702  4651 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.569722  4646 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.570744  4598 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.632737  4416 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:47.656876  4653 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:47.649724  4642 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:47.668274  4524 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:47.668929  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.669098  4524 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.673343  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:47.673642  4509 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:47.658393  4531 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.808s	user 0.013s	sys 0.002s
I20260709 13:00:47.674880  4531 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.597219  4542 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.639763  4658 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.659623  4596 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.678604  4542 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.678617  4658 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:47.680228  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:47.681241  1635 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
W20260709 13:00:47.681499  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.660465  3963 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:47.660815  4651 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:47.682734  1525 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:47.660404  4494 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.684029  4494 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:47.674212  4509 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.686740  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:47.687417  4509 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:47.674201  1627 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:00:47.688349  4509 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:47.652226  4373 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:47.689009  4509 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:47.689541  4373 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:47.689978  4509 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.672312  4524 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.690659  4373 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.691872  4531 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:47.702033  4653 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:47.703014  4653 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:47.609494  4606 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.663117  4416 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.658435  4599 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:47.730871  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 337972us
I20260709 13:00:47.729341  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:47.716042  4504 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:47.721596  4623 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.851s	user 0.015s	sys 0.000s
W20260709 13:00:47.724329  4656 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:00:47.753918  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:47.754987  2352 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:47.763579  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:47.738279  4416 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:47.774123  2347 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEARNER]: Advancing to term 6
I20260709 13:00:47.739357  4310 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.740644  4504 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.738449  4599 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:47.651460  4650 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.783768  4663 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:47.771052  4623 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:47.781311  4504 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:47.778929  4522 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.800032  4509 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.800864  4522 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:47.787748  4572 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:47.794657  4657 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:00:47.746714  4501 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.811378  4501 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.816155  4671 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.800047  4598 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.805751  4667 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:47.803296  4602 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:47.815907  4640 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:00:47.796936  4638 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.813670  4605 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:47.893393  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.024596  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:47.910622  1630 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:00:47.914688  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1724) took 2863 ms. Trace:
W20260709 13:00:47.916704  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.919695  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 552) took 1310 ms. Trace:
I20260709 13:00:47.927239  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2223) took 2299 ms. Trace:
I20260709 13:00:47.785553  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:47.929442  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 405) took 1296 ms. Trace:
I20260709 13:00:47.944425  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:47.802838  4509 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.840868  4638 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:47.830930  4641 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.075721  4598 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:48.076620  4598 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.077122  4598 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:47.841424  4667 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:47.837126  4671 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.005141  4416 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:47.834734  4602 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.840600  4606 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:48.027658  1630 pending_rounds.cc:85] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 2
I20260709 13:00:48.014086  4629 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:46.535874   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:48.031029  1559 rpcz_store.cc:276] 0709 13:00:46.632479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.678179 (+ 45700us) service_pool.cc:224] Handling call
0709 13:00:47.928757 (+1250578us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400230d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:47.929126 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":447203712}
I20260709 13:00:48.028842  1629 rpcz_store.cc:276] 0709 13:00:46.608829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.609022 (+   193us) service_pool.cc:224] Handling call
0709 13:00:46.609444 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:46.983913 (+374469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.918834 (+934921us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b54001b7454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:47.919405 (+   571us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":858820480}
I20260709 13:00:48.029275  1707 rpcz_store.cc:276] 0709 13:00:45.627565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.939645 (+312080us) service_pool.cc:224] Handling call
0709 13:00:47.605880 (+1666235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.607060 (+  1180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.926560 (+319500us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b5400224550. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:47.926945 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3830683008}
I20260709 13:00:47.853061  4650 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:48.101209  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.103983  4649 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.795s	user 0.003s	sys 0.001s
I20260709 13:00:48.104740  4416 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:48.096019  4629 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:48.105530  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.082080  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 122508us
I20260709 13:00:48.085948  2345 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:47.831820  4599 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.108028  4629 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.094281  1630 pending_rounds.cc:107] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:00:48.111812  1630 raft_consensus.cc:2889] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:48.112210  4629 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:48.111325  4599 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.122681  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.123011  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.123051  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.123947  1627 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:48.124497  1634 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:48.125082  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.125130  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:48.028055  1561 rpcz_store.cc:276] 0709 13:00:45.051550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.632106 (+580556us) service_pool.cc:224] Handling call
0709 13:00:45.632617 (+   511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.399533 (+766916us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.399582 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:46.400471 (+   889us) log.cc:844] Serialized 26 byte log entry
0709 13:00:46.545224 (+144753us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:46.545263 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.912229 (+1366966us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54003048d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:47.912353 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.912457 (+   104us) raft_consensus.cc:1685] finished
0709 13:00:47.912473 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.914305 (+  1832us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10396,"raft.queue_time_us":33213,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":19428,"spinlock_wait_cycles":469162880,"thread_start_us":674622,"threads_started":2,"wal-append.queue_time_us":244799}
I20260709 13:00:48.129252  4606 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:48.124586  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:48.132988  4606 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:48.137804  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.138590  1635 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.136325  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:48.157706  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.035s	sys 0.050s
W20260709 13:00:48.159759  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.036s	sys 0.050s
I20260709 13:00:48.142400  1634 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:48.156951  4667 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:48.187750  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.163040  4647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.009s	sys 0.006s
I20260709 13:00:48.234640  1553 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:48.284181  4612 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:48.305985  4612 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:48.283970  4667 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:00:48.208853  2347 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:00:48.202231  4647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.011s	sys 0.006s
W20260709 13:00:48.227203  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.314637  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.238817  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 468913us
W20260709 13:00:48.268994  4644 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.942s	user 0.008s	sys 0.011s
I20260709 13:00:48.287628  4661 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.309902  4667 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:48.328634  4667 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:48.329429  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 492) took 1288 ms. Trace:
I20260709 13:00:48.306587  4493 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:48.330037  1710 rpcz_store.cc:276] 0709 13:00:47.030684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.387083 (+356399us) service_pool.cc:224] Handling call
0709 13:00:47.387505 (+   422us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:47.429370 (+ 41865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.429419 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:47.448022 (+ 18603us) log.cc:844] Serialized 52 byte log entry
0709 13:00:47.713231 (+265209us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:47.730562 (+ 17331us) log.cc:844] Serialized 14 byte log entry
0709 13:00:48.309382 (+578820us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b18003cb620. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:00:48.314640 (+  5258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.314812 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.316665 (+  1853us) raft_consensus.cc:1685] finished
0709 13:00:48.316846 (+   181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.318600 (+  1754us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37865,"prepare.queue_time_us":16195,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":70189,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":6731,"spinlock_wait_cycles":8895104,"thread_start_us":235896,"threads_started":1,"wal-append.queue_time_us":299288}
I20260709 13:00:48.332893  4398 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.333655  4398 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:48.208024  4671 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:48.327970  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.327826  2345 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:48.339712  4521 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.340758  4624 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:48.320606  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 868657us
W20260709 13:00:48.208226  4452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.025s	sys 0.057s
I20260709 13:00:48.338779  4432 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:48.355020  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.341370  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:48.359580  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.360716  4605 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.340759  4521 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:48.349628  4398 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:48.361743  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 433388us
I20260709 13:00:48.363885  4521 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:48.364607  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:48.365819  1556 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:48.235435  4598 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:48.367506  4605 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.337061  4606 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:48.357366  4452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.028s	sys 0.061s
W20260709 13:00:48.373210  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 170888us
I20260709 13:00:48.346509  4089 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.308991  4612 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.384068  1634 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:48.350026  4644 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.356720  4432 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:48.381690  4598 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:48.395501  4496 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:48.406241  2348 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:48.406044  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:48.402642  4494 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.718s	user 0.000s	sys 0.027s
I20260709 13:00:48.402765  4496 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.408138  4496 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.408342  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:48.404479  4612 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:48.429291  4606 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.430243  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.413640  4605 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:48.443007  4691 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:00:48.448064  1632 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:00:48.432593  1675 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:48.450148  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 120837us
I20260709 13:00:48.450400  4521 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:48.451934  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46194
I20260709 13:00:48.452926  1710 raft_consensus.cc:606] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Received request to transfer leadership
I20260709 13:00:48.453341  1710 raft_consensus.cc:614] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:48.396395  4661 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.395792  4624 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.454846  4311 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:48.456512  4661 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:48.457788  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.458784  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.460160  1629 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:48.459838  4521 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:48.461298  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.461397  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:48.463040  1634 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:48.457190  4624 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.467483  4549 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:48.467767  4418 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:48.468127  4602 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:48.468955  4680 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.318s	user 0.005s	sys 0.000s
I20260709 13:00:48.469468  4327 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:48.413517  4667 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.485152  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:48.479061  4646 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:48.487353  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 79231.8us
I20260709 13:00:48.478413  4418 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:48.482197  4327 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:00:48.499485  4653 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.735s	user 0.010s	sys 0.014s
I20260709 13:00:48.499637  4671 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:48.493232  1560 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:48.477188  4606 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:48.485324  4598 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:48.502722  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:48.500960  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:48.504329  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.487066  4549 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:48.506799  4689 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:00:48.507259  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1701) took 1507 ms. Trace:
I20260709 13:00:48.507735  1709 rpcz_store.cc:276] 0709 13:00:46.999417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.158360 (+158943us) service_pool.cc:224] Handling call
0709 13:00:48.505173 (+1346813us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54002e90d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:48.506980 (+  1807us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349460736}
I20260709 13:00:48.498849  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:48.496447  4445 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.515687  4445 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:48.518050  1554 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:00:48.505460  4327 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.486685  4624 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:00:48.499015  4418 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:48.519424  4445 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:48.524729  4432 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:48.522418  1629 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:00:48.500198  4653 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.527084  1557 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:48.530107  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:48.529317  4539 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.533939  1561 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:48.533946  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.529155  4398 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.536743  4644 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:48.537664  2349 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:48.538964  4549 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:48.550448  4531 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.551013  4531 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:48.551468  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.552913  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.554077  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:48.553776  4539 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.554809  1631 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:48.558313  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.558797  4423 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.559513  1702 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:48.559862  1707 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:48.559592  4423 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:48.561457  4675 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.493s	user 0.001s	sys 0.001s
I20260709 13:00:48.561097  4398 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.562486  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:48.544173  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.545933  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.548197  4327 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:48.548616  2351 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:48.558058  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.553278  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:48.541039  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:48.568138  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:48.568588  1634 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.570537  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2322) took 1471 ms. Trace:
I20260709 13:00:48.570863  1709 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:48.571004  1630 rpcz_store.cc:276] 0709 13:00:47.099463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.118302 (+ 18839us) service_pool.cc:224] Handling call
0709 13:00:47.149759 (+ 31457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.112552 (+962793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.570242 (+457690us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b54001383d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:48.570458 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1746539264}
I20260709 13:00:48.569207  1707 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.572727  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:48.573515  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 71482.5us
I20260709 13:00:48.573952  4605 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:48.580654  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.601704  4252 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.605005  1635 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:48.586655  1702 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:48.617899  1630 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:48.625644  1632 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:48.628924  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:48.569201  4598 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.630359  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.580009  4605 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:48.628741  4432 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:48.580111  4493 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.646126  1707 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
W20260709 13:00:48.629295  4599 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.003s
I20260709 13:00:48.700736  4614 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
W20260709 13:00:48.652627  4696 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:00:48.654001  1560 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.649761  4644 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:48.658046  4645 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.662495  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:48.671101  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:48.670416  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:48.672152  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:48.677069  1635 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:48.677369  4493 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:48.679140  4653 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:48.679734  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.735437  4418 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:48.680863  4606 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:48.684175  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.746131  1705 raft_consensus.cc:2410] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:48.747507  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:48.752662  4667 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:48.609802  4539 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.645387  4605 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:48.786616  4504 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.783s	user 0.017s	sys 0.002s
I20260709 13:00:48.787424  4504 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.686172  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:48.790521  4695 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:00:48.790825  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.692139  2351 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:48.693418  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.802120  1707 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:48.693651  4432 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.694782  4452 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:48.698038  4687 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.604s	user 0.004s	sys 0.001s
I20260709 13:00:48.724404  1634 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:00:48.579180  4624 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
W20260709 13:00:48.721963  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 242969us
W20260709 13:00:48.729362  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 152093us
I20260709 13:00:48.732681  4398 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.680341  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:48.740301  4645 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.757721  4602 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.761750  4423 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.762408  4424 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:48.762547  4598 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.765682  1702 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:48.766431  1709 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:48.770327  4531 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:48.863722  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.865726  4423 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:48.867143  4645 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:48.868422  4398 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.869628  1708 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:48.871630  4627 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:48.872136  4627 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:48.829727  4653 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.774269  4644 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.737946  4493 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:48.684871  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:48.800979  1701 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:48.772753  4496 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.812252  1631 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:48.814416  4504 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.815347  4432 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.816532  4452 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.819276  4418 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:48.728741  4252 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:48.828531  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1703) took 1827 ms (client timeout 2126 ms). Trace:
W20260709 13:00:48.829028  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1735) took 1841 ms (client timeout 2101 ms). Trace:
W20260709 13:00:48.839061  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.903888  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:48.908241  1560 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
W20260709 13:00:48.903481  1556 rpcz_store.cc:269] 0709 13:00:46.988163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.048089 (+ 59926us) service_pool.cc:224] Handling call
0709 13:00:48.828791 (+1780702us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540026b1d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:48.828949 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2785672576}
I20260709 13:00:48.911943  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.899873  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:48.903370  1703 rpcz_store.cc:269] 0709 13:00:47.000495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.096443 (+ 95948us) service_pool.cc:224] Handling call
0709 13:00:48.828076 (+1731633us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b540016ce54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:48.828308 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29569408}
I20260709 13:00:48.882144  4653 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.837847  1634 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:48.845782  4539 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:00:48.861922  4424 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.869167  4398 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.946274  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.773783  1706 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:48.942982  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 284094us
I20260709 13:00:48.770112  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:48.877285  2347 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
W20260709 13:00:48.893383  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 53730.7us
I20260709 13:00:48.898886  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.901266  1674 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:48.938767  1704 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:48.941324  2351 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:48.943859  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.947368  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:48.953227  1560 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:48.957168  4452 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.958501  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:48.959394  1557 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:48.960501  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.961082  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.961925  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.962329  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:48.962013  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.963032  1701 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.963104  1703 raft_consensus.cc:2410] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:48.963371  1634 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:48.949086  4418 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.964156  4418 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:48.964485  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.964623  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.965864  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:48.966002  1634 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:00:48.960464  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:48.884830  4521 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:00:48.972574  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1739) took 1976 ms. Trace:
I20260709 13:00:48.841948  1630 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:00:48.976843  1636 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:48.977842  4504 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.981841  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.982518  1701 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:00:49.000332  1634 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:49.000231  4504 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.000867  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:49.001679  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:49.002226  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:49.002864  1701 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:49.003119  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:48.983049  4598 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.003794  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:48.983661  4597 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:49.004298  1634 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:48.985528  4606 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.005713  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:49.006137  4606 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.004307  4598 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.008749  4606 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.986979  1557 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:49.010416  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.011265  1557 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:49.014794  2349 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:49.015352  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:49.016775  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:49.019047  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.009604  4598 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.989956  1705 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.994233  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:49.006704  1634 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 6 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:00:49.004823  1701 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 6 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:00:48.999311  4432 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.988886  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:49.046579  1703 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:49.014022  4606 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.016629  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:49.107053  2352 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:48.987838  1702 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:49.003177  4504 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:48.975016  1553 rpcz_store.cc:276] 0709 13:00:46.996229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.180359 (+184130us) service_pool.cc:224] Handling call
0709 13:00:48.947179 (+1766820us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002521d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:48.972463 (+ 25284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2361118592}
I20260709 13:00:49.048487  1631 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
W20260709 13:00:49.049767  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.057313  1701 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:49.057868  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.063346  1627 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:00:48.885008  4627 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.072072  2348 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:49.077329  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:49.079702  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:49.057096  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:49.084508  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:49.085970  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 212995us
I20260709 13:00:49.022349  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:49.091002  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.078146  4432 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.106122  1634 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:49.118606  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:49.121737  1702 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:49.123149  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 563) took 1004 ms. Trace:
I20260709 13:00:49.113066  1627 pending_rounds.cc:85] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 3
I20260709 13:00:49.088786  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:49.123744  1627 pending_rounds.cc:107] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:49.127437  1627 raft_consensus.cc:2889] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:49.129642  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.116078  1706 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:49.123667  1632 rpcz_store.cc:276] 0709 13:00:48.131770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.659382 (+527612us) service_pool.cc:224] Handling call
0709 13:00:48.659847 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.660279 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:48.660725 (+   446us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.120588 (+459863us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.122740 (+  2152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:49.122822 (+    82us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:49.122875 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.122953 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.123089 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":905,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":231,"thread_start_us":451230,"threads_started":1}
I20260709 13:00:49.137272  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:49.142633  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:49.174115  1553 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:49.148768  1702 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:49.155578  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:49.156150  4398 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:49.155814  4432 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:49.156255  4427 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:49.156450  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:49.159224  2344 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:49.160480  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2335) took 1048 ms. Trace:
I20260709 13:00:49.161563  4496 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:49.163143  1634 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:00:49.163645  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:49.164451  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:49.164904  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:49.165067  4252 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:49.165268  2353 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 787) took 1689 ms (client timeout 1647 ms). Trace:
I20260709 13:00:49.165724  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:49.165890  4418 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:49.166145  4313 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:00:49.167421  2351 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:49.166890  4606 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:49.163632  2350 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.168172  1556 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:49.169450  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:49.176194  1701 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:49.179595  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.181741  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2338) took 1068 ms. Trace:
I20260709 13:00:49.189600  2348 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:49.190193  1627 rpcz_store.cc:276] 0709 13:00:48.112083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.939915 (+827832us) service_pool.cc:224] Handling call
0709 13:00:49.043514 (+103599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.160149 (+116635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.160393 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.193387  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.196941  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:49.194195  2353 rpcz_store.cc:269] 0709 13:00:47.354554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.354790 (+   236us) service_pool.cc:224] Handling call
0709 13:00:48.913983 (+1559193us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b5400550354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:49.044098 (+130115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277174528}
I20260709 13:00:49.215750  1560 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:00:49.228973  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 59649.1us
I20260709 13:00:49.219112  1636 rpcz_store.cc:276] 0709 13:00:48.113101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.575699 (+462598us) service_pool.cc:224] Handling call
0709 13:00:48.886270 (+310571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.999301 (+113031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.181418 (+182117us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b540010ccd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:49.181650 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713217280}
I20260709 13:00:49.190120  4644 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:49.236928  1702 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:49.240868  1704 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:00:49.241182  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:49.243208  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:49.197048  2345 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:49.252372  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.253705  4653 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:49.254132  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 344128us
I20260709 13:00:49.254639  4639 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.255196  4627 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:49.256049  1628 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:49.258322  4606 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:49.253206  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.270375  1555 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:49.260838  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:49.265616  1706 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:00:49.261991  1561 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:49.266933  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.269176  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 557) took 1155 ms. Trace:
I20260709 13:00:49.274426  1628 rpcz_store.cc:276] 0709 13:00:48.111351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.115777 (+  4426us) service_pool.cc:224] Handling call
0709 13:00:48.940870 (+825093us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b5800135698. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:00:49.266993 (+326123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7282560}
I20260709 13:00:49.278890  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:49.279417  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 49436.5us
I20260709 13:00:49.292018  1556 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:49.300484  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.301393  1555 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:49.269229  4418 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:49.299592  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:49.244004  4398 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:00:49.294204  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.294827  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:49.303805  1553 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:49.275905  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:49.232138  4427 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.279053  4424 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:49.319682  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:49.293063  4423 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:49.320636  2350 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:49.292032  4599 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:49.330799  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:00:49.320240  4605 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.006s
I20260709 13:00:49.309908  4418 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.311919  4327 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:49.345446  1557 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:49.352474  1561 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:00:49.353638  2318 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:49.356516  1555 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:49.281610  4639 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.330714  4423 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:49.381238  2353 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:49.362001  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:49.350526  4327 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:49.381664  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.345281  4597 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:49.385954  4363 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.379252  1556 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:49.393757  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:49.394752  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 493) took 2363 ms. Trace:
W20260709 13:00:49.432287  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:49.452811  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.389511  4602 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:49.460848  1703 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:49.454248  1708 rpcz_store.cc:276] 0709 13:00:47.031018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.621763 (+590745us) service_pool.cc:224] Handling call
0709 13:00:48.869124 (+1247361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.870184 (+  1060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.390094 (+519910us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540016ce50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:49.394659 (+  4565us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2777599744}
I20260709 13:00:49.456926  1707 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:49.350409  4418 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.459201  1702 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
W20260709 13:00:49.396899  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 445858us
I20260709 13:00:49.463217  1704 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:49.381307  4427 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:49.397128  4311 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:49.362555  4432 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:49.454620  4612 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:49.352137  2344 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
W20260709 13:00:49.469672  4690 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.109s	user 0.005s	sys 0.000s
I20260709 13:00:49.508790  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:49.393486  4423 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:49.396240  4327 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:49.355895  4605 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.460783  4602 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:49.378913  4599 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:49.521062  4697 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.134s	user 0.001s	sys 0.001s
I20260709 13:00:49.531435  4427 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.395823  4597 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:49.577181  1630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.057s	sys 0.149s
I20260709 13:00:49.505805  4612 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.600296  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 796) took 1111 ms. Trace:
I20260709 13:00:49.503310  4311 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:49.534595  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:49.592293  1630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.058s	sys 0.149s
I20260709 13:00:49.600960  2347 rpcz_store.cc:276] 0709 13:00:48.488845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.493047 (+  4202us) service_pool.cc:224] Handling call
0709 13:00:49.599925 (+1106878us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b5400478954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:49.600195 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":739378432}
W20260709 13:00:49.619923  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 152979us
I20260709 13:00:49.624881  1704 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:49.583439  4661 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:49.632557  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 473031us
I20260709 13:00:49.635174  1675 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:49.608271  4612 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:49.602079  4597 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:49.504870  4363 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:49.646565  2318 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:49.623844  4432 raft_consensus.cc:2764] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:00:49.665486  1555 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:00:49.667744  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.637732  4424 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:49.675453  1558 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:49.680603  1674 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:49.618301  4427 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:49.681725  4424 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.683405  4427 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:49.684571  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 409) took 1011 ms. Trace:
I20260709 13:00:49.684415  4424 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:49.685333  4424 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:49.687318  4424 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:49.687740  4424 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:49.688871  4424 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.690459  4424 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:49.691255  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44857.6us
I20260709 13:00:49.692512  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:49.693130  1556 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:49.684969  1558 rpcz_store.cc:276] 0709 13:00:48.672986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.673301 (+   315us) service_pool.cc:224] Handling call
0709 13:00:48.673890 (+   589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.678159 (+1004269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.684287 (+  6128us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b54003304d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:49.684500 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164840704}
I20260709 13:00:49.654695  4645 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:49.713516  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.713614  1675 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:49.658317  4496 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:49.716475  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 120464us
W20260709 13:00:49.716089  4704 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.114s	user 0.001s	sys 0.001s
I20260709 13:00:49.728945  1455 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 186) took 10922 ms. Trace:
I20260709 13:00:49.729648  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 408) took 1083 ms. Trace:
W20260709 13:00:49.730566  4705 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:00:49.674505  4661 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:49.731575  4711 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:00:49.732695  4661 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:49.734268  4599 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:49.734867  4599 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:49.735421  4599 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:49.719892  4602 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.736436  4599 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.742244  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:49.742748  4605 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:49.743196  4539 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:49.708493  4715 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:49.743642  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:49.744076  4715 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.744462  4539 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:49.744462  4715 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:49.745051  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:49.730134  1554 rpcz_store.cc:276] 0709 13:00:48.646408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.651889 (+  5481us) service_pool.cc:224] Handling call
0709 13:00:48.652477 (+   588us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:48.653100 (+   623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.653142 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:48.654450 (+  1308us) log.cc:844] Serialized 26 byte log entry
0709 13:00:49.086822 (+432372us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:49.086891 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.087001 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:49.729378 (+642377us) raft_consensus.cc:1685] finished
0709 13:00:49.729399 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.729556 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16751,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":160,"thread_start_us":424776,"threads_started":1,"wal-append.queue_time_us":655315}
W20260709 13:00:49.750560  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 77474.3us
I20260709 13:00:49.751241  4521 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:49.753652  4496 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.729452  1455 rpcz_store.cc:276] 0709 13:00:38.807064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.807340 (+   276us) service_pool.cc:224] Handling call
0709 13:00:49.728672 (+10921332us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:44.762728 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:44.855716 (+ 92988us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:44.855790 (+    74us) write_op.cc:435] Acquired schema lock
0709 13:00:44.855837 (+    47us) tablet.cc:662] Decoding operations
0709 13:00:44.949729 (+ 93892us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:44.949824 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:44.949853 (+    29us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:00:44.951307 (+  1454us) tablet.cc:701] Row locks acquired
0709 13:00:44.951361 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:00:44.951670 (+   309us) write_op.cc:270] Start()
0709 13:00:44.951849 (+   179us) write_op.cc:276] Timestamp: P: 1783602044951600 usec, L: 0
0709 13:00:44.951872 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:00:44.952720 (+   848us) log.cc:844] Serialized 6846 byte log entry
0709 13:00:45.399340 (+446620us) op_driver.cc:464] REPLICATION: finished
0709 13:00:46.022018 (+622678us) write_op.cc:301] APPLY: starting
0709 13:00:46.022215 (+   197us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:46.022681 (+   466us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:46.022862 (+   181us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:46.111836 (+ 88974us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:46.112266 (+   430us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:00:46.112315 (+    49us) write_op.cc:312] APPLY: finished
0709 13:00:46.112783 (+   468us) log.cc:844] Serialized 119 byte log entry
0709 13:00:46.113836 (+  1053us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:46.114029 (+   193us) write_op.cc:454] Released schema lock
0709 13:00:46.114103 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:46.116775 (+  2672us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:46.116809 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1657,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":559502,"apply.run_cpu_time_us":5826,"apply.run_wall_time_us":132136,"num_ops":17,"prepare.queue_time_us":839793,"prepare.run_cpu_time_us":37489,"prepare.run_wall_time_us":379547,"raft.queue_time_us":197241,"raft.run_cpu_time_us":33929,"raft.run_wall_time_us":397819,"replication_time_us":447287,"thread_start_us":1079578,"threads_started":4,"wal-append.queue_time_us":215402,"wal-append.run_cpu_time_us":53985,"wal-append.run_wall_time_us":1941952}]]}
I20260709 13:00:49.802938  4496 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:49.805600  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 60568.9us
W20260709 13:00:49.806686  1705 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.754s	user 0.013s	sys 0.000s
I20260709 13:00:49.811484  1705 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:00:49.812325  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 505) took 1053 ms. Trace:
I20260709 13:00:49.812785  1705 rpcz_store.cc:276] 0709 13:00:48.758928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.769785 (+ 10857us) service_pool.cc:224] Handling call
0709 13:00:49.812229 (+1042444us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:49.857710  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.858572  1526 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:49.818311  4711 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.085s	user 0.003s	sys 0.000s
I20260709 13:00:49.864573  1675 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:49.816480  4598 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:49.879992  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 68768.6us
I20260709 13:00:49.881047  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:49.696380  4311 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.857338  4661 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.862648  4521 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:49.807353  4597 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:49.900955  1675 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:49.890192  4539 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:49.927829  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:49.910470  4496 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:49.939416  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55646.4us
I20260709 13:00:49.914150  4598 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:49.923825  4504 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:49.931344  4539 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:49.950672  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 258213us
I20260709 13:00:49.911338  4661 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.938553  4496 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.936957  4432 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:49.951565  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 504) took 1193 ms. Trace:
I20260709 13:00:49.959515  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:49.958264  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.961124  1556 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:49.958035  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.959455  1710 rpcz_store.cc:276] 0709 13:00:48.758448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.791982 (+ 33534us) service_pool.cc:224] Handling call
0709 13:00:48.833790 (+ 41808us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:48.870673 (+ 36883us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.870710 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:48.871770 (+  1060us) log.cc:844] Serialized 52 byte log entry
0709 13:00:49.187460 (+315690us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:49.677647 (+490187us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.950993 (+273346us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.951207 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.951304 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:49.951347 (+    43us) raft_consensus.cc:1685] finished
0709 13:00:49.951360 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.951483 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31719,"prepare.queue_time_us":79984,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":112,"raft.queue_time_us":10812,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":9595,"thread_start_us":1009137,"threads_started":3,"wal-append.queue_time_us":636041}
I20260709 13:00:49.964913  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:49.950517  4539 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:49.953151  4639 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:49.971485  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.971187  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:49.954341  4653 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:49.976125  4598 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:49.977385  4717 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:49.977975  4717 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.978392  4717 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:49.978555  4707 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.593s	user 0.005s	sys 0.001s
I20260709 13:00:49.958317  4311 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:49.980983  4703 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.626s	user 0.001s	sys 0.003s
I20260709 13:00:49.974589  4418 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.982038  4459 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:49.958680  4496 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:49.983531  4653 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.984143  4653 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.987708  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:49.985217  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2242) took 1212 ms. Trace:
I20260709 13:00:49.989079  1559 rpcz_store.cc:276] 0709 13:00:48.772408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.343603 (+571195us) service_pool.cc:224] Handling call
0709 13:00:49.359716 (+ 16113us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:49.379484 (+ 19768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:49.379535 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:49.380364 (+   829us) log.cc:844] Serialized 26 byte log entry
0709 13:00:49.814274 (+433910us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:49.814333 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.814442 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:49.984929 (+170487us) raft_consensus.cc:1685] finished
0709 13:00:49.984952 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.985150 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":433281,"threads_started":1,"wal-append.queue_time_us":564298}
I20260709 13:00:49.991314  4728 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:49.992046  4728 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.991346  4644 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:49.994356  4644 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:49.963081  4363 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:49.997602  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 317053us
I20260709 13:00:49.995536  4644 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.007323  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1747) took 1167 ms. Trace:
I20260709 13:00:49.985203  4715 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.007840  1561 rpcz_store.cc:276] 0709 13:00:48.840089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.364027 (+523938us) service_pool.cc:224] Handling call
0709 13:00:49.364581 (+   554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:49.365170 (+   589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:49.365324 (+   154us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:49.451492 (+ 86168us) log.cc:844] Serialized 26 byte log entry
0709 13:00:49.878772 (+427280us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:49.878827 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.878908 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.007025 (+128117us) raft_consensus.cc:1685] finished
0709 13:00:50.007055 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.007250 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25253,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":60510,"thread_start_us":347873,"threads_started":1,"wal-append.queue_time_us":457289}
W20260709 13:00:50.010330  4721 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:00:50.011446  4728 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:50.009967  4715 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:49.971405  4605 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:50.025195  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.037086  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:49.963732  4313 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:00:50.048828  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:50.049767  1706 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:50.034963  4605 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.062441  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:50.034137  4496 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:49.986500  4423 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:50.065374  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 77738.8us
I20260709 13:00:50.035305  4363 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:00:50.073468  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:50.076192  1561 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.082453  4661 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:50.083745  4714 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:00:50.086148  4313 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:50.086745  4313 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.087184  4313 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:50.088089  4725 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:50.088585  4725 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:50.089769  4725 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.091354  4725 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:50.094275  4423 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:50.095194  1552 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:50.045974  4727 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:50.096170  4727 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:00:50.096386  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 59463.1us
I20260709 13:00:50.097522  4727 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.098253  4313 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.002056  4639 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:00:50.082862  4496 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:50.074299  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:50.065171  4605 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.042889  4717 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.116230  4736 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 13:00:50.117015  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 80167.2us
I20260709 13:00:50.118983  4311 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:50.114071  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 146777us
I20260709 13:00:50.039943  4521 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.107636  4597 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:50.123092  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:50.125193  4598 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:50.125663  4598 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:50.126109  4598 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.100247  4730 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:50.113695  4644 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:50.135957  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:50.104414  4424 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:00:50.115252  4732 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:50.152767  4732 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:50.124449  4661 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.153869  4732 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.154258  4661 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.155055  4732 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.156854  4661 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:50.137115  1561 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:50.133164  4311 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.139305  4729 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:50.140421  4327 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.529s	user 0.012s	sys 0.000s
I20260709 13:00:50.141281  4313 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:50.143740  4627 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.867s	user 0.008s	sys 0.004s
I20260709 13:00:50.146399  4612 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.147297  4738 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:00:50.149803  4597 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:50.133857  4496 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:50.135366  4423 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:00:50.159261  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:50.166523  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:50.168445  4734 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:50.168862  4644 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:50.185681  4612 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:50.176829  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 67128.5us
I20260709 13:00:50.185926  2353 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:50.145099  4539 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.178853  4496 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.204437  4743 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:50.204766  4496 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:50.204964  4743 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:50.206085  4743 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.207401  4743 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.208719  4598 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.202687  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:50.209782  2351 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:50.179394  4730 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.212954  4730 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:50.214848  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 944894us
I20260709 13:00:50.215749  1602 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:50.216092  4539 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:50.196540  4661 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:50.217753  4736 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:50.218176  4629 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:50.185705  4311 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:50.218840  4629 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:50.220140  4629 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.221143  4717 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:50.218034  4661 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:50.198853  4729 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.172845  4327 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:50.202929  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 43795.8us
I20260709 13:00:50.214344  4644 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.222783  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 564) took 1369 ms. Trace:
W20260709 13:00:50.202245  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 139969us
I20260709 13:00:50.178102  4741 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:50.235001  2317 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:50.240057  4727 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:50.240702  4727 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:50.238639  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 129619us
I20260709 13:00:50.241430  4727 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.232182  1630 rpcz_store.cc:276] 0709 13:00:48.853721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.020088 (+166367us) service_pool.cc:224] Handling call
0709 13:00:49.020472 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.020766 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:49.021802 (+  1036us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.222397 (+1200595us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.222458 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:50.222512 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.222585 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.222705 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":611911,"prepare.queue_time_us":238,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":612839,"thread_start_us":588196,"threads_started":1,"wal-append.queue_time_us":798797}
I20260709 13:00:50.235476  1525 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:50.240494  2317 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:50.253718  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.267891  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:50.267812  1675 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:50.216128  4597 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:50.276816  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 972460us
I20260709 13:00:50.232649  4624 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:50.278021  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:50.286448  2350 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:50.239244  4741 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.272660  4311 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.293370  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:50.299005  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:50.311508  2351 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:50.312219  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:50.280349  4313 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:50.302495  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:50.313431  1556 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:50.302603  1561 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:50.313449  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:50.315330  2350 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:50.322474  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.323546  1631 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 3.
I20260709 13:00:50.326025  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.327811  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:50.329139  1635 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:50.293244  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:50.331931  2345 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:50.332656  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:50.316287  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.331799  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.334290  1630 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:50.334961  1632 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
W20260709 13:00:50.335942  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.339927  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:50.341856  4747 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:00:50.342639  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.344074  1634 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:00:50.344234  4747 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:50.344914  4747 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:50.279614  4597 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.349442  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.353641  1627 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:50.345446  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.269601  4539 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:00:50.350919  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2129) took 1309 ms. Trace:
I20260709 13:00:50.325193  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.285549  4624 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:50.395604  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.290696  4639 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.403237  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:50.404219  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.394443  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.410009  1629 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:50.293551  4741 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:50.299039  4739 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.350159  4531 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:50.317828  4627 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.349983  4613 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.412734  1628 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:50.417968  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:50.418499  1636 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:50.385763  1633 rpcz_store.cc:276] 0709 13:00:48.957840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.088986 (+131146us) service_pool.cc:224] Handling call
0709 13:00:49.108178 (+ 19192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.141559 (+ 33381us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:49.629812 (+488253us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.237517 (+607705us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.266721 (+ 29204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.266809 (+    88us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:50.266890 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.266997 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.267173 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":300091,"prepare.queue_time_us":553413,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":93,"raft.queue_time_us":371092,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":10975,"spinlock_wait_cycles":69166848,"thread_start_us":737638,"threads_started":2,"wal-append.queue_time_us":342265}
I20260709 13:00:50.317559  4452 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.397279  4313 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:50.423959  4625 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.440667  4625 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.438773  4739 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.445926  4739 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:50.446202  4598 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:50.446722  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.434079  4613 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.447031  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.447669  1636 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:50.447825  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.448688  1628 raft_consensus.cc:2410] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:50.449061  4613 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:50.451939  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:50.421782  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.458266  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:50.419199  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 79548.2us
I20260709 13:00:50.401705  4624 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.434901  1630 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:50.435802  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.448607  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.447412  1633 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 3.
I20260709 13:00:50.443167  4625 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:50.385763  4597 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:50.459024  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:50.419143  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 183780us
I20260709 13:00:50.457574  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:50.488799  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:50.468120  1636 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:50.464857  1635 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:50.489558  1710 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:50.490257  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:50.491161  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:50.492053  1629 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.492575  1631 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:00:50.408444  4605 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.489293  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:50.496397  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:50.497900  2350 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 4: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:50.498481  4313 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.499282  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:50.499610  4605 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:50.500067  1560 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:50.501497  1635 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:50.501883  1635 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:50.502136  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:50.503077  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:50.503634  1507 auto_rebalancer.cc:893] Removing replace marker failed: Replica a2ba7fc60f9641cdac25f1e016ef91d8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 13:00:50.504148  1507 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:00:50.504532  1507 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:00:50.460424  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:50.406971  4661 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.506506  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:50.498932  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:50.507761  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:50.508405  2350 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:50.461998  4739 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:50.414820  4639 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.502732  1635 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.497933  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:50.502749  1631 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:50.508999  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:50.514209  1710 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:50.523820  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:50.526185  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:50.527734  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:50.529697  4749 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:50.530332  4749 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:50.530812  4749 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:50.534947  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:50.509689  4739 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.541749  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:50.542727  1709 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:50.545910  4742 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:50.546706  4739 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:50.509107  4729 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:50.548063  1635 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.549355  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:50.505146  4727 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.551649  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:50.552454  4625 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:50.511349  4730 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:50.560355  1631 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:00:50.553064  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:50.495718  4728 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.463371  4627 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.546597  4742 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:50.517547  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:50.573933  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 56519.2us
I20260709 13:00:50.575084  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:50.576395  4729 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:50.555156  4539 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:50.570758  1636 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:00:50.577864  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 312838us
I20260709 13:00:50.578912  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.456573  4452 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.581688  2345 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:50.582589  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.582899  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:50.583570  1636 raft_consensus.cc:2410] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:50.584019  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:50.584482  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:50.585235  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:50.585530  4725 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:50.586011  1704 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:50.577937  4539 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:50.587066  2317 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:50.588193  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.553026  4625 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:50.593531  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.588073  1634 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:50.597707  1635 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:00:50.567236  4598 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:00:50.564821  4730 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.561501  4613 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:50.624742  1632 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:50.565749  4605 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:50.632849  4732 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.634163  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:50.642484  1707 raft_consensus.cc:2410] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:00:50.587234  4539 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.598641  1635 pending_rounds.cc:85] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 3
W20260709 13:00:50.598713  4653 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.614s	user 0.013s	sys 0.001s
I20260709 13:00:50.642508  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:50.651961  1702 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:50.588780  4629 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:50.650249  1635 pending_rounds.cc:107] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:50.650007  1708 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:00:50.653877  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:50.655141  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 127513us
W20260709 13:00:50.642643  4722 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:00:50.639775  4730 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:50.656153  1635 raft_consensus.cc:2889] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:50.655490  1629 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:50.661273  4625 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.665800  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:50.671697  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:50.672837  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.672636  1635 raft_consensus.cc:2376] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8. Re-sending same reply.
I20260709 13:00:50.649433  4725 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:50.536329  4747 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.682834  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:50.663044  4629 raft_consensus.cc:697] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:50.636514  4728 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.683269  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:50.687005  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:50.687842  1702 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.691761  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:50.694231  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:50.651407  4732 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.657011  4653 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.701269  4613 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:50.716697  4531 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:50.720578  4531 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:50.722155  4737 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:00:50.723555  4653 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:50.684579  4742 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.683214  4725 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:50.708119  4743 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.500s	user 0.011s	sys 0.003s
I20260709 13:00:50.737848  1629 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:00:50.744282  1706 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
W20260709 13:00:50.741779  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 175284us
I20260709 13:00:50.746810  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:00:50.748917  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 161920us
I20260709 13:00:50.783397  2317 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:50.750375  1702 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:50.783890  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:50.790045  4639 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:50.796535  4627 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:50.797796  4751 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:00:50.782990  4539 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:50.801461  1627 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:50.782846  4732 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:50.804730  4746 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.288s	user 0.002s	sys 0.000s
I20260709 13:00:50.749681  4756 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 13:00:50.809306  4539 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.831156  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:50.848310  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:50.812178  4452 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:50.821950  4748 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.224s	user 0.002s	sys 0.000s
W20260709 13:00:50.833810  4496 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.628s	user 0.007s	sys 0.005s
W20260709 13:00:50.821376  4752 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 13:00:50.858083  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 107310us
I20260709 13:00:50.859473  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:50.858721  2348 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:00:50.821306  4612 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:50.847955  4311 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.003s
I20260709 13:00:50.822593  4728 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:50.865592  1553 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:50.854525  4539 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:50.872712  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.834074  4767 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:50.878158  4772 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:50.870949  4612 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.878767  4772 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:50.879204  4750 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.276s	user 0.001s	sys 0.001s
I20260709 13:00:50.879107  4612 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.882184  4763 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:50.747723  4613 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:50.887400  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 193251us
W20260709 13:00:50.892131  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:50.893168  4755 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.317s	user 0.003s	sys 0.000s
I20260709 13:00:50.893410  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:50.876828  4311 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:50.895097  2351 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:50.899593  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:50.901001  4732 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:50.902863  4732 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:50.903465  4732 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.822819  4629 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:50.907169  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:50.909653  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:50.910823  1553 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:50.913285  4521 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:50.913975  4521 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:50.915136  4732 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:50.907399  4725 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:50.930071  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 58612.2us
W20260709 13:00:50.934420  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 345497us
I20260709 13:00:50.864420  4496 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.879364  4772 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:50.932374  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:50.958372  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:50.959533  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:50.960996  4775 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:50.961663  4504 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:51.004810  1710 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:00:51.055378  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.000280  1560 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:51.070742  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 70112.2us
W20260709 13:00:51.076728  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 144462us
I20260709 13:00:51.010268  4521 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:51.080036  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:51.083952  4767 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:51.084254  4783 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:00:50.985275  4777 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:51.090265  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:51.093081  4767 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.093629  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:51.093977  2347 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:51.095867  4767 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:51.098001  4778 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:00:51.084574  4732 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:51.099335  4732 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:51.100378  4732 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:50.983194  4775 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.984968  4624 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.102041  4775 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:51.104131  4624 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:51.088053  4625 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:00:51.081048  4725 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.106426  4777 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:51.108125  4504 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:00:51.109305  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.111212  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.086547  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.084652  4452 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:00:51.118993  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 367381us
I20260709 13:00:51.122087  4624 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:51.119015  4452 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.127259  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:51.127314  2317 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:51.128566  2349 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:51.131906  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:51.132663  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:51.133347  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:51.134184  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:51.134464  1630 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:51.134356  1633 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:51.135928  2347 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:51.139647  1634 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:51.127231  4749 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.596s	user 0.013s	sys 0.003s
I20260709 13:00:51.123198  4732 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:51.142349  1629 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:51.127040  4452 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:00:51.143481  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 105171us
I20260709 13:00:51.144826  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:51.145078  1631 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 13:00:51.145872  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 65934us
I20260709 13:00:51.146920  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:51.119765  4725 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.132917  4311 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:51.120433  4504 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:00:51.108569  4777 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:51.151484  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:51.153990  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:51.155947  1633 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:51.122824  4624 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.144510  4767 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:51.148648  4769 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.161044  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:51.196345  1553 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.145092  4749 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:51.153122  4784 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:51.177030  4741 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.687s	user 0.014s	sys 0.000s
I20260709 13:00:51.237063  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:51.246330  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:51.247051  1559 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:51.186411  4625 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:51.249275  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 84601.1us
I20260709 13:00:51.165625  4767 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:51.163513  4311 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:51.241256  4763 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:51.246876  4597 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.712s	user 0.012s	sys 0.006s
I20260709 13:00:51.160488  4612 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.255071  4725 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:51.254782  4597 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.256532  4531 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:51.244925  4496 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:51.161309  4764 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:00:51.256886  4612 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:51.257905  4764 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:51.258270  4764 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:51.266296  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:51.275667  4531 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:51.276480  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:51.277017  4749 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:51.277884  4531 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:51.278467  4531 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:51.279237  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:51.279495  4531 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:51.280718  4785 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:51.284497  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:51.235812  4765 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:00:51.285770  4785 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:51.285621  4763 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.288539  4763 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:51.288130  4493 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:00:51.254067  4730 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.575s	user 0.009s	sys 0.008s
I20260709 13:00:51.292034  4493 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:51.291411  4624 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:51.280030  1704 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:51.311769  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.316574  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:51.316742  4493 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.319231  2349 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:00:51.320741  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:51.252942  4768 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:51.322894  2318 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:51.321254  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:51.327160  4612 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:51.166092  4769 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.327790  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:51.329032  2353 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 944) took 1863 ms (client timeout 2091 ms). Trace:
W20260709 13:00:51.329492  2353 rpcz_store.cc:269] 0709 13:00:49.466085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.466262 (+   177us) service_pool.cc:224] Handling call
0709 13:00:51.328827 (+1862565us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b5400642dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:51.328974 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554722176}
I20260709 13:00:51.348650  1705 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:51.348650  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:00:51.351343  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 191872us
I20260709 13:00:51.248473  4741 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:51.325968  4763 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.362289  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:51.357158  1704 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:51.330566  4624 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.325978  4768 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:51.374230  4785 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:00:51.380738  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:51.378854  4763 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.383903  4763 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:51.372290  4768 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.372179  4728 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:00:51.387575  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:51.388509  4767 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.390048  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:51.321826  4730 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.380771  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:51.392796  4779 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:51.259506  4764 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.394466  4764 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:51.394891  4787 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:00:51.397341  1702 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:51.398595  1702 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:51.372633  4791 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:51.399217  4791 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.399564  4791 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.406338  1553 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:51.422818  1554 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:51.425824  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 130727us
I20260709 13:00:51.426801  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:51.428179  1704 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:51.431033  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:51.431810  1633 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:51.371928  4613 raft_consensus.cc:2764] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:00:51.436369  1705 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:51.384213  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:51.370133  4311 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:51.442080  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 62286.5us
I20260709 13:00:51.454633  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:51.424522  4504 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:51.447969  4725 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.463316  4725 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:51.466351  4795 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:00:51.460338  4432 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:00:51.468521  4769 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:51.474002  4779 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:51.474344  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.474555  4779 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.475016  4779 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:51.476431  2348 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:51.457947  4252 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.445390  4785 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.426075  4612 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:00:51.406620  4775 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.479277  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:51.445793  4624 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:51.486719  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 99228.5us
W20260709 13:00:51.459153  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.497327  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:51.426265  4728 consensus_queue.cc:1059] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 13:00:51.481611  4539 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.001s
I20260709 13:00:51.502175  1705 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:51.457648  2345 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:00:51.517423  1704 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:51.511636  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:51.478739  4741 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:51.495373  1527 connection.cc:256] RPC call timeout handler was delayed by -1.2401754460000234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
W20260709 13:00:51.530022  1527 connection.cc:256] RPC call timeout handler was delayed by -1.2356754460000161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:00:51.530369  1527 connection.cc:256] RPC call timeout handler was delayed by -1.0727754460000369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
W20260709 13:00:51.530630  1527 connection.cc:256] RPC call timeout handler was delayed by -1.0592121920000181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 13:00:51.479677  4785 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:51.538090  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:51.539230  1553 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:51.540405  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:51.500813  4730 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:51.490252  4252 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.221s
W20260709 13:00:51.533735  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 79322.5us
W20260709 13:00:51.726164  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.802668  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 291097us
I20260709 13:00:51.851161  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:51.875262  1630 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
W20260709 13:00:51.910727  1702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.007s	sys 0.015s
W20260709 13:00:51.911515  1702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.008s	sys 0.015s
I20260709 13:00:51.922413  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:51.922526  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:51.924165  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:51.924983  1629 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:51.794657  4732 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.647s	user 0.005s	sys 0.005s
I20260709 13:00:51.928646  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:51.929957  2349 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 3. Current term is 5. Ops: []
W20260709 13:00:51.847596  4452 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.696s	user 0.008s	sys 0.004s
W20260709 13:00:51.642846  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.800489  4801 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 13:00:51.796132  4772 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.792s	user 0.008s	sys 0.003s
I20260709 13:00:51.933183  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:51.582271  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.875131  4767 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.542963  4775 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:51.541164  1553 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:51.948210  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:51.523133  4725 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:51.966183  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.890861  4416 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:52.008397  2318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.003s	sys 0.024s
I20260709 13:00:51.731802  4777 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:52.009001  2318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.003s	sys 0.024s
I20260709 13:00:51.967810  4531 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:51.963445  1675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.689s (SENT) [suppressed 9 similar messages]
W20260709 13:00:52.039806  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 111410us
I20260709 13:00:52.011978  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:52.009526  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:52.042487  1559 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:52.035950  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 589612us
I20260709 13:00:51.796571  4791 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:52.053056  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.057308  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:52.104781  2348 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:51.514727  4539 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:51.501150  4624 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:52.120832  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:51.803994  4779 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:52.005671  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:52.143397  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:52.118726  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 77837.8us
I20260709 13:00:52.118790  1553 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:52.128356  2349 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:51.731288  4794 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:51.767679  4768 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:51.943794  4452 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:52.175014  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2266) took 2847 ms. Trace:
I20260709 13:00:52.117141  4785 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:52.175784  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:52.177013  1703 rpcz_store.cc:276] 0709 13:00:49.308257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.308457 (+   200us) service_pool.cc:224] Handling call
0709 13:00:49.316710 (+  8253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.461504 (+144794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.154642 (+2693138us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54002e77d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:52.154954 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320652160}
W20260709 13:00:52.179610  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 130536us
I20260709 13:00:52.180150  2348 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:51.969066  4772 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:52.036919  4416 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:52.199770  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:52.112418  4777 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:52.228310  1553 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
W20260709 13:00:52.232350  4807 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:00:52.233798  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2284) took 1095 ms. Trace:
I20260709 13:00:52.234364  1702 rpcz_store.cc:276] 0709 13:00:51.138875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.399339 (+260464us) service_pool.cc:224] Handling call
0709 13:00:51.399666 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:51.399912 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:51.399939 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:51.400444 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:00:51.912121 (+511677us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:51.912189 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.912304 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.233438 (+321134us) raft_consensus.cc:1685] finished
0709 13:00:52.233495 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.233709 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":511245,"threads_started":1,"wal-append.queue_time_us":588087}
I20260709 13:00:52.238214  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:52.239413  1553 raft_consensus.cc:2368] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7.
I20260709 13:00:52.195425  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2282) took 1603 ms. Trace:
I20260709 13:00:52.195772  4418 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:52.011844  4767 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:52.213045  4802 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:52.242617  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 307314us
I20260709 13:00:52.253965  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:52.244992  1710 rpcz_store.cc:276] 0709 13:00:50.591982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.385954 (+793972us) service_pool.cc:224] Handling call
0709 13:00:51.386329 (+   375us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:51.429672 (+ 43343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:51.429704 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:51.443036 (+ 13332us) log.cc:844] Serialized 52 byte log entry
0709 13:00:51.906216 (+463180us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:51.935134 (+ 28918us) log.cc:844] Serialized 14 byte log entry
0709 13:00:51.935370 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.935448 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.195133 (+259685us) raft_consensus.cc:1685] finished
0709 13:00:52.195159 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.195352 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29971,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":28721,"raft.run_cpu_time_us":468,"raft.run_wall_time_us":27821,"spinlock_wait_cycles":14840192,"thread_start_us":425952,"threads_started":1,"wal-append.queue_time_us":732100}
W20260709 13:00:52.243919  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 280546us
I20260709 13:00:52.189528  4613 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:00:52.263801  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:52.186224  1629 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:52.268352  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:52.269130  1553 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.271284  1559 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:52.288549  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:52.289290  1559 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:52.292313  4539 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:52.294883  4416 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:52.320533  4416 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:52.295046  4767 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:52.296192  4624 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:52.297080  4791 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:52.298154  4768 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:52.299378  4772 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:52.300348  4779 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:52.301039  4804 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:52.301405  4452 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:52.302275  4602 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:52.303357  4814 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:52.270871  4764 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.871s	user 0.010s	sys 0.005s
I20260709 13:00:52.315346  1527 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:52.316676  4713 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 2.621s	user 0.017s	sys 0.026s
I20260709 13:00:52.319435  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:52.325773  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:52.262632  4802 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:52.327482  1553 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:52.281123  4775 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:00:52.329541  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:52.333796  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 134135us
I20260709 13:00:52.328181  4252 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:52.336943  1559 raft_consensus.cc:2410] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:00:52.337371  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 521) took 2065 ms. Trace:
I20260709 13:00:52.339498  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:52.339082  2352 rpcz_store.cc:276] 0709 13:00:50.271524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.271679 (+   155us) service_pool.cc:224] Handling call
0709 13:00:52.319171 (+2047492us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x7b5800003c98. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:00:52.337274 (+ 18103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10563456}
I20260709 13:00:52.331588  4814 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.340852  1553 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:52.331363  4804 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:52.342558  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:52.341761  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:52.347754  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:52.348032  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 517) took 3978 ms. Trace:
I20260709 13:00:52.335666  4539 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:52.348486  2346 rpcz_store.cc:276] 0709 13:00:48.369937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.370102 (+   165us) service_pool.cc:224] Handling call
0709 13:00:48.394241 (+ 24139us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:48.439064 (+ 44823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.439122 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:48.469870 (+ 30748us) log.cc:844] Serialized 52 byte log entry
0709 13:00:49.012494 (+542624us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b180041cb60. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:00:49.361419 (+348925us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:49.361492 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.361575 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.315640 (+2954065us) spinlock_profiling.cc:243] Waited 4.06 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:52.347624 (+ 31984us) raft_consensus.cc:1685] finished
0709 13:00:52.347649 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.347830 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6273,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":28870,"spinlock_wait_cycles":14197248,"thread_start_us":347751,"threads_started":1,"wal-append.queue_time_us":1142390}
I20260709 13:00:52.350147  4539 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.350632  4539 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:52.353691  4777 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:52.356859  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:52.362003  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:52.362645  4817 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:52.377027  4767 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:52.818526  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 456591us
I20260709 13:00:52.818893  4767 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:52.378428  4802 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:52.363265  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:52.369768  1702 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:52.364648  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.073526  1708 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:53.074815  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:53.075053  2318 leader_election.cc:336] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.997s (SENT)
I20260709 13:00:52.368142  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:52.342777  4814 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:53.081539  1710 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.355296  4764 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:52.361413  4777 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:53.088006  2318 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:52.418051  4804 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.089538  4798 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:53.090093  4798 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:53.090480  4798 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.094456  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.095124  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.095638  1708 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.096403  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.098345  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:52.393266  4819 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:00:53.098882  1556 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:52.578720  4756 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:53.098950  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:53.099772  4819 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:53.099475  4756 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:53.100154  4819 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:53.101954  4756 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:53.103194  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:52.638358  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1752) took 3608 ms. Trace:
I20260709 13:00:53.104311  1552 rpcz_store.cc:276] 0709 13:00:49.029865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.503089 (+473224us) service_pool.cc:224] Handling call
0709 13:00:49.515570 (+ 12481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.095692 (+580122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.638081 (+2542389us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b5400312254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:52.638286 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1332427520}
I20260709 13:00:53.105705  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.106489  1552 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:52.640830  4612 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:53.088094  1525 connection.cc:256] RPC call timeout handler was delayed by -1.364307690999965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19970000000000002s for a response
I20260709 13:00:53.107942  4612 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:53.107396  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.108875  1556 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:53.109041  4612 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.109776  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.110157  4612 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.110512  1556 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:53.110658  4756 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:52.644984  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 510) took 2187 ms (client timeout 1827 ms). Trace:
W20260709 13:00:53.112109  1706 rpcz_store.cc:269] 0709 13:00:50.457803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.457918 (+   115us) service_pool.cc:224] Handling call
0709 13:00:52.644677 (+2186759us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400077e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:52.644850 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655576320}
I20260709 13:00:53.113365  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:53.113963  1706 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.111403  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.114228  4311 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:53.114987  1556 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:53.116133  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:52.689942  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:52.698844  4427 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:53.118438  4427 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:53.118814  4427 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:53.119721  4427 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.121017  4427 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.707085  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.121848  1704 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
W20260709 13:00:52.726672  4790 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.626s	user 0.006s	sys 0.021s
I20260709 13:00:53.127735  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.128652  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 425) took 1596 ms. Trace:
I20260709 13:00:53.128981  1556 rpcz_store.cc:276] 0709 13:00:51.532031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.127448 (+1595417us) service_pool.cc:224] Handling call
0709 13:00:53.128592 (+  1144us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:52.732904  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2342) took 2396 ms (client timeout 2865 ms). Trace:
W20260709 13:00:53.129652  1632 rpcz_store.cc:269] 0709 13:00:50.336888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.349090 (+ 12202us) service_pool.cc:224] Handling call
0709 13:00:52.732612 (+2383522us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54003b83d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:52.732784 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553628032}
I20260709 13:00:53.130012  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:52.748903  2351 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 523) took 2443 ms (client timeout 1661 ms). Trace:
I20260709 13:00:53.130767  1556 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:53.131302  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2279) took 2540 ms. Trace:
I20260709 13:00:53.131426  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 431) took 1058 ms. Trace:
I20260709 13:00:53.131742  1556 rpcz_store.cc:276] 0709 13:00:52.073302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.129775 (+1056473us) service_pool.cc:224] Handling call
0709 13:00:53.131372 (+  1597us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:53.131700  1709 rpcz_store.cc:276] 0709 13:00:50.590961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.826205 (+235244us) service_pool.cc:224] Handling call
0709 13:00:50.826633 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:50.827030 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.827061 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:50.827566 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:00:50.958704 (+131138us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:50.958751 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.968419 (+  9668us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.726204 (+1757785us) spinlock_profiling.cc:243] Waited 8.76 ms on lock 0x7b24001df6d8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda244739c4 00007fda24471acb 00007fda2462cdc9 00007fda2462ca55 00007fda2462c09d 00007fda2725e10e 00007fda2727588c 00007fda27274bb6 00007fda272747d8 00007fda2727e221
0709 13:00:53.130918 (+404714us) raft_consensus.cc:1685] finished
0709 13:00:53.130943 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:53.131240 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2107,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":94,"spinlock_wait_cycles":20145024,"thread_start_us":74155,"threads_started":1,"wal-append.queue_time_us":238173}
I20260709 13:00:52.769567  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:53.134799  1555 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:53.135378  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 428) took 1380 ms. Trace:
I20260709 13:00:53.135692  1555 rpcz_store.cc:276] 0709 13:00:51.755508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.134133 (+1378625us) service_pool.cc:224] Handling call
0709 13:00:53.134451 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:53.135142 (+   691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.135322 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:53.136898  1555 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:52.772770  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.772563  4398 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:52.776005  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2264) took 1941 ms (client timeout 1685 ms). Trace:
I20260709 13:00:53.133062  1556 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:52.776751  4459 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.139369  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 424) took 1607 ms. Trace:
I20260709 13:00:53.139308  4459 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:53.139786  1556 rpcz_store.cc:276] 0709 13:00:51.531865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.132581 (+1600716us) service_pool.cc:224] Handling call
0709 13:00:53.132868 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:53.139108 (+  6240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.139288 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:52.783649  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:52.823535  1527 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:53.142694  1561 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:53.142967  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 319462us
I20260709 13:00:53.145028  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:52.827205  4539 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:52.843936  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 802) took 2613 ms. Trace:
I20260709 13:00:52.889922  4820 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:53.147778  4747 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:00:53.148124  4820 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:52.891203  4730 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.147693  2344 rpcz_store.cc:276] 0709 13:00:50.230123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.231327 (+  1204us) service_pool.cc:224] Handling call
0709 13:00:52.753299 (+2521972us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b5800003c98. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:00:52.753610 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:52.754150 (+   540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:52.754190 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:52.754855 (+   665us) log.cc:844] Serialized 26 byte log entry
0709 13:00:52.812486 (+ 57631us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:52.813439 (+   953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.815149 (+  1710us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.837726 (+ 22577us) raft_consensus.cc:1685] finished
0709 13:00:52.838497 (+   771us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.843632 (+  5135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4644,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":170,"spinlock_wait_cycles":28294528,"thread_start_us":55564,"threads_started":1,"wal-append.queue_time_us":60904}
I20260709 13:00:52.925268  4598 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:53.147403  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43868.5us
I20260709 13:00:53.150293  4598 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:00:53.150822  4820 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:52.944473  1558 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
W20260709 13:00:52.963750  1636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2147) took 2379 ms (client timeout 1997 ms). Trace:
I20260709 13:00:53.133117  1710 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
W20260709 13:00:53.152493  1636 rpcz_store.cc:269] 0709 13:00:50.583867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.584191 (+   324us) service_pool.cc:224] Handling call
0709 13:00:52.963483 (+2379292us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b540010d6d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:52.963661 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":857982336}
W20260709 13:00:52.965987  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.966662  4728 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.153455  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.153803  4728 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:52.976678  1458 catalog_manager.cc:5697] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:53.156363  4728 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:52.970798  4831 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:52.979034  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.157215  4831 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:52.993836  4836 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:53.003650  4727 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:53.005882  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.011516  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 588) took 1160 ms. Trace:
I20260709 13:00:53.016566  4830 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:53.024824  4661 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:53.018453  4597 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:53.026382  4759 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 2.449s	user 0.011s	sys 0.018s
I20260709 13:00:53.162225  1631 rpcz_store.cc:276] 0709 13:00:51.851427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.851546 (+   119us) service_pool.cc:224] Handling call
0709 13:00:51.922943 (+ 71397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:52.331397 (+408454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:52.331449 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:52.341670 (+ 10221us) log.cc:844] Serialized 26 byte log entry
0709 13:00:52.654398 (+312728us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:52.654449 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.010926 (+356477us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540010c7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:53.011000 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:53.011064 (+    64us) raft_consensus.cc:1685] finished
0709 13:00:53.011076 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:53.011436 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35376,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":5034,"spinlock_wait_cycles":576265472,"thread_start_us":162812,"threads_started":2,"wal-append.queue_time_us":290895}
W20260709 13:00:53.036402  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.039637   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:53.107389  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:53.183812  1552 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:53.041622  4496 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.049932  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:53.186094  2350 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:53.050642  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:53.187588  2347 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:53.051853  4493 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.149331  4539 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:53.054452  4252 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:53.055132  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 952) took 1120 ms. Trace:
I20260709 13:00:53.047899  4835 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:53.057463  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2265) took 2222 ms. Trace:
I20260709 13:00:53.058171  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.196782  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:53.194370  2348 rpcz_store.cc:276] 0709 13:00:51.934380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:52.143013 (+208633us) service_pool.cc:224] Handling call
0709 13:00:53.054881 (+911868us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b54007bff54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:53.055054 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73259776}
I20260709 13:00:53.197515  1703 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:53.062616  4327 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:52.371987  4715 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:53.109867  1525 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:53.212105  4715 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:53.212769  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 103017us
I20260709 13:00:53.185418  4764 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.213871  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:53.214057  4764 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:53.214447  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:53.185575  4627 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.215278  1634 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 6
I20260709 13:00:53.215193  4627 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.111404  4756 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:53.216367  4764 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:52.803637  1701 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:53.145265  4398 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:53.145763  4416 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:53.138934  1554 rpcz_store.cc:269] 0709 13:00:50.834722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.847980 (+ 13258us) service_pool.cc:224] Handling call
0709 13:00:52.775696 (+1927716us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b5400330ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:52.775914 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257886464}
I20260709 13:00:53.153178  2317 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:53.234678  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 81605.5us
W20260709 13:00:53.130733  2351 rpcz_store.cc:269] 0709 13:00:50.305669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.313080 (+  7411us) service_pool.cc:224] Handling call
0709 13:00:52.748504 (+2435424us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b54006202d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:52.748730 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1175945984}
I20260709 13:00:52.974187  4805 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.979388  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:52.981165  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2153) took 1851 ms. Trace:
I20260709 13:00:53.157874  1705 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
W20260709 13:00:53.158944  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.159776  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:53.162833  1708 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:53.160742  1710 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:53.164026  1527 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:53.156479  1458 catalog_manager.cc:5697] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 5, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 5 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:53.165604  1558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2247) took 3203 ms (client timeout 2158 ms). Trace:
W20260709 13:00:53.242691  1558 rpcz_store.cc:269] 0709 13:00:49.957588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.957756 (+   168us) service_pool.cc:224] Handling call
0709 13:00:53.160508 (+3202752us) spinlock_profiling.cc:243] Waited 2.97 s on lock 0x7b540038bdd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:53.160627 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6829023360}
I20260709 13:00:53.237562  1628 rpcz_store.cc:276] 0709 13:00:51.130101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.130439 (+   338us) service_pool.cc:224] Handling call
0709 13:00:51.142845 (+ 12406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:51.158900 (+ 16055us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:51.160798 (+  1898us) log.cc:844] Serialized 14 byte log entry
0709 13:00:51.457673 (+296875us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:52.306898 (+849225us) spinlock_profiling.cc:243] Waited 8.38 ms on lock 0x7b180003fac0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a
0709 13:00:52.312916 (+  6018us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b2c00070338. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:00:52.327341 (+ 14425us) log.cc:844] Serialized 26 byte log entry
0709 13:00:52.327579 (+   238us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:52.327636 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.328307 (+   671us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.703422 (+375115us) spinlock_profiling.cc:243] Waited 6.49 ms on lock 0x7b180003fac0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:00:52.980870 (+277448us) raft_consensus.cc:1685] finished
0709 13:00:52.980896 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.981093 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62155,"prepare.queue_time_us":1722,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":71192,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":3699,"spinlock_wait_cycles":40763776,"thread_start_us":246433,"threads_started":1,"wal-append.queue_time_us":301750}
W20260709 13:00:53.243964  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 85510.4us
I20260709 13:00:53.168063  1557 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:53.188465  1632 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:53.189054  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:53.189230  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:53.053848  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 690596us
W20260709 13:00:53.192004  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 113880us
W20260709 13:00:53.246621  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 82691.6us
I20260709 13:00:53.246235  1630 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:00:53.247351  1676 leader_election.cc:336] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.661s (SENT)
I20260709 13:00:53.200403  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:53.248031  1636 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.196604  1560 rpcz_store.cc:276] 0709 13:00:50.835093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.837057 (+  1964us) service_pool.cc:224] Handling call
0709 13:00:50.862009 (+ 24952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.063957 (+201948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.057142 (+1993185us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54000bd454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:53.057382 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317138304}
I20260709 13:00:53.167400  4661 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:53.258106  4734 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:53.243342  4327 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:53.168532  4777 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:53.207737  2318 leader_election.cc:336] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.580s (SENT)
I20260709 13:00:53.196208  4459 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:53.178298  4830 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:53.215658  1634 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.7
I20260709 13:00:52.834991  1559 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:53.149852  4730 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.218282  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.273612  1703 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:00:53.218926  2347 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
W20260709 13:00:53.219430  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.236488  1706 raft_consensus.cc:2410] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:53.246337  1629 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.200338  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:53.276337  1627 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:53.249426  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2248) took 3291 ms (client timeout 2447 ms). Trace:
I20260709 13:00:53.196843  4252 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.250768  1633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:54730
I20260709 13:00:53.241905  4539 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:53.256021  1458 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:00:53.202811  4835 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:53.101081  4819 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.259976  1527 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:53.259209  4767 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:53.156543  4747 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.258678  4734 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.270274  1632 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:53.272069  1634 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Aborting config change with OpId 6.7: Aborted: Op aborted by new leader
I20260709 13:00:53.256278  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:53.277793  1557 rpcz_store.cc:269] 0709 13:00:49.957995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.958146 (+   151us) service_pool.cc:224] Handling call
0709 13:00:53.005730 (+3047584us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b5800002798. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5eee 0000000000375939 00007fda2c6291e0 00007fda2c6f20c4 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:00:53.249340 (+243610us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2815488}
I20260709 13:00:53.195611  4496 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.291621  1527 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:53.264513  2318 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:53.259145  4597 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:53.299275  1630 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:53.318722  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:53.300418  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 148546us
I20260709 13:00:53.319802  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:53.271552  4817 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.306955  1634 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:53.199465  4798 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.213140  4715 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:53.309082  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 101522us
I20260709 13:00:53.328804  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.321245  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.343466  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.273033  4327 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:53.309744  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.345163  1559 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:53.345314  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:53.314661  1702 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:53.307467  1635 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.323962  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.326774  1627 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:53.354699  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:53.167578  4836 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.330622  1458 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:00:53.338587  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.356768  1556 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:53.344414  1706 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:53.311875  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.285246  4252 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.368606  1552 raft_consensus.cc:2410] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:00:53.288750  4539 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.376375  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.377436  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.296221  4504 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:53.315143  4734 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.385772  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.377713  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.407104  1553 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:53.376629  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:53.407835  1701 raft_consensus.cc:2368] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8.
I20260709 13:00:53.410014  2347 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:53.389911  1552 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:00:53.392172  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.412866  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.394346  1458 catalog_manager.cc:5697] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 6, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:00:53.397890  4597 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:53.369994  4605 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:53.335856  4644 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.405180  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.385159  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.391060  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:53.418146  1554 raft_consensus.cc:2368] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5.
I20260709 13:00:53.419463  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:53.420456  1705 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:53.303248  4764 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:00:53.249275  4805 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.418785  4715 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.456245  1635 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:53.369383  4727 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:53.379458  4539 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:53.458360  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.464224  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.375563  4767 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:53.404235  4835 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:53.467722  1629 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:53.506850  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:53.469312  1561 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:53.393203  4627 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:53.342628  4416 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:00:53.481814  1458 catalog_manager.cc:5697] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 5 to 6, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:00:53.485634  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:53.514823  1552 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:53.466641  4605 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.486606  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:53.487877  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:53.483191  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.495754  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.418213  4597 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:53.427431  1490 debug-util.cc:398] Leaking SignalData structure 0x7b0800039cc0 after lost signal to thread 434
W20260709 13:00:53.462692  4834 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:00:53.500648  1701 raft_consensus.cc:2410] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:53.500300  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:53.522434  1635 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:53.523586  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:53.527107  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 207424us
I20260709 13:00:53.521136  1458 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 2 to 4, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:00:53.503078  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 157894us
I20260709 13:00:53.515318  2353 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:53.426195  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:53.514258  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 234926us
I20260709 13:00:53.471719  4756 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:53.518957  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:53.522504  1702 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:53.475464  4763 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 13:00:53.511510  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 219969us
I20260709 13:00:53.525068  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:53.378499  4817 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:53.539461  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:53.693539  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:53.550324  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.693149  1636 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
W20260709 13:00:53.694597  1674 connection.cc:256] RPC call timeout handler was delayed by -1.7330161560000192s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
I20260709 13:00:53.694909  1632 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:53.695176  1674 connection.cc:256] RPC call timeout handler was delayed by -1.5935161560000211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
W20260709 13:00:53.696069  1674 connection.cc:256] RPC call timeout handler was delayed by -1.5683161559999803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
W20260709 13:00:53.696627  1674 connection.cc:256] RPC call timeout handler was delayed by -1.5548161560000153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
I20260709 13:00:53.691704  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:53.692782  2347 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:53.656224  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:00:53.696995  1674 connection.cc:256] RPC call timeout handler was delayed by -1.4810161560000097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260709 13:00:53.623974  4846 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:53.628536  4848 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 13:00:53.742069  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 50468us
I20260709 13:00:53.711361  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:53.743160  1636 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 3.
I20260709 13:00:53.743896  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:53.741235  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:53.713809  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:53.521867  4605 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.520540  4627 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.588776  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:53.652104  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.571185  1458 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 6 to 9, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 9 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:53.554090  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:53.687106  4836 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:53.673110  4837 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:00:53.629406  4849 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:00:53.687981  4427 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.567s	user 0.006s	sys 0.009s
I20260709 13:00:53.558929  4598 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:53.768842  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:53.740439  1553 raft_consensus.cc:2410] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:00:53.769678  1631 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:53.770289  1557 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:53.514120  4767 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:53.747795  1635 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:53.748025  1627 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:00:53.788683  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:53.772254  1458 catalog_manager.cc:5697] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 2 to 5, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 5 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:53.713809  1674 connection.cc:256] RPC call timeout handler was delayed by -1.4684161559999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
I20260709 13:00:53.757052  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:53.715530  1490 debug-util.cc:398] Leaking SignalData structure 0x7b08000a3080 after lost signal to thread 1493
W20260709 13:00:53.791997  1490 debug-util.cc:398] Leaking SignalData structure 0x7b080037cae0 after lost signal to thread 1507
W20260709 13:00:53.793834  1490 debug-util.cc:398] Leaking SignalData structure 0x7b0800380ea0 after lost signal to thread 1585
W20260709 13:00:53.795527  1490 debug-util.cc:398] Leaking SignalData structure 0x7b08002fdae0 after lost signal to thread 1660
W20260709 13:00:53.795873  1490 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9340 after lost signal to thread 1661
I20260709 13:00:53.529901  4597 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:53.764053  1629 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:53.764595  1708 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.507947  4727 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:53.584486  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 264577us
I20260709 13:00:53.785876  1633 raft_consensus.cc:606] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Received request to transfer leadership
I20260709 13:00:53.801299  1633 raft_consensus.cc:614] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:00:53.791496  1674 connection.cc:256] RPC call timeout handler was delayed by -1.1003161560000194s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 13:00:53.797441  1490 debug-util.cc:398] Leaking SignalData structure 0x7b08003132a0 after lost signal to thread 1734
I20260709 13:00:53.550799  4831 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:53.802975  1490 debug-util.cc:398] Leaking SignalData structure 0x7b08003530a0 after lost signal to thread 1735
W20260709 13:00:53.805873  1490 debug-util.cc:398] Leaking SignalData structure 0x7b080033efe0 after lost signal to thread 2377
W20260709 13:00:53.807574  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.809173  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:53.811896  1702 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:53.746395  4765 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:53.756479  4416 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:53.816256  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:53.758088  4846 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:53.831135  1629 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:53.830183  1675 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:53.836982  1675 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:53.836956  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:53.837658  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.837901  1631 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.831890  1674 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:53.520339  4327 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.843439  1635 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:53.830389  4831 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:53.879612  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.798744  4427 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:53.840728  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 355488us
W20260709 13:00:53.883252  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46330.6us
I20260709 13:00:53.884232  1675 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:53.884312  1629 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:53.801856  4830 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:53.819990  4798 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:53.886518  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:00:53.831398  4493 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.005s
I20260709 13:00:53.886756  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2294) took 1178 ms. Trace:
W20260709 13:00:53.898214  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.786913  4627 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.917986  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.919083  1702 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
W20260709 13:00:53.921555  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.925437  1708 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:00:53.934962  1458 catalog_manager.cc:5697] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 4, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 4 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:53.936136  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 280018us
I20260709 13:00:53.811203  4730 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:53.846350  4814 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.765s	user 0.009s	sys 0.009s
I20260709 13:00:53.879587  4836 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:53.828270  4862 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:53.912125  1704 rpcz_store.cc:276] 0709 13:00:52.708545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.122630 (+414085us) service_pool.cc:224] Handling call
0709 13:00:53.122981 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:53.123213 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:53.305243 (+182030us) log.cc:844] Serialized 14 byte log entry
0709 13:00:53.885857 (+580614us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:53.885941 (+    84us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:53.886124 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.886470 (+   346us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:53.886671 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15030,"prepare.queue_time_us":276094,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":135019,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":746561,"threads_started":3,"wal-append.queue_time_us":347688}
I20260709 13:00:53.880218  4856 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:53.844626  4598 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:53.912539  4847 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:53.954355  4830 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:53.829219  4765 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:53.955799  1706 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:53.907122  4496 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:53.975082  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:53.914063  4427 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:53.962642  1631 raft_consensus.cc:2410] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:53.942245  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:53.935043  4835 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:53.973810  1629 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:53.978952  1458 catalog_manager.cc:5697] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
W20260709 13:00:53.978047  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 221105us
I20260709 13:00:53.984614  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:53.986054  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:53.942234  4854 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:53.961515  4730 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:53.960327  4493 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:54.037882  1458 catalog_manager.cc:5697] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 3 to 6, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 6 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:53.991715  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 250653us
I20260709 13:00:54.058516  1527 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:53.943492  4805 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:53.975905  4856 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:53.955478  4814 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:53.844645  4767 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.096269  1458 catalog_manager.cc:5697] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 4 to 6, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:00:54.044169  4854 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.088999  4862 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:54.056187  4730 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:54.217875  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 386034us
I20260709 13:00:54.219324  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:00:54.195806  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 283514us
I20260709 13:00:54.219729  1458 catalog_manager.cc:5697] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 3, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 3 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:54.054740  4493 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:00:54.217854  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 231869us
I20260709 13:00:54.221263  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:54.205019  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 318571us
W20260709 13:00:54.223142  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.229776  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:54.231153  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:54.230149  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:54.233407  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:54.238845  1675 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:54.242187  1601 leader_election.cc:400] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8.
I20260709 13:00:54.212875  4831 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:54.242677  1601 leader_election.cc:403] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:00:54.243031  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:54.241151  1458 catalog_manager.cc:5697] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:54.209383  4854 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:54.228307  4493 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:54.231432  4873 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
W20260709 13:00:54.293085  4644 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.810s	user 0.010s	sys 0.000s
I20260709 13:00:54.294368  4765 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:54.298133  4874 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:54.318944  4644 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:54.231671  4459 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:54.322065  4459 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.323025  4459 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:54.301373  4882 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.326191  4882 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:54.233672  4830 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:54.096401  4846 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.330657  4830 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:54.331871  4830 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.332644  4830 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:54.242843  4427 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.335680  4814 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:54.338318  4867 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 13:00:54.341059  4627 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:54.218828  4730 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:54.350972  2353 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.593s	user 0.012s	sys 0.000s
I20260709 13:00:54.303109  4885 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:54.243290  4861 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:54.365561  4605 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.579s	user 0.006s	sys 0.006s
I20260709 13:00:54.307947  4886 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.313143  4871 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:54.315785  4715 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.815s	user 0.012s	sys 0.000s
W20260709 13:00:54.316984  4252 consensus_meta.cc:287] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.919s	user 0.005s	sys 0.007s
I20260709 13:00:54.367061  4605 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:54.367843  4252 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:54.369074  1558 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:54.352072  4734 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.923s	user 0.014s	sys 0.004s
I20260709 13:00:54.370100  4605 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:54.349467  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 128240us
I20260709 13:00:54.370994  1710 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:54.370568  4734 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.371783  1630 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:54.372225  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:54.372732  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:54.374595  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:54.244611  4857 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:54.380496  4857 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:54.381336  4597 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.000s
W20260709 13:00:54.294399  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 67194.4us
I20260709 13:00:54.355222  2353 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:00:54.241674  4496 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:54.382089  4857 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:54.211719  4764 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:00:54.360852  4831 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:54.307216  4866 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 13:00:54.243506  4727 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:54.272712  4817 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:54.370456  4252 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:54.232992  4870 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.228976  4860 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:54.389176  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:00:54.405131  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.411726  1632 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:54.412602  4888 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:54.415165  4888 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:54.416365  4870 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:54.394563  1458 catalog_manager.cc:5697] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 1 to 3, now has a pending config: VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194), VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 3 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:00:54.417606  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:54.418330  1708 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:54.416451  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 539) took 1265 ms. Trace:
I20260709 13:00:54.421661  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2374) took 1035 ms. Trace:
I20260709 13:00:54.414475  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:54.422209  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:54.423977  1553 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:00:54.426476  1553 pending_rounds.cc:85] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 6
I20260709 13:00:54.426831  1553 pending_rounds.cc:107] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:00:54.427233  1553 raft_consensus.cc:2889] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:54.301198  4873 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8.
I20260709 13:00:54.400105  4734 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:54.435000  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 196230us
I20260709 13:00:54.420001  1703 rpcz_store.cc:276] 0709 13:00:53.151491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.279352 (+127861us) service_pool.cc:224] Handling call
0709 13:00:53.279715 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:53.303739 (+ 24024us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:53.303774 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:53.379123 (+ 75349us) log.cc:844] Serialized 26 byte log entry
0709 13:00:53.697464 (+318341us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:53.697555 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.697665 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:54.416217 (+718552us) raft_consensus.cc:1685] finished
0709 13:00:54.416240 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:54.416387 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":165992,"spinlock_wait_cycles":5637376,"thread_start_us":229600,"threads_started":1,"wal-append.queue_time_us":517945}
I20260709 13:00:54.441324  1703 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:54.441658  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:54.442082  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1730) took 1028 ms. Trace:
W20260709 13:00:54.442566  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.453451  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:54.444350  1559 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:54.454936  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1764) took 1116 ms. Trace:
I20260709 13:00:54.455047  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:54.402040  4856 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:54.446096  4612 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.335s	user 0.015s	sys 0.000s
I20260709 13:00:54.454631  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:54.459249  1703 rpcz_store.cc:276] 0709 13:00:53.414658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.440520 (+1025862us) service_pool.cc:224] Handling call
0709 13:00:54.441007 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.441813 (+   806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:54.442019 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:54.465858  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:54.466300  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.382131  4597 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.442657  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:54.442834  4888 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.466763  1553 raft_consensus.cc:2376] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6. Re-sending same reply.
I20260709 13:00:54.455492  1559 rpcz_store.cc:276] 0709 13:00:53.338480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.375822 (+ 37342us) service_pool.cc:224] Handling call
0709 13:00:53.390786 (+ 14964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.444082 (+1053296us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400236ca0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27651dc0 00007fda276dc1c0 00007fda276dd231 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c
0709 13:00:54.454593 (+ 10511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:54.454869 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312454912}
I20260709 13:00:54.432121  4605 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:54.433259  4882 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:54.480007  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.468695  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:54.498919  1559 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:54.425709  1635 rpcz_store.cc:276] 0709 13:00:53.386492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.879274 (+492782us) service_pool.cc:224] Handling call
0709 13:00:53.880693 (+  1419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:53.912115 (+ 31422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:53.912149 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:53.937719 (+ 25570us) log.cc:844] Serialized 26 byte log entry
0709 13:00:54.214856 (+277137us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:54.420273 (+205417us) log.cc:844] Serialized 14 byte log entry
0709 13:00:54.421261 (+   988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:54.421347 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:54.421394 (+    47us) raft_consensus.cc:1685] finished
0709 13:00:54.421410 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:54.421586 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4206,"prepare.queue_time_us":87318,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":246147,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":19834,"thread_start_us":360668,"threads_started":2,"wal-append.queue_time_us":388138}
W20260709 13:00:54.564713  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 321709us
I20260709 13:00:54.393321  4885 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:54.590054  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1769) took 1216 ms. Trace:
I20260709 13:00:54.598865  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:54.602960  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:54.465991  4747 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:54.606729  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 152194us
I20260709 13:00:54.607344  1635 raft_consensus.cc:2410] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:54.609869  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:54.569279  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:54.612269  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:54.468109  4874 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:54.468321  4727 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.621311  4860 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:54.621917  4860 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.622332  4860 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:54.622720  4252 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.623277  4252 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.464102  4892 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:54.597492  1554 rpcz_store.cc:276] 0709 13:00:53.373992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.505435 (+131443us) service_pool.cc:224] Handling call
0709 13:00:53.700735 (+195300us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:53.741839 (+ 41104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:53.741873 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:53.756728 (+ 14855us) log.cc:844] Serialized 52 byte log entry
0709 13:00:54.205778 (+449050us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:54.472555 (+266777us) log.cc:844] Serialized 14 byte log entry
0709 13:00:54.485249 (+ 12694us) log.cc:844] Serialized 14 byte log entry
0709 13:00:54.589530 (+104281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:54.589648 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:54.589713 (+    65us) raft_consensus.cc:1685] finished
0709 13:00:54.589734 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:54.589949 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125827,"prepare.queue_time_us":211241,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3787,"raft.queue_time_us":157283,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":55785,"thread_start_us":575640,"threads_started":2,"wal-append.queue_time_us":429647}
I20260709 13:00:54.631964  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:54.411149  4661 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.633098  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:54.642498  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:54.616144  4873 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.625705  4765 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.651597  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:54.628826  4871 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.633304  4252 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:54.634828  4888 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:54.661757  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:54.636986  4747 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:54.587863  4605 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:54.649286  4892 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:54.666502  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:54.659983  1674 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:54.665344  4661 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:54.665393  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:54.626376  4891 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:54.670360  1554 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:54.652247  4715 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.647764  4861 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:54.682826  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1728) took 1269 ms. Trace:
I20260709 13:00:54.667624  4887 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:54.687608  1552 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:54.695930  1708 rpcz_store.cc:276] 0709 13:00:53.413037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.419135 (+1006098us) service_pool.cc:224] Handling call
0709 13:00:54.419539 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:54.431653 (+ 12114us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:54.441626 (+  9973us) log.cc:844] Serialized 14 byte log entry
0709 13:00:54.682150 (+240524us) log.cc:844] Serialized 14 byte log entry
0709 13:00:54.682346 (+   196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:54.682408 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:54.682482 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:54.682593 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:54.682761 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31457,"prepare.queue_time_us":11433,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":87,"raft.queue_time_us":38958,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":167,"thread_start_us":208263,"threads_started":1}
I20260709 13:00:54.680346  1635 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:00:54.636660  4727 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.749595  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:54.698311  4539 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.178s	user 0.016s	sys 0.004s
I20260709 13:00:54.763298  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:54.753123  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:54.758502  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.764178  1635 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:54.764360  1554 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:54.699200  4861 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:54.700974  4887 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.668669  4765 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.749082  2318 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:54.762497  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:54.687398  4777 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:00:54.765901  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 99548.5us
I20260709 13:00:54.706569  4612 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:54.770442  4887 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:54.750721  4814 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.750313  4644 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:54.754571  4897 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.785048  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:54.749339  4895 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:54.795722  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:54.757967  4876 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.700950  4715 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.813288  1554 raft_consensus.cc:2376] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6. Re-sending same reply.
W20260709 13:00:54.819623  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.809886  4644 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.749198  4891 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:54.767843  4861 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:54.819069  4627 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.770151  4539 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.817864  4831 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:54.835381  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 87271.9us
I20260709 13:00:54.809854  4873 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.836167  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:54.820171  4874 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.803545  4814 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.841619  1635 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:54.819916  4252 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:54.789007  4612 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.845074  1561 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:54.850167  4910 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:54.850646  4847 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.891s	user 0.013s	sys 0.003s
I20260709 13:00:54.850867  4910 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:54.851159  4847 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:54.851337  4910 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:54.853847  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.831099  4597 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:54.857175  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1731) took 1437 ms. Trace:
I20260709 13:00:54.857175  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:54.858139  1705 rpcz_store.cc:276] 0709 13:00:53.419884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.450277 (+1030393us) service_pool.cc:224] Handling call
0709 13:00:54.450706 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.450972 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:54.451001 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:54.454224 (+  3223us) log.cc:844] Serialized 26 byte log entry
0709 13:00:54.813298 (+359074us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:54.813338 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:54.813419 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:54.856922 (+ 43503us) raft_consensus.cc:1685] finished
0709 13:00:54.856945 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:54.857102 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":358766,"threads_started":1,"wal-append.queue_time_us":399553}
I20260709 13:00:54.860061  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:54.860621  1705 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:54.815384  4897 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.868790  4814 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:54.870702  4539 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:54.846585  4252 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:54.871954  4493 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.582s	user 0.007s	sys 0.004s
I20260709 13:00:54.872066  4765 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:54.839715  4873 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:54.875550  4459 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.552s	user 0.014s	sys 0.000s
I20260709 13:00:54.827801  4892 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:54.878460  4901 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:00:54.863620  4885 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:54.846408  4496 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.889951  4605 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:54.844251  4874 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.836716  4627 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:54.895018  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:54.937927  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 83044.6us
I20260709 13:00:54.799003  4747 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:54.897437  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2295) took 2188 ms. Trace:
W20260709 13:00:54.926858  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 86957.3us
I20260709 13:00:54.930807  4398 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:54.932950  4854 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.575s	user 0.011s	sys 0.000s
I20260709 13:00:54.921020  4416 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:54.899457  1735 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:43857 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:43857 timed out after 15.000s (SENT)
I20260709 13:00:54.825850  4876 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.951946  1556 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:54.947511  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:54.941000  1709 rpcz_store.cc:276] 0709 13:00:52.708842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.162826 (+453984us) service_pool.cc:224] Handling call
0709 13:00:53.163100 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:53.314307 (+151207us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:53.634824 (+320517us) log.cc:844] Serialized 14 byte log entry
0709 13:00:53.943736 (+308912us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:53.943796 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:53.943849 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:54.897160 (+953311us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540014dcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:54.897216 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:54.897366 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141916,"prepare.queue_time_us":85667,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":25,"raft.queue_time_us":45284,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":11334,"spinlock_wait_cycles":347207040,"thread_start_us":537668,"threads_started":2,"wal-append.queue_time_us":664069}
I20260709 13:00:54.953877  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:54.954672  1709 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:54.957669  1554 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:00:54.958052  1554 pending_rounds.cc:85] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 8
I20260709 13:00:54.958338  1554 pending_rounds.cc:107] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 6.9
I20260709 13:00:54.958581  1554 raft_consensus.cc:2889] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:54.952380  4327 consensus_queue.cc:1059] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:00:54.925446  4814 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:00:54.946065  4715 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:54.923237  4493 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.964011  1705 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:54.966045  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:54.967149  4860 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:54.967713  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:54.968783  1705 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:54.971061  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:54.935410  4459 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:54.981187  4627 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:54.981520  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:54730 (request call id 81) took 1765 ms. Trace:
I20260709 13:00:54.983428  1633 rpcz_store.cc:276] 0709 13:00:53.216441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.240153 (+ 23712us) service_pool.cc:224] Handling call
0709 13:00:54.981239 (+1741086us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b54002da0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d4853 00007fda2c6a260a 00007fda25c287d4 00007fda25c28762 00007fda25c28691 00007fda25c2860c 00007fda25c278c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:54.981389 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1167650048}
I20260709 13:00:54.985203  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:54.943710  4496 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:54.981413  4493 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:54.987633  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:54.988974  4252 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:54.990090  4459 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:54.991573  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1734) took 1503 ms. Trace:
I20260709 13:00:54.994946  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:55.000602  1710 rpcz_store.cc:276] 0709 13:00:53.488093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.495459 (+  7366us) service_pool.cc:224] Handling call
0709 13:00:54.991333 (+1495874us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b5400223b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:54.991488 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1958869376}
I20260709 13:00:55.002205  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.002722  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:55.014506  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.015587  1710 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:55.015914  4874 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:55.016708  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:54.957367  4416 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:55.017783  1710 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:55.019429  4843 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.626s	user 0.008s	sys 0.015s
I20260709 13:00:55.019918  4416 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:55.021639  4903 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.203s	user 0.002s	sys 0.000s
I20260709 13:00:55.024806  4876 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:55.025712  4897 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:55.026027  4496 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:55.027621  4860 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:55.004005  4763 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:55.009876  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:54.952423  4854 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:55.029423  4763 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:55.029565  4612 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:55.030185  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:55.031159  1703 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:55.028757  4644 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:55.031901  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:55.011281  4891 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:55.048755  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.050354  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.051276  1709 raft_consensus.cc:2410] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:00:54.998245  4803 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 13:00:55.056450  4830 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.703s	user 0.013s	sys 0.000s
I20260709 13:00:55.057255  4830 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:55.059490  1627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2372) took 1724 ms (client timeout 1502 ms). Trace:
W20260709 13:00:55.059984  1627 rpcz_store.cc:269] 0709 13:00:53.335259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.354271 (+ 19012us) service_pool.cc:224] Handling call
0709 13:00:55.059293 (+1705022us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b540017c354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:55.059402 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902124416}
I20260709 13:00:55.063786  1632 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:55.064217  1632 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:55.060140  4830 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:55.069983  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1741) took 1545 ms (client timeout 1678 ms). Trace:
I20260709 13:00:55.070955  1702 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:55.071823  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1739) took 1548 ms. Trace:
I20260709 13:00:55.072165  1702 rpcz_store.cc:276] 0709 13:00:53.523357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.070252 (+1546895us) service_pool.cc:224] Handling call
0709 13:00:55.070723 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.071523 (+   800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.071757 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:55.070641  1701 rpcz_store.cc:269] 0709 13:00:53.524564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.623907 (+ 99343us) service_pool.cc:224] Handling call
0709 13:00:55.065312 (+1441405us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b540036ddd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:55.069893 (+  4581us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2092641536}
I20260709 13:00:55.075047  1701 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:55.075697  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:55.077883  4902 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 13:00:55.083238  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:55.078564  1627 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:55.089496  1702 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:55.093943  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 595) took 1697 ms. Trace:
I20260709 13:00:55.114087  1629 rpcz_store.cc:276] 0709 13:00:53.396177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.935505 (+539328us) service_pool.cc:224] Handling call
0709 13:00:53.954875 (+ 19370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:53.981162 (+ 26287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.093629 (+1112467us) spinlock_profiling.cc:243] Waited 5.92 ms on lock 0x7b540017b454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:55.093866 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13614976}
I20260709 13:00:55.095568  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1768) took 1738 ms. Trace:
I20260709 13:00:55.115728  1558 rpcz_store.cc:276] 0709 13:00:53.356863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.460648 (+103785us) service_pool.cc:224] Handling call
0709 13:00:53.489231 (+ 28583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:54.369633 (+880402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.095302 (+725669us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b5400271fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:55.095501 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2022826368}
I20260709 13:00:55.113430  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1742) took 1394 ms. Trace:
I20260709 13:00:55.117530  1702 rpcz_store.cc:276] 0709 13:00:53.719240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.073217 (+1353977us) service_pool.cc:224] Handling call
0709 13:00:55.089237 (+ 16020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.113122 (+ 23885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.113355 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:55.124960  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 143557us
I20260709 13:00:55.163543  1710 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:00:55.164716  1709 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:55.163763  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 216356us
W20260709 13:00:55.163389  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 80251us
I20260709 13:00:55.167768  1710 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:55.169086  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.170681  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.172031  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.173358  1710 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:55.175756  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.175844  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.138656  4605 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:55.174523  1632 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:55.184075  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.184990  2346 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:55.180522  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:55.030673  4763 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:55.201246  4905 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.401s	user 0.000s	sys 0.002s
I20260709 13:00:55.089814  4427 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 13:00:55.174558  4882 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.211354  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.211551  4882 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.212098  1553 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:55.192269  4777 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.212759  4727 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:55.213088  4777 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:55.213882  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:55.214079  4882 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:55.214519  2348 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.210168  1627 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:55.193161  1705 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:55.207481  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:55.193903  4819 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:55.199791  4730 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.810s	user 0.015s	sys 0.021s
I20260709 13:00:55.200536  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.224589  2346 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:55.201813  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:55.203511  4661 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.507s	user 0.004s	sys 0.023s
I20260709 13:00:55.205039  4831 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:55.183670  4767 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.192301  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.218943  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.241398  1553 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:55.191020  1710 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:55.220901  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:55.223495  1558 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:00:55.226027  2352 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:55.227125  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.231010  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.209947  1632 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:55.233516  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:55.183341  4605 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:55.257742  1558 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:55.236788  2351 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:55.237766  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.244673  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:55.247115  1627 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:55.248564  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:55.244797  4856 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.243418  1555 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:55.259004  1705 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:55.260635  1553 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:55.261433  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.190182  4878 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:00:55.263608  1554 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:55.266692  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.235384  4882 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.266206  4817 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.282480  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.268756  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.284703  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:55.282511  4882 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.285943  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:55.266399  4856 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:55.287125  1632 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:00:55.287631  1632 pending_rounds.cc:85] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 2
I20260709 13:00:55.287032  4910 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.287910  1632 pending_rounds.cc:107] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:55.261571  4727 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.288239  1632 raft_consensus.cc:2889] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:55.287631  4882 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:55.288700  4727 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.264375  4756 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.269169  4891 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:00:55.270074  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.270212  4418 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:00:55.270097  4907 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.456s	user 0.000s	sys 0.003s
I20260709 13:00:55.270366  4851 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:55.270812  4893 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.296481  4893 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.298874  4893 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:55.271350  4860 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:55.272164  4862 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:55.299777  4860 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:55.299739  4862 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:55.294513  4418 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:55.302438  4862 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:55.246052  4767 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.304307  4418 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:55.304879  4767 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:55.254086  4605 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.277194  4917 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:55.294902  4756 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:55.277076  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.308629  4756 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:55.309000  4820 raft_consensus.cc:3055] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:55.273885  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.266687  4830 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:55.252242  4427 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:00:55.252894  4887 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:55.281821  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.320089  1558 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:55.231324  4730 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.268059  1710 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:55.290833  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.293694  1554 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:55.300305  4860 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:55.301936  4861 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.278225  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:55.326140  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:55.326506  1627 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:55.312176  1561 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.279417  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:55.313833  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 213931us
I20260709 13:00:55.288259  4856 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:55.322137  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:55.324780  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.309443  4820 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:55.314291  4427 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:55.326799  1560 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:00:55.483078  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 161045us
I20260709 13:00:55.363629  4851 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.114s
I20260709 13:00:55.344926  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:55.365288  1705 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
W20260709 13:00:55.331583  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.471221  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.475839  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:55.479252  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:55.441787  1708 raft_consensus.cc:2368] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7.
I20260709 13:00:55.440740  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2171) took 1062 ms. Trace:
I20260709 13:00:55.314345  4830 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.296216  4493 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:55.484259  1558 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:55.481216  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.479007  1555 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:55.535442  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.484265  2318 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:55.349938  2352 raft_consensus.cc:2410] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:55.422341  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:55.561895  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 77719.6us
I20260709 13:00:55.548890  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:55.563632  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 84445.7us
I20260709 13:00:55.564213  1635 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.552076  1561 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:55.557566  1632 rpcz_store.cc:276] 0709 13:00:54.377946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.063256 (+685310us) service_pool.cc:224] Handling call
0709 13:00:55.063611 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.440269 (+376658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.440643 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:55.557176  1627 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:55.556582  1560 raft_consensus.cc:2410] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:00:55.572621  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:55.574448  1632 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
W20260709 13:00:55.575366  1556 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.622s	user 0.020s	sys 0.004s
I20260709 13:00:55.575949  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:55.576344  1556 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:55.577033  1632 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:55.562860  2318 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:55.579361  2349 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:55.579515  2346 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:55.583623  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.585433  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.523754  4327 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:55.422008  4418 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.590608  2350 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:00:55.544404  4427 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.590919  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.547150  4830 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:55.598970  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:55.546972  4868 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.419s	user 0.000s	sys 0.003s
I20260709 13:00:55.543650  4493 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:55.606501  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:55.394500  4891 consensus_queue.cc:1059] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.205s
I20260709 13:00:55.608419  2348 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:55.618845  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.611402  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:55.612169  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.587280  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.621349  1555 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:55.561509  4644 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.621721  1556 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:55.580363  4876 raft_consensus.cc:2764] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:55.618315  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:55.580734  4661 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:55.604841  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.612747  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.544231  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.592334  4873 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:55.625603  1710 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:00:55.613672  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.603426  4910 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:55.547057  4860 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:55.591250  4820 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.607132  1632 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:55.756603  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:55.628171  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.757944  1632 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:55.681031  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.682751  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:55.683229  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1756) took 2375 ms (client timeout 2818 ms). Trace:
W20260709 13:00:55.763538  1557 rpcz_store.cc:269] 0709 13:00:53.308308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.328495 (+ 20187us) service_pool.cc:224] Handling call
0709 13:00:55.683039 (+2354544us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b540038a4d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:55.683166 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":982490496}
I20260709 13:00:55.686590  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:55.748708  1635 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:55.768275  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:55.686590  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.770365  1702 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:55.778901  1635 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 6 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:00:55.759341  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:55.787305  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.788484  1708 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:00:55.791934  2349 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 4: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:55.711599  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:55.791213  1708 pending_rounds.cc:85] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 4
I20260709 13:00:55.796471  1708 pending_rounds.cc:107] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:00:55.797971  1702 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:55.600939  4327 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:55.799139  1708 raft_consensus.cc:2889] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:55.793105  4817 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:55.801610  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:55.807400  1636 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:55.800958  1632 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:55.801898  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2165) took 2583 ms. Trace:
I20260709 13:00:55.821131  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:55.656356  4876 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:55.819977  1634 rpcz_store.cc:276] 0709 13:00:53.211383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.214706 (+  3323us) service_pool.cc:224] Handling call
0709 13:00:53.215066 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:53.406626 (+191560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:53.406661 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:53.407161 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:00:53.785619 (+378458us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:53.785679 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.785814 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:55.477090 (+1691276us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:00:55.794267 (+317177us) raft_consensus.cc:1685] finished
0709 13:00:55.794294 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.794584 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55999360,"thread_start_us":302521,"threads_started":1,"wal-append.queue_time_us":497592}
I20260709 13:00:55.664456  4777 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:55.673679  4661 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:55.607923  4418 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:55.657299  4920 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:00:55.833079  1555 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:00:55.658845  4598 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.824299  4327 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:55.836737  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2383) took 1038 ms. Trace:
I20260709 13:00:55.855538  1632 rpcz_store.cc:276] 0709 13:00:54.798772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.835499 (+1036727us) service_pool.cc:224] Handling call
0709 13:00:55.835999 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.836270 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.836312 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:55.836375 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:55.836427 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.836508 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.836658 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:55.866490  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 521616us
W20260709 13:00:55.842566  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 279797us
I20260709 13:00:55.870734  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:55.881232  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.868849  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:55.843917  4730 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:55.662045  4493 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:55.791724  4756 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.658279  4887 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:55.843395  1634 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:00:55.972855  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2382) took 1200 ms. Trace:
I20260709 13:00:55.626551  4644 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.831290  4860 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.844180  4876 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:00:55.961266  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 92555.7us
W20260709 13:00:55.962533  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 91917.4us
I20260709 13:00:55.850183  4817 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:55.838299  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 355323us
I20260709 13:00:55.996608  2318 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:55.682651  4861 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:55.994602  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:55.973459  1634 rpcz_store.cc:276] 0709 13:00:54.773664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.837677 (+1064013us) service_pool.cc:224] Handling call
0709 13:00:55.843008 (+  5331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.972404 (+129396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.972767 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:56.000924  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:55.865054  4819 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.000s
I20260709 13:00:56.001813  1634 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:56.003090  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:55.835682  1635 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:56.006493  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2380) took 1257 ms. Trace:
I20260709 13:00:56.007026  1635 rpcz_store.cc:276] 0709 13:00:54.749063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.798467 (+1049404us) service_pool.cc:224] Handling call
0709 13:00:55.831901 (+ 33434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:56.006202 (+174301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.006432 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:55.994858  1628 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.008s	sys 0.029s
I20260709 13:00:55.850772  4598 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:55.966670  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 534) took 2917 ms. Trace:
I20260709 13:00:55.998420  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:56.009908  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:56.011273  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.017251  1675 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:56.024035  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:56.025354  1708 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 6 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:00:55.994537  4493 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.031104  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:56.031803  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:56.012979  1707 rpcz_store.cc:276] 0709 13:00:53.049310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.241593 (+192283us) service_pool.cc:224] Handling call
0709 13:00:53.241894 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:53.512272 (+270378us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:53.624145 (+111873us) log.cc:844] Serialized 14 byte log entry
0709 13:00:53.955606 (+331461us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:53.956278 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 13:00:53.956448 (+   170us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:53.956483 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.860463 (+1903980us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b540016c954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:55.860536 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:55.860603 (+    67us) raft_consensus.cc:1685] finished
0709 13:00:55.966393 (+105790us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.966604 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79084,"prepare.queue_time_us":111532,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":266967,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":7857,"spinlock_wait_cycles":332546304,"thread_start_us":374511,"threads_started":2,"wal-append.queue_time_us":642325,"wal-append.run_cpu_time_us":2628,"wal-append.run_wall_time_us":1003689}
I20260709 13:00:56.032943  4910 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:55.996811  4817 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:55.986286  4756 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.033705  4861 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.034222  4835 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.034292  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:56.037266  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:56.007805  4887 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:56.045331  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:56.017766  1628 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.009s	sys 0.030s
I20260709 13:00:56.076233  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1724) took 2742 ms. Trace:
I20260709 13:00:56.076905  1706 rpcz_store.cc:276] 0709 13:00:53.361901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.383423 (+ 21522us) service_pool.cc:224] Handling call
0709 13:00:53.383839 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:53.968872 (+585033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.075899 (+2107027us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b540014d2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:56.076151 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1314818944}
I20260709 13:00:56.008148  4917 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:56.037246  4605 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.003s
I20260709 13:00:56.063681  1633 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:56.060328  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:56.101101  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55914.6us
I20260709 13:00:56.102142  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:56.102487  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:56.102877  1635 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:56.104569  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:56.035651  4836 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:56.055193  4910 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:56.072320  4835 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.012657  4831 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:56.064455  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:56.041998  1634 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
W20260709 13:00:56.064226  1600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.012s	sys 0.021s
I20260709 13:00:56.021270  4730 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.030836  4661 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:56.056510  4819 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:56.100313  4924 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.307s	user 0.001s	sys 0.001s
I20260709 13:00:56.060467  4817 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.060578  4777 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:56.058507  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:56.153024  1704 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:56.144384  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:56.105868  4767 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.071719  4756 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:56.105059  4908 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:00:56.157835  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:00:56.159308  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 490600us
W20260709 13:00:56.134737  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:56.161008  2317 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.019s	sys 0.032s
I20260709 13:00:56.154587  1635 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:00:56.072043  4861 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:00:56.164069  2317 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.019s	sys 0.033s
I20260709 13:00:56.100739  4887 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.168089  1559 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:56.141055  4851 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.101783  4917 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.104756  4605 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:56.131045  4910 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.118296  4882 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:56.187350  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:56.184377  4927 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
W20260709 13:00:56.190029  2353 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 825) took 3091 ms (client timeout 2059 ms). Trace:
I20260709 13:00:56.189085  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1733) took 2717 ms. Trace:
I20260709 13:00:56.195163  1703 rpcz_store.cc:276] 0709 13:00:53.471782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.464059 (+992277us) service_pool.cc:224] Handling call
0709 13:00:54.464462 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.031558 (+567096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.188499 (+1156941us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b5400224a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:56.189004 (+   505us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1302768384}
W20260709 13:00:56.192728  2353 rpcz_store.cc:269] 0709 13:00:53.098572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.098697 (+   125us) service_pool.cc:224] Handling call
0709 13:00:56.189636 (+3090939us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b540002b2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:56.189786 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":956094720}
W20260709 13:00:56.173628  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 609073us
I20260709 13:00:56.168705  4819 raft_consensus.cc:697] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:56.205906  1559 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:56.178550  4908 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:56.151541  4730 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.163959  4767 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.166755  4661 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:00:56.145416  4831 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.179503  4930 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:56.188858  4803 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.012s	sys 0.026s
I20260709 13:00:56.172124  4756 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
W20260709 13:00:56.199062  4925 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.347s	user 0.002s	sys 0.000s
I20260709 13:00:56.119050  4876 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:56.261822  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 227630us
I20260709 13:00:56.267138  1557 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:56.225363  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.207710  4882 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:56.226117  4908 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:56.186755  4851 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.263190  4930 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:56.270449  4803 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.012s	sys 0.027s
I20260709 13:00:56.351869  1555 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.258785  4933 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:56.353987  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 249514us
I20260709 13:00:56.357504  4932 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:56.361323  4598 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:56.368055  4932 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.368223  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:56.368501  4932 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.368299  4598 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:56.354741  4930 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.363122  4882 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:56.370162  4598 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:56.370594  4598 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:56.371207  4598 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.364053  4836 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:56.339108  4908 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.341406  4934 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:56.365880  4803 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:56.375089  4934 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.375541  4934 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.378775  4835 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:56.379590  4835 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:56.271552  4418 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:56.355055  4844 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:56.381397  4819 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.431613  4916 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:56.435004  4730 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:56.437140  4831 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:56.437638  4531 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:56.444289  1627 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:56.445159  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2182) took 1210 ms. Trace:
I20260709 13:00:56.445554  1627 rpcz_store.cc:276] 0709 13:00:55.234942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.413062 (+1178120us) service_pool.cc:224] Handling call
0709 13:00:56.444014 (+ 30952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:56.444839 (+   825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.445092 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:56.419142  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 539) took 1253 ms. Trace:
I20260709 13:00:56.448154  2351 rpcz_store.cc:276] 0709 13:00:55.166008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.166179 (+   171us) service_pool.cc:224] Handling call
0709 13:00:56.418883 (+1252704us) spinlock_profiling.cc:243] Waited 60.2 ms on lock 0x7b54004f4a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:56.419049 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138540416}
I20260709 13:00:56.367014  4933 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:56.439710  4887 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:56.404848  4935 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.411890  4864 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:56.462131  4936 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.107s	user 0.000s	sys 0.002s
I20260709 13:00:56.428552  4873 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:00:56.477954  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 404353us
I20260709 13:00:56.476456  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2169) took 2100 ms. Trace:
I20260709 13:00:56.399904  4605 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:56.525547  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2361) took 3194 ms. Trace:
I20260709 13:00:56.524540  1631 rpcz_store.cc:276] 0709 13:00:54.376375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.376523 (+   148us) service_pool.cc:224] Handling call
0709 13:00:54.376968 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:54.636093 (+259125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:54.636124 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:54.636171 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:54.636236 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.476181 (+1839945us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b5400197654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:56.476229 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.476395 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595349376}
I20260709 13:00:56.526100  1630 rpcz_store.cc:276] 0709 13:00:53.331308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.333538 (+  2230us) service_pool.cc:224] Handling call
0709 13:00:53.333870 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.372222 (+1038352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.525164 (+2152942us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54003b8dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:56.525443 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2386276224}
I20260709 13:00:56.529945  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:48236 (request call id 13) took 1502 ms. Trace:
I20260709 13:00:56.535323  1627 rpcz_store.cc:276] 0709 13:00:55.001132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.459934 (+1458802us) service_pool.cc:224] Handling call
0709 13:00:56.503434 (+ 43500us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:56.544042  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:56.544947  1707 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:56.544095  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:56.554390  4854 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:56.555266  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:56.556085  1704 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:56.557570  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:56.557842  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1780) took 1303 ms. Trace:
I20260709 13:00:56.558238  1558 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:56.559405  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:56.558480  1556 rpcz_store.cc:276] 0709 13:00:55.254281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.622573 (+368292us) service_pool.cc:224] Handling call
0709 13:00:55.624090 (+  1517us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:55.624799 (+   709us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.624851 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:55.780691 (+155840us) log.cc:844] Serialized 52 byte log entry
0709 13:00:56.199460 (+418769us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:56.199524 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.199652 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:56.557548 (+357896us) raft_consensus.cc:1685] finished
0709 13:00:56.557579 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.557769 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1425,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":58400,"thread_start_us":357949,"threads_started":1,"wal-append.queue_time_us":615155}
I20260709 13:00:56.560698  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:56.561919  4940 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:56.564970  4940 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:56.561942  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1735) took 3063 ms. Trace:
I20260709 13:00:56.566013  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2285) took 1250 ms. Trace:
I20260709 13:00:56.566444  1557 rpcz_store.cc:276] 0709 13:00:55.315301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.765177 (+449876us) service_pool.cc:224] Handling call
0709 13:00:55.765628 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:56.272397 (+506769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.565782 (+293385us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54003205d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:56.565952 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152776448}
I20260709 13:00:56.567391  4493 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:56.567685  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:56.567795  4830 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.951s	user 0.012s	sys 0.002s
I20260709 13:00:56.567963  4493 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:56.568526  4830 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.558014  1710 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:56.561055  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:56.566053  1709 rpcz_store.cc:276] 0709 13:00:53.499065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.051899 (+1552834us) service_pool.cc:224] Handling call
0709 13:00:55.164519 (+112620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.165174 (+   655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.561736 (+1396562us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54001e8f50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:56.561874 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258169216}
I20260709 13:00:56.561305  1558 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:56.577792  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:56.578476  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:56.568576  1557 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:56.579071  1556 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:56.581051  1556 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:56.582880  1555 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:56.582913  1556 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:00:56.586006  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51680.2us
W20260709 13:00:56.585233  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 102600us
I20260709 13:00:56.598474  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:56.599468  1707 raft_consensus.cc:2376] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6. Re-sending same reply.
W20260709 13:00:56.603026  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.485417  4416 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.473008  4933 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.605538  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:56.606393  1710 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:56.608083  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:56.608934  1707 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:56.607864  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:56.610256  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:56.610577  1558 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:56.491715  4727 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:56.491318  4887 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.524534  4873 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:56.478771  4861 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:56.623817  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 424903us
I20260709 13:00:56.625939  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:56.621774  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:00:56.627827  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 412921us
I20260709 13:00:56.629225  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:56.598820  4730 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:56.603998  4854 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.632983  1558 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:56.639890  4427 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:56.618626  4416 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:56.644532  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 371876us
W20260709 13:00:56.647280  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.015s	sys 0.012s
I20260709 13:00:56.488576  4935 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:56.493952  4864 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:56.651628  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.016s	sys 0.012s
I20260709 13:00:56.634522  4730 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:56.644806  4891 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.006s	sys 0.013s
I20260709 13:00:56.661202  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:56.653220  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 549530us
W20260709 13:00:56.660214  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.569316  4493 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.968628  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2173) took 2589 ms. Trace:
I20260709 13:00:56.970630  4644 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:56.643193  4854 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:56.979316  1675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.798s (SENT) [suppressed 46 similar messages]
I20260709 13:00:57.014550  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2178) took 2245 ms. Trace:
I20260709 13:00:56.981240  1632 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:56.979586  1554 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:56.979540  1561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1781) took 1725 ms (client timeout 2206 ms). Trace:
I20260709 13:00:57.014995  1628 rpcz_store.cc:276] 0709 13:00:54.378767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.067450 (+688683us) service_pool.cc:224] Handling call
0709 13:00:55.483315 (+415865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.483597 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:56.075673 (+592076us) log.cc:844] Serialized 14 byte log entry
0709 13:00:56.478597 (+402924us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:56.478675 (+    78us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:56.478743 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.478838 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.968445 (+489607us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b5400344250. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:56.968559 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15119,"prepare.queue_time_us":792624,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":176548,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":30740,"spinlock_wait_cycles":955307904,"thread_start_us":990831,"threads_started":2}
W20260709 13:00:57.016397  1561 rpcz_store.cc:269] 0709 13:00:55.254534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.285695 (+ 31161us) service_pool.cc:224] Handling call
0709 13:00:56.967042 (+1681347us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b54003c0fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:56.979474 (+ 12432us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58864512}
I20260709 13:00:56.982820  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 442) took 2558 ms. Trace:
W20260709 13:00:56.767439  1560 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.014s	sys 0.035s
I20260709 13:00:56.984782  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 831) took 1450 ms. Trace:
I20260709 13:00:56.635614  4946 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:56.985585  1633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2377) took 2279 ms (client timeout 2244 ms). Trace:
W20260709 13:00:56.661587  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:56.679428  4891 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.006s	sys 0.014s
W20260709 13:00:56.904590  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 352502us
I20260709 13:00:56.663753  4861 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
W20260709 13:00:56.986776  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 325794us
I20260709 13:00:56.663233  4873 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.602794  4940 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:56.657668  4730 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:56.903832  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 312989us
I20260709 13:00:57.038334  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 600) took 1936 ms. Trace:
W20260709 13:00:57.039021  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 377899us
I20260709 13:00:57.038767  1632 rpcz_store.cc:276] 0709 13:00:55.101698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.983295 (+881597us) service_pool.cc:224] Handling call
0709 13:00:56.980999 (+997704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.016131 (+ 35132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.038145 (+ 22014us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b5400343850. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:57.038274 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2293282176}
W20260709 13:00:56.878551  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 247345us
I20260709 13:00:56.631892  4887 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.003469  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:00:57.047958  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44647.4us
W20260709 13:00:57.003005  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.012346  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 601) took 1910 ms. Trace:
I20260709 13:00:57.039202  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:57.050000  1634 rpcz_store.cc:276] 0709 13:00:55.102002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.133570 (+1031568us) service_pool.cc:224] Handling call
0709 13:00:56.134004 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:56.134307 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:56.618318 (+484011us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.011751 (+393433us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:57.011870 (+   119us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:57.012003 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.012134 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.012279 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141196,"prepare.queue_time_us":382117,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":32097,"thread_start_us":717350,"threads_started":2}
W20260709 13:00:57.019300  1560 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.022s	sys 0.046s
W20260709 13:00:57.020001  1633 rpcz_store.cc:269] 0709 13:00:54.706250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.031644 (+325394us) service_pool.cc:224] Handling call
0709 13:00:56.918158 (+1886514us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400343854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:56.985469 (+ 67311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2369700224}
I20260709 13:00:57.015506  1636 rpcz_store.cc:276] 0709 13:00:54.769420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.809280 (+1039860us) service_pool.cc:224] Handling call
0709 13:00:55.814596 (+  5316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.814941 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:56.200955 (+386014us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.013830 (+812875us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:57.013912 (+    82us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:57.014032 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.014168 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.014487 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":639986,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":6674,"thread_start_us":1168367,"threads_started":3,"wal-append.queue_time_us":485143}
W20260709 13:00:57.007215  4817 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.847s	user 0.013s	sys 0.005s
W20260709 13:00:57.029625  4327 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.063s	user 0.012s	sys 0.012s
I20260709 13:00:57.017926  1552 rpcz_store.cc:276] 0709 13:00:54.424888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.433943 (+  9055us) service_pool.cc:224] Handling call
0709 13:00:54.434292 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:54.697695 (+263403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.706354 (+2008659us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54003200d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:56.982741 (+276387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581472000}
I20260709 13:00:57.019238  2350 rpcz_store.cc:276] 0709 13:00:55.534399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.534824 (+   425us) service_pool.cc:224] Handling call
0709 13:00:56.920650 (+1385826us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b54007620d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:56.984702 (+ 64052us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91259648}
I20260709 13:00:57.030328  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:57.041838  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:57.041270  1602 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:57.044139  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:57.035460  4934 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.659s	user 0.017s	sys 0.003s
I20260709 13:00:56.987309  4954 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.010859  4727 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:57.046489  4531 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.067655  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:57.075181  2346 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:57.066892  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:57.081465  2351 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:57.058076  4749 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:57.001784  4756 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:00:57.071511  2349 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 6
W20260709 13:00:57.022009  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:57.091428  4882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.005s	sys 0.010s
I20260709 13:00:57.066324  4817 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:57.176481  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2292) took 1858 ms. Trace:
I20260709 13:00:57.179211  1554 rpcz_store.cc:276] 0709 13:00:55.317707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.806281 (+488574us) service_pool.cc:224] Handling call
0709 13:00:56.979410 (+1173129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.025671 (+ 46261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.176252 (+150581us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54003c0fd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:57.176412 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2696848256}
I20260709 13:00:57.109795  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:57.071204  4891 consensus_queue.cc:1059] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 13:00:57.014130  4950 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:00:57.080231  4727 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:00:57.104190  2349 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.7
I20260709 13:00:57.072873  4767 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:57.075285  4850 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:57.191495  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 152358us
I20260709 13:00:57.195748  2352 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:57.239144  2318 leader_election.cc:400] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7.
I20260709 13:00:57.257429  2318 leader_election.cc:403] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:57.257839  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:57.257901  1635 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:57.086900  4946 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:57.066581  4941 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.004s	sys 0.007s
I20260709 13:00:57.105705  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:57.223603  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:57.075898  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:57.105500  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:57.282176  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:57.175352  4945 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:00:57.109520  4882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.006s	sys 0.010s
I20260709 13:00:57.298231  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:57.174671  4930 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.805s	user 0.006s	sys 0.007s
W20260709 13:00:57.242254  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 212009us
I20260709 13:00:57.311081  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:57.312765  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:57.242537  2349 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Aborting config change with OpId 6.7: Aborted: Op aborted by new leader
I20260709 13:00:57.102710  4952 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.313913  2349 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:57.236920  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 257630us
I20260709 13:00:57.264668  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1785) took 2009 ms. Trace:
I20260709 13:00:57.115970  4897 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:57.102972  4493 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:57.092274  4749 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:57.109408  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:57.325243  2351 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:57.275200  2347 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.280189  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:57.274266  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:57.274137  4830 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:57.068861  4881 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.179s
I20260709 13:00:57.275348  4917 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:57.295373  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56443.7us
I20260709 13:00:57.071856  4955 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.246628  4727 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:57.307351  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:57.356694  2350 raft_consensus.cc:2410] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:57.076653  4934 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.234596  4970 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.341930  4835 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:57.246230  4891 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:57.316444  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:57.319073  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:57.314682  1560 rpcz_store.cc:276] 0709 13:00:55.255517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.664136 (+408619us) service_pool.cc:224] Handling call
0709 13:00:55.664443 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.664658 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:56.173087 (+508429us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.256421 (+1083334us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:57.256488 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:57.264395 (+  7907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.264466 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.264612 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45516,"prepare.queue_time_us":701928,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":50301,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":81060,"thread_start_us":1579556,"threads_started":3}
I20260709 13:00:57.321122  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:57.073153  4327 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.081188  4954 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:57.314822  4961 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
W20260709 13:00:57.296895  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 255845us
I20260709 13:00:57.377816  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:57.380640  2347 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:57.353296  4830 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.265879  4850 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:57.104274  4531 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.299814  2344 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.315841  4763 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:57.299742  4893 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:57.315964  4767 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:57.299708  4851 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:57.321334  4416 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:57.348125  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:57.233467  4734 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:00:57.381731  4908 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.007s	user 0.008s	sys 0.010s
W20260709 13:00:57.347766  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 67347.5us
I20260709 13:00:57.352418  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:57.329388  4749 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.326234  4974 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.348361  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:57.263972  4756 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:00:57.407104  4949 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 13:00:57.330614  4935 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:57.634775  1706 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.003s	sys 0.041s
I20260709 13:00:57.345166  4861 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:00:57.371399  2346 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:57.371758  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:57.341351  1561 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
W20260709 13:00:57.378917  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 329408us
I20260709 13:00:57.382898  1602 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:57.642663  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:57.350059  4864 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:57.350905  4897 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:00:57.542779  4851 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:57.650769  2346 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:00:57.651046  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 269550us
I20260709 13:00:57.377710  4965 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:57.396098  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.382087  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:57.424624  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 105639us
I20260709 13:00:57.366892  4835 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:57.382076  4932 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.013s	user 0.010s	sys 0.004s
W20260709 13:00:57.582753  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:57.380046  4860 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.376s	user 0.014s	sys 0.008s
W20260709 13:00:57.618680  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 270379us
I20260709 13:00:57.357877  1675 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:57.621236  4756 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:00:57.396162  4967 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:00:57.416114  4934 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:00:57.626843  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 80899us
I20260709 13:00:57.664674  4864 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:57.382215  4910 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.181s	user 0.015s	sys 0.005s
I20260709 13:00:57.341778  4840 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:57.635418  1706 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.003s	sys 0.041s
W20260709 13:00:57.635344  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.023s	sys 0.012s
W20260709 13:00:57.396708  4966 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:00:57.420641  4941 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:57.725353  4941 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.384838  4958 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:57.407614  4850 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:57.321624  4930 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.643573  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:57.383740  4819 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7.
I20260709 13:00:57.408550  4817 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:57.645761  2352 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
W20260709 13:00:57.736570  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 93061.8us
I20260709 13:00:57.417228  4887 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:57.543335  4598 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.172s	user 0.018s	sys 0.004s
I20260709 13:00:57.649648  2351 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "169e9f6ed9684060b36e93c778925461"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:60938
I20260709 13:00:57.321977  4952 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.658628  2348 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:57.749174  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.427313  4767 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:57.758266  4598 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:57.423967  4763 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:57.392396  4954 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:57.671033  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 313222us
W20260709 13:00:57.407769  4427 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.767s	user 0.015s	sys 0.000s
I20260709 13:00:57.670351  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:57.674978  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.630996  4956 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:57.664012  4851 consensus_queue.cc:1059] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 13:00:57.703279  1703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.029s	sys 0.074s
W20260709 13:00:57.792321  1703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.030s	sys 0.074s
I20260709 13:00:57.649806  4416 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:57.402518  4830 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:57.711907  4957 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.347s	user 0.001s	sys 0.000s
I20260709 13:00:57.664175  4835 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:57.712971  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.616827  4734 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.380730  4891 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:57.488562  4893 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.370421  4970 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:57.416366  4946 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:57.385341  4933 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.765s	user 0.009s	sys 0.008s
I20260709 13:00:57.667090  4982 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:57.846401  1507 auto_rebalancer.cc:893] Removing replace marker failed: Replica ba0548b7d2a94e8598475a46c279fe0e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:00:57.846596  4982 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:57.846895  1507 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:00:57.847215  1507 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:00:57.847903  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:57.849424  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:57.850392  4933 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:57.852128  4955 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:57.852764  4933 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:57.853226  2345 raft_consensus.cc:2410] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:57.853417  4986 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:57.853931  4986 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.854302  4986 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.856408  4891 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:57.702512  4861 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:00:57.721336  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.024s	sys 0.012s
W20260709 13:00:57.894197  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.634245  4908 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:57.706925  4860 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:57.726274  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 463) took 1106 ms. Trace:
I20260709 13:00:57.659201  4935 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:00:57.729851  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 553) took 1178 ms. Trace:
I20260709 13:00:57.710929  4854 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:57.675702  4932 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:57.725904  4941 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:57.742197  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 76269.1us
I20260709 13:00:57.631407  4974 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.744438  2351 raft_consensus.cc:606] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:57.489409  4962 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.763603  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:57.760553  1557 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:57.728770  4958 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:57.734647  4831 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:57.956638  4598 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:57.956794  4831 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.957278  4598 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.958096  4974 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:57.957973  4598 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.958441  4897 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:57.960588  4598 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:57.961561  4979 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.395s	user 0.001s	sys 0.000s
W20260709 13:00:57.962229  4727 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.593s	user 0.005s	sys 0.006s
I20260709 13:00:57.962656  4727 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:57.963419  4990 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:00:57.964797  4727 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:57.966315  4730 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.902s	user 0.007s	sys 0.003s
I20260709 13:00:57.769834  4981 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:00:57.724099  4864 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:57.966835  4730 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.968894  4864 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:57.729244  4940 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:57.786783  2352 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:57.759554  4767 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.787667  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:00:57.788419  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 118138us
I20260709 13:00:57.780077  4862 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.726996  4910 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.760627  4882 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.802707  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:57.804283  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:57.749101  4892 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:00:57.765851  4881 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:00:57.826072  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:57.820672  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:57.825591  1631 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:57.792510  4956 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.951205  1706 rpcz_store.cc:276] 0709 13:00:56.551027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.551168 (+   141us) service_pool.cc:224] Handling call
0709 13:00:56.551537 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:56.551796 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:56.954596 (+402800us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.721495 (+766899us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.729314 (+  7819us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.729467 (+   153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:57.729531 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:57.729587 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.729669 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.729791 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63972,"prepare.queue_time_us":858110,"prepare.run_cpu_time_us":165,"prepare.run_wall_time_us":2987,"raft.queue_time_us":76794,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":50155,"thread_start_us":1037206,"threads_started":3,"wal-append.queue_time_us":336763}
I20260709 13:00:57.824781  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:57.812638  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:57.988184  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:57.734910  4756 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.792158  4876 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:57.813139  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:57.716327  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:57.995513  1632 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:57.844616  4851 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:57.951828  2351 raft_consensus.cc:614] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:57.951951  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:57.952195  1556 rpcz_store.cc:276] 0709 13:00:56.620344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.621314 (+   970us) service_pool.cc:224] Handling call
0709 13:00:56.621763 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.622203 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.622245 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:56.622883 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 13:00:56.698268 (+ 75385us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:56.698325 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.698423 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.726068 (+1027645us) raft_consensus.cc:1685] finished
0709 13:00:57.726087 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.726216 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5789,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":13592,"thread_start_us":74907,"threads_started":1,"wal-append.queue_time_us":480549}
I20260709 13:00:57.956336  4958 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:57.969676  4974 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.771453  4427 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:57.792042  4836 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.141s
I20260709 13:00:57.794895  4416 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.978193  1707 raft_consensus.cc:2376] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7. Re-sending same reply.
I20260709 13:00:57.976661  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:57.988498  1629 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:57.988816  1636 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:58.051777  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:58.054409  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:58.050788  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.054674  2351 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:58.055243  1636 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:57.983379  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 195774us
I20260709 13:00:57.983579  1627 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 4.
I20260709 13:00:57.989696  1630 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
W20260709 13:00:58.053397  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 76845.3us
I20260709 13:00:58.056025  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:57.825755  1633 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:58.056439  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:57.995679  4840 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:57.990185  1628 raft_consensus.cc:2393] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:58.057067  1630 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:57.989753  1706 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:57.951282  2348 raft_consensus.cc:2376] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11. Re-sending same reply.
I20260709 13:00:58.058820  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:57.825163  4777 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:58.026304  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:58.028417  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 312156us
I20260709 13:00:58.063148  1632 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:58.028684  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:58.066179  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:57.934654  4933 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.847177  4982 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:57.846452  4978 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:00:58.029826  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:58.030862  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:57.992434  4956 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.071311  1631 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:00:58.033984  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:58.071969  1710 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:57.990978  4940 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:00:58.034415  1705 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:57.825930  4965 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.035032  2347 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:58.035710  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 236043us
I20260709 13:00:58.038898  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:58.039758  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 87910.9us
I20260709 13:00:58.085945  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:57.806639  4987 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.299s
I20260709 13:00:57.894982  4861 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:00:58.054271  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.056283  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:57.989846  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 548) took 1445 ms. Trace:
I20260709 13:00:58.097602  1703 rpcz_store.cc:276] 0709 13:00:56.544880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.545032 (+   152us) service_pool.cc:224] Handling call
0709 13:00:56.545430 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.545920 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:56.942901 (+396981us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b1800351700. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda2b0d23c7 00007fda276e49a7 00007fda276f40bf 00007fda276f4038
0709 13:00:57.075406 (+132505us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.792767 (+717361us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:57.793262 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:00:57.793396 (+   134us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:57.793439 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.793543 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.971209 (+177666us) raft_consensus.cc:1685] finished
0709 13:00:57.989661 (+ 18452us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.989788 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":826119,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":767,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":31473,"spinlock_wait_cycles":11177856,"thread_start_us":735465,"threads_started":2,"wal-append.queue_time_us":555030}
I20260709 13:00:58.059264  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:58.100193  2348 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:58.101018  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:58.101577  2348 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
W20260709 13:00:58.036079  4988 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:00:58.111387  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 958) took 1068 ms. Trace:
I20260709 13:00:58.111786  2349 rpcz_store.cc:276] 0709 13:00:57.042612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.042749 (+   137us) service_pool.cc:224] Handling call
0709 13:00:57.043149 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:57.328163 (+285014us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:57.328205 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:57.328798 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:00:57.643376 (+314578us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:57.765190 (+121814us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.786839 (+ 21649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.959993 (+173154us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b180038a8e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419
0709 13:00:57.960934 (+   941us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b540081d8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:57.960982 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:58.111149 (+150167us) raft_consensus.cc:1685] finished
0709 13:00:58.111168 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.111300 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27971,"prepare.queue_time_us":115024,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":21467,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":6332,"spinlock_wait_cycles":67342848,"thread_start_us":371496,"threads_started":2,"wal-append.queue_time_us":387979}
I20260709 13:00:58.048846  4974 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.114068  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:58.115186  4974 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:57.996606  4882 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.115690  4730 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.116109  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:58.063007  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:00:58.119529  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56614.3us
I20260709 13:00:58.119810  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:58.120559  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:58.120970  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:58.121618  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:00:58.122841  4493 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:58.123265  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:58.123587  4493 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:58.125365  4840 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:58.126048  1602 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:57.957253  4831 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.126623  4864 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:58.064862  4836 consensus_queue.cc:1243] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:58.127547  4864 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:58.127717  4836 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:58.128605  4864 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.130779  4730 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:58.131212  4730 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:58.132227  4892 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:58.134148  5000 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:58.033744  4981 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:58.135375  4981 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:58.068482  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:58.137347  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:58.138378  4970 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.138710  4844 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:58.139117  4844 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:58.139406  4985 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:58.139801  4985 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:00:58.068823  1701 raft_consensus.cc:2410] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:58.061714  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:58.141105  1633 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.040071  4767 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.071477  1708 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:58.028465  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:57.990924  4862 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.046977  4935 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.057543  4416 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.047435  4984 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:58.102380  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:57.970736  4962 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.102622  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:58.057859  4994 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:58.103804  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 47608.8us
I20260709 13:00:58.034608  4851 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.107905  4734 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.108332  4989 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:00:58.108511  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:58.109095  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:58.176995  2348 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:58.096666  4965 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.118578  4882 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.126515  4727 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:58.134055  4993 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:00:58.143365  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:58.063045  4756 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.068548  4777 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:57.991501  4877 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:58.150738  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:58.151706  2346 raft_consensus.cc:2368] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6.
I20260709 13:00:58.073513  4887 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:58.052850  4427 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:58.177268  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.176949  2352 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:00:58.203830  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.151696  4661 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.892858  4763 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:58.183063  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 95082.6us
I20260709 13:00:58.205978  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:58.160056  4817 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:58.212622  4531 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.162858  4954 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:58.213721  4954 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.214118  4954 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:58.191730  4996 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:00:58.187729  4916 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:00:58.183280  4917 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.216529  4917 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.216583  4730 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.218086  4803 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:58.179369  4727 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.219282  4727 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.220528  4661 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.196352  4932 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.183518  4861 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.223151  4861 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:00:58.226912  4427 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.227630  4763 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.227777  4851 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:57.893411  4963 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:00:58.087116  4933 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.096369  4854 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:00:58.140051  4844 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.190541  4997 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:58.247578  4997 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:58.248198  4997 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.190864  4975 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:58.250195  4975 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.250706  4975 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.160924  4493 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:00:58.165812  4984 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:58.169906  4952 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.207144  4870 raft_consensus.cc:3055] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:58.201798  4830 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:58.218847  4951 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:58.180729  4994 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:58.190783  4893 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.140560  4985 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.166793  4935 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.199709  4877 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:58.282498  4877 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.283061  4975 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.284236  4877 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:58.245539  4931 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.519s	user 0.007s	sys 0.011s
W20260709 13:00:58.285676  4931 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.719s	user 0.009s	sys 0.011s
I20260709 13:00:58.285789  4820 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:58.274518  4873 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.204439  4887 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:00:58.195500  4965 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.259601  4984 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.259747  4870 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:58.244617  4917 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.255262  4844 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.263748  4951 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:58.261492  4862 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.274518  4958 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.275457  4994 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.278483  4930 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.263371  4910 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:58.288884  4997 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.289994  4908 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.266459  4959 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:58.283095  4935 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.291116  4887 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.292079  4975 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:58.299751  4950 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.929s	user 0.014s	sys 0.020s
I20260709 13:00:58.301792  4985 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:58.308113  4950 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.014s	sys 0.021s
W20260709 13:00:58.305984  4992 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.402s	user 0.000s	sys 0.001s
I20260709 13:00:58.308205  4951 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.309314  4986 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.313354  4951 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.304746  4959 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.316431  4962 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.317603  4982 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.318035  4958 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.318734  4981 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.319082  4986 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.322810  4982 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:58.320863  4959 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.324188  4997 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.325505  4981 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.330955  4984 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.333513  4984 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.152405  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:58.167724  1629 raft_consensus.cc:2393] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:00:58.381697  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.382875  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:58.384749  1629 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4.
I20260709 13:00:58.165829  1627 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:58.387967  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:58.390861  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:58.396430  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2299) took 2380 ms. Trace:
I20260709 13:00:58.179728  1631 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:58.422083  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2387) took 3247 ms. Trace:
I20260709 13:00:58.422801  1635 rpcz_store.cc:276] 0709 13:00:55.174983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.165395 (+990412us) service_pool.cc:224] Handling call
0709 13:00:56.166018 (+   623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.261595 (+1095577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.421120 (+1159525us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001383d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:58.421763 (+   643us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2510597504}
I20260709 13:00:58.397122  1557 rpcz_store.cc:276] 0709 13:00:56.015752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.579697 (+563945us) service_pool.cc:224] Handling call
0709 13:00:56.580020 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.951184 (+1371164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.395884 (+444700us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400230854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:58.396205 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2714513792}
W20260709 13:00:58.448581  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 242527us
I20260709 13:00:58.176499  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:58.450306  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:58.451527  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 275003us
W20260709 13:00:58.197741  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.274518  4835 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:58.200470  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 49822.3us
I20260709 13:00:58.208251  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:58.229856  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:58.478456  2346 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.478803  2353 raft_consensus.cc:2368] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6.
I20260709 13:00:58.478528  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:58.184136  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:58.236689  4661 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.251929  4730 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.290210  4876 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:58.437361  1633 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:00:58.208256  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:58.685221  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:58.539706  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 62054.8us
I20260709 13:00:58.507809  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:58.726694  2346 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:58.685683  1703 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:58.611095  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:58.544628  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:58.851780  1628 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:00:58.852285  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.852645  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2197) took 1007 ms. Trace:
I20260709 13:00:58.851656  4862 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.853775  4862 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.857579  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:58.684208  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:58.858696  1631 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
W20260709 13:00:58.511799  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 359290us
W20260709 13:00:58.514488  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 370921us
I20260709 13:00:58.853060  1628 rpcz_store.cc:276] 0709 13:00:57.845127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.796202 (+951075us) service_pool.cc:224] Handling call
0709 13:00:58.843695 (+ 47493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:58.852346 (+  8651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.852563 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:58.671310  1527 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.024s	sys 0.009s
I20260709 13:00:58.720594  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:58.682469  4831 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.555s	user 0.017s	sys 0.002s
I20260709 13:00:58.870893  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:58.578032  4941 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.618s	user 0.008s	sys 0.003s
W20260709 13:00:58.804893  1525 leader_election.cc:336] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.634s (SENT)
I20260709 13:00:58.521903  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:58.874251  1527 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.025s	sys 0.015s
I20260709 13:00:58.562110  4982 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.478926  4997 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.514070  4975 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:58.512617  4984 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:00:58.562709  4994 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:58.895098  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.520689  4835 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:58.612601  4962 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:00:58.891348  2353 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 964) took 1837 ms (client timeout 1810 ms). Trace:
I20260709 13:00:58.875626  4862 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.690311  4870 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:58.905144  4727 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.685s	user 0.010s	sys 0.003s
W20260709 13:00:58.900282  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.901348  1634 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:58.901221  1628 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
W20260709 13:00:58.682483  2345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 961) took 1638 ms (client timeout 1897 ms). Trace:
I20260709 13:00:58.889216  4831 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.919286  4952 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:58.918398  1707 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:58.887193  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:58.898976  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:58.290316  4777 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.906051  1629 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:58.920866  1628 pending_rounds.cc:85] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 3
I20260709 13:00:58.925663  2347 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:58.920977  2353 rpcz_store.cc:269] 0709 13:00:57.053658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.081401 (+ 27743us) service_pool.cc:224] Handling call
0709 13:00:58.229696 (+1148295us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b5800003c98. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:00:58.881257 (+651561us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13300096}
W20260709 13:00:58.931041  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59042.4us
I20260709 13:00:58.912216  2352 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:58.909832  4930 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.936347  4930 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:58.938841  4763 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.711s	user 0.014s	sys 0.000s
I20260709 13:00:58.920145  4870 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:58.908421  4954 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.694s	user 0.006s	sys 0.009s
W20260709 13:00:58.923686  2345 rpcz_store.cc:269] 0709 13:00:57.044274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.046986 (+  2712us) service_pool.cc:224] Handling call
0709 13:00:58.681986 (+1635000us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b5400690854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:58.682236 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1326952832}
I20260709 13:00:58.946641  4876 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:00:58.925370  4952 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:00:58.949625  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.932819  1636 raft_consensus.cc:2376] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 4. Re-sending same reply.
I20260709 13:00:58.950054  2351 raft_consensus.cc:2410] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:00:58.926803  1628 pending_rounds.cc:107] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:00:58.958056  1628 raft_consensus.cc:2889] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:58.942586  1633 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:58.958840  1631 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:58.953063  2347 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:58.959272  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2199) took 1028 ms. Trace:
I20260709 13:00:58.920387  4727 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.913306  4984 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:58.913997  4975 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:00:58.957551  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:58.953429  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.960209  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:58.959785  1633 rpcz_store.cc:276] 0709 13:00:57.931238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.855813 (+924575us) service_pool.cc:224] Handling call
0709 13:00:58.892380 (+ 36567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:58.959038 (+ 66658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.959219 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:58.879253  4941 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.964531  1709 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:58.965878  4727 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:58.966029  4777 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:58.967638  4952 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:00:58.967613  4416 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.794s	user 0.004s	sys 0.008s
I20260709 13:00:58.969125  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:58.973584  4749 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.258s	user 0.012s	sys 0.008s
W20260709 13:00:58.973701  4756 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.782s	user 0.008s	sys 0.008s
W20260709 13:00:58.974619  4951 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.003s
I20260709 13:00:58.976346  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:58.956853  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 816301us
I20260709 13:00:58.939284  4930 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.920140  4962 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:58.983346  2346 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:58.908632  4887 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:58.979993  2347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "946eea23a2334d678895281941dd9926"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:60938
I20260709 13:00:58.980419  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:58.988612  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:58.977278  4933 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:58.930996  4982 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:58.992628  1631 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.994756  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:59.016944  4951 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:59.021024  1627 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:58.984649  4327 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:59.026153  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2324) took 1041 ms. Trace:
I20260709 13:00:58.962445  4954 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.030920  1709 rpcz_store.cc:276] 0709 13:00:57.985027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.985179 (+   152us) service_pool.cc:224] Handling call
0709 13:00:59.023437 (+1038258us) spinlock_profiling.cc:243] Waited 801 ms on lock 0x7b5400246b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:59.026070 (+  2633us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1841651456}
I20260709 13:00:59.034483  1702 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:59.035460  2351 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:59.035737  2345 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:59.036432  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 844) took 1234 ms. Trace:
I20260709 13:00:58.995776  4749 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:59.036487  4877 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.752s	user 0.008s	sys 0.007s
I20260709 13:00:59.020380  4952 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:59.040947  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:59.037377  2351 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:00:59.046401  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61904.7us
I20260709 13:00:59.043264  2347 raft_consensus.cc:606] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:59.053000  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 248269us
I20260709 13:00:59.057047  2347 raft_consensus.cc:614] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:59.057279  1705 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 6 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:00:59.026094  4958 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.059307  1634 raft_consensus.cc:2368] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7.
I20260709 13:00:59.062273  1633 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:59.064927  1629 raft_consensus.cc:2410] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:59.068706  1628 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:59.072660  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.073427  2346 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:58.989274  4416 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:59.069195  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:59.074302  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 978) took 1033 ms. Trace:
I20260709 13:00:59.060350  4951 consensus_queue.cc:1243] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:00:59.083683  4873 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.796s	user 0.007s	sys 0.008s
W20260709 13:00:59.085388  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.097640  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.080384  2346 rpcz_store.cc:276] 0709 13:00:58.040877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.037708 (+996831us) service_pool.cc:224] Handling call
0709 13:00:59.054549 (+ 16841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.073953 (+ 19404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.074235 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.089936  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:59.091516  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:59.086207  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:59.108807  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:59.093578  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:59.119640  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 65800.6us
I20260709 13:00:59.038264  2345 rpcz_store.cc:276] 0709 13:00:57.802377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.984730 (+1182353us) service_pool.cc:224] Handling call
0709 13:00:59.035442 (+ 50712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.036115 (+   673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.036345 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.121086  2352 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:59.094919  4835 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:59.095219  4860 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.093704  1708 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:59.121454  1634 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:59.122166  4767 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:59.123338  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 972) took 1296 ms. Trace:
W20260709 13:00:59.092267  4861 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.857s	user 0.007s	sys 0.009s
W20260709 13:00:59.094018  4956 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.940s	user 0.006s	sys 0.012s
I20260709 13:00:59.125092  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:58.992717  4876 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:59.092074  4959 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.764s	user 0.008s	sys 0.006s
W20260709 13:00:59.123911  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.130195  1631 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:59.130724  4860 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.133563  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 845) took 1101 ms. Trace:
I20260709 13:00:59.127959  4835 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:59.093847  4844 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:59.126335  4756 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:59.098621  4887 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:59.128837  2348 rpcz_store.cc:276] 0709 13:00:57.826648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.982472 (+1155824us) service_pool.cc:224] Handling call
0709 13:00:59.123261 (+140789us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.138458  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2333) took 1150 ms. Trace:
I20260709 13:00:59.121564  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:59.093739  4954 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:59.148408  2352 rpcz_store.cc:276] 0709 13:00:58.026428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.044694 (+1018266us) service_pool.cc:224] Handling call
0709 13:00:59.127365 (+ 82671us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.143657  4831 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:59.151121  1525 leader_election.cc:336] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.690s (SENT)
I20260709 13:00:59.125672  4958 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:59.139096  4767 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.143519  4956 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.174304  1634 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:59.168543   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:59.174494  1633 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:59.177397  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:59.183741  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.186777  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:59.170353  1703 rpcz_store.cc:276] 0709 13:00:57.987897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.099429 (+111532us) service_pool.cc:224] Handling call
0709 13:00:58.099696 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:58.879830 (+780134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.108861 (+229031us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b540014dcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:00:59.138361 (+ 29500us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":836601088}
I20260709 13:00:59.168748  4876 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:59.159243  4844 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:59.169932  4997 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:59.195987  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2400) took 1018 ms. Trace:
I20260709 13:00:59.092561  4984 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.209388  4777 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:59.210318  4777 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:59.212112  2349 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:59.159765  4756 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.199817  1631 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:59.215121  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2402) took 1044 ms. Trace:
I20260709 13:00:59.198834  1634 rpcz_store.cc:276] 0709 13:00:58.170155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.143100 (+972945us) service_pool.cc:224] Handling call
0709 13:00:59.174096 (+ 30996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.188789 (+ 14693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.189014 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.219656  1629 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:59.173750  4767 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.216102  1631 rpcz_store.cc:276] 0709 13:00:58.170532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.147976 (+977444us) service_pool.cc:224] Handling call
0709 13:00:59.194119 (+ 46143us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.214813 (+ 20694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.215053 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.226828  1627 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:00:59.213604  4844 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:59.230952  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 85720.6us
I20260709 13:00:59.172407  4959 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.221717  4756 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.223299  2317 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:59.237149  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2407) took 1065 ms. Trace:
I20260709 13:00:59.241783  1634 rpcz_store.cc:276] 0709 13:00:58.175865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.230206 (+1054341us) service_pool.cc:224] Handling call
0709 13:00:59.236560 (+  6354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.236786 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.236814 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:59.236860 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:59.236914 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.236969 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.237070 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.243664  1628 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:00:59.258580  1633 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:59.234136  4981 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.771s	user 0.006s	sys 0.009s
I20260709 13:00:59.267360  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.267417  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:00:59.268419  1557 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:59.270419  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 248714us
I20260709 13:00:59.271072  4777 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:59.271528  4777 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.271394  2317 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:59.272126  4777 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:59.273774  4327 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:59.269114  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:59.274801  4327 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:59.275738  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:59.277624  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.246284  5016 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
I20260709 13:00:59.278836  1634 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:59.281807  1627 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:00:59.279819  1636 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:59.259881  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:59.259476  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2404) took 1088 ms. Trace:
I20260709 13:00:59.282862  1633 rpcz_store.cc:276] 0709 13:00:58.170994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.221161 (+1050167us) service_pool.cc:224] Handling call
0709 13:00:59.258349 (+ 37188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.259157 (+   808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.259408 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.283875  4950 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:59.284164  4835 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:59.284615  4950 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:59.286031  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:59.286437  1629 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:00:59.286773  2346 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:59.287155  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2405) took 1115 ms. Trace:
I20260709 13:00:59.288153  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2408) took 1105 ms. Trace:
I20260709 13:00:59.288295  1629 rpcz_store.cc:276] 0709 13:00:58.171265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.231978 (+1060713us) service_pool.cc:224] Handling call
0709 13:00:59.250686 (+ 18708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.286896 (+ 36210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.287106 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.290136  2352 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:00:59.292429  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:59.264317  4941 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:59.298336  4820 consensus_meta.cc:287] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.012s	user 0.005s	sys 0.010s
I20260709 13:00:59.299389  2350 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:00:59.288684  1627 rpcz_store.cc:276] 0709 13:00:58.182674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.234142 (+1051468us) service_pool.cc:224] Handling call
0709 13:00:59.265955 (+ 31813us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.287906 (+ 21951us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.288098 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.301874  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:59.287232  1628 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:59.260775  4962 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:59.313876  2347 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:59.176324  4975 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:00:59.324280  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:59.323547  2350 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 969) took 1976 ms (client timeout 1963 ms). Trace:
I20260709 13:00:59.329157  1633 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:59.327250  2350 rpcz_store.cc:269] 0709 13:00:57.346944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.363890 (+ 16946us) service_pool.cc:224] Handling call
0709 13:00:59.302877 (+1938987us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b540061fb54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:59.323435 (+ 20558us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4412526976}
I20260709 13:00:59.307102  4416 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:59.306077  4862 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:00:59.341336  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 399160us
I20260709 13:00:59.342897  1629 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 6 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:00:59.333137   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:00:59.348939  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.346859  4994 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:59.370153  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:59.376890  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.381448  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.380998  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 136813us
W20260709 13:00:59.395478  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 523630us
I20260709 13:00:59.370253  4862 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:59.402051  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.408812  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.392915  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:00:59.414227  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 137215us
I20260709 13:00:59.385314  4982 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:59.419569  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.420622  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:59.421226  5020 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:00:59.422814  4756 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:59.424984  2344 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.882s	user 0.010s	sys 0.010s
I20260709 13:00:59.426047  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:59.426802  5023 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:59.427644  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2399) took 1257 ms. Trace:
I20260709 13:00:59.430307  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.430610  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.431758  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.431963  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.432976  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:59.433729  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.438550  4860 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:00:59.439379  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 72480.8us
I20260709 13:00:59.400151  4911 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.439630  1558 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:59.440044  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:59.440977  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.400017  4958 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:59.458999  1630 rpcz_store.cc:276] 0709 13:00:58.169958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.887020 (+717062us) service_pool.cc:224] Handling call
0709 13:00:58.916500 (+ 29480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:58.925659 (+  9159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:58.925693 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:58.989533 (+ 63840us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.331235 (+341702us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:59.331273 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.331334 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:59.427344 (+ 96010us) raft_consensus.cc:1685] finished
0709 13:00:59.427363 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.427552 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6233,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"thread_start_us":294099,"threads_started":1,"wal-append.queue_time_us":402696}
I20260709 13:00:59.473090  1557 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:00:59.465636  1554 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:59.465823  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:59.466580  4862 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:59.467319  4911 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.467497  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:59.472311  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:59.474028  1552 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:59.474087  1561 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:59.419112  4956 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:59.476048  1553 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:59.476491  1559 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:00:59.474496  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 48619.8us
I20260709 13:00:59.472247  4959 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:59.478366  1558 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:59.478734  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.458559  4416 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:59.496173  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:59.496269  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:59.495127  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.491292  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:59.501061  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:59.503029  1559 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.472996  2344 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
W20260709 13:00:59.506047  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.506620  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 164210us
I20260709 13:00:59.518476  2353 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:59.520864  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.521917  1702 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:00:59.529227  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.533739  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:59.535009  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.535717  1701 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:59.535786  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:59.536295  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.536808  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:59.537101  1710 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:00:59.537142  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.537811  5027 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:59.538005  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.538384  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:59.539127  1707 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.540231  4844 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:59.541828  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61907.1us
I20260709 13:00:59.543203  5018 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:59.543768  5018 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.543959  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.544235  5018 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:59.545794  1703 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:59.542083  4887 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.548558  1703 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:59.501004  4930 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.549158  4930 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.550819  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.551419  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:59.517074  4951 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:59.552596  1705 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:59.552657  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.553434  1557 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:00:59.555969  4835 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:00:59.561645  5019 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:00:59.562418  1629 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:59.565297  4959 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:59.565834  5024 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:00:59.566293  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:59.567479  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2196) took 1745 ms. Trace:
I20260709 13:00:59.567840  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:59.568739  1629 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:59.569931  4994 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:00:59.572286  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.565948  4959 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.567521  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:59.513833  4982 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:59.573710  1630 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:00:59.573834  4982 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:59.574844  4982 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.576076  4962 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:59.576462  4962 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:59.569976  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2336) took 1049 ms. Trace:
I20260709 13:00:59.578087  1704 rpcz_store.cc:276] 0709 13:00:58.520201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.535905 (+ 15704us) service_pool.cc:224] Handling call
0709 13:00:58.976849 (+440944us) spinlock_profiling.cc:243] Waited 7.66 ms on lock 0x7b5800155498. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5eee 0000000000375939 00007fda2c6291e0 00007fda2c6f20c4 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:00:58.979809 (+  2960us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:58.986528 (+  6719us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:58.986570 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.027882 (+ 41312us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.362505 (+334623us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:59.362552 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.362624 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:59.569536 (+206912us) raft_consensus.cc:1685] finished
0709 13:00:59.569555 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.569814 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":887,"raft.queue_time_us":91493,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":36991,"spinlock_wait_cycles":17607552,"thread_start_us":296148,"threads_started":1,"wal-append.queue_time_us":409710}
I20260709 13:00:59.581647  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:59.581745  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:59.582515  1552 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:00:59.582707  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:00:59.583395  5020 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:59.583946  5020 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:59.584658  4997 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:59.584940  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:59.585235  4861 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:00:59.585078  4997 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:59.586027  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.586166  4861 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:59.586705  4861 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:00:59.587157  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:59.587790  2345 raft_consensus.cc:2376] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7. Re-sending same reply.
I20260709 13:00:59.587663  4861 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.589361  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 971) took 1738 ms. Trace:
I20260709 13:00:59.589751  2353 rpcz_store.cc:276] 0709 13:00:57.803681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.019007 (+1215326us) service_pool.cc:224] Handling call
0709 13:00:59.506630 (+487623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.518838 (+ 12208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.542107 (+ 23269us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b540005bc50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:59.542263 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1042941312}
W20260709 13:00:59.590801  2344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 953) took 2550 ms (client timeout 1603 ms). Trace:
W20260709 13:00:59.591238  2344 rpcz_store.cc:269] 0709 13:00:57.040541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.046887 (+  6346us) service_pool.cc:224] Handling call
0709 13:00:59.590686 (+2543799us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.592442  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.593134  1555 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:59.548259  4887 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:59.538033  1709 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:00:59.586215  4997 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.567893  1635 rpcz_store.cc:276] 0709 13:00:57.821761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.613666 (+791905us) service_pool.cc:224] Handling call
0709 13:00:58.749516 (+135850us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:58.848164 (+ 98648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:58.848205 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:58.852738 (+  4533us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.245865 (+393127us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:59.246964 (+  1099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.247057 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:59.567258 (+320201us) raft_consensus.cc:1685] finished
0709 13:00:59.567274 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.567420 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14149,"raft.queue_time_us":38709,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":286,"thread_start_us":392669,"threads_started":1,"wal-append.queue_time_us":518398}
I20260709 13:00:59.560577  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.634119  1703 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:59.635748  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:59.623725  1561 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:59.617311  1553 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:00:59.612344  1559 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:00:59.639461  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:59.640074  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.642486  1706 raft_consensus.cc:2410] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:59.644145  1630 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:59.646104  1701 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:59.652987  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.653396  4844 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:59.729283  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:59.729954  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 78679us
I20260709 13:00:59.666393  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.737226  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:59.745119  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 325124us
I20260709 13:00:59.735117  1708 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:59.730370  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:59.739964  1705 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:00:59.821105  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:59.821882  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.822607  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:59.823577  2353 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:59.812745  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:00:59.825595  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 87570.8us
I20260709 13:00:59.838619  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:00:59.839401  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.838665  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:59.839403  1628 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:00:59.842918  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.666543  4861 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:59.843577  1709 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
W20260709 13:00:59.758512  1601 connection.cc:256] RPC call timeout handler was delayed by -2.3970908989999771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
W20260709 13:00:59.844400  1601 connection.cc:256] RPC call timeout handler was delayed by -2.3793038739999588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1653s for a response
W20260709 13:00:59.844696  1601 connection.cc:256] RPC call timeout handler was delayed by -2.3226038739999808s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 13:00:59.844950  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 330477us
I20260709 13:00:59.824281  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 974) took 1890 ms. Trace:
I20260709 13:00:59.839840  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:59.848697  1561 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:59.839699  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:59.851680  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:59.847373  2349 rpcz_store.cc:276] 0709 13:00:57.947014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.114875 (+167861us) service_pool.cc:224] Handling call
0709 13:00:59.732262 (+1617387us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540055ffd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:59.824226 (+ 91964us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432761728}
I20260709 13:00:59.577394  4962 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:59.912358  5023 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:00:59.912983  5023 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:00:59.914883  5023 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:59.924391  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:59.925244  1709 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:59.925923  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.858250  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:59.859337  2350 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:00:59.855036  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:59.860255  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.860980  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:59.928054  1704 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.862373  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:59.862921  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:00:59.868256  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.929169  1627 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:59.854071  4844 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:59.585084  5020 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.873579  5029 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:59.874634  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.941643  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:00:59.942194  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7673038739999924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23330000000000004s for a response
I20260709 13:00:59.942384  2346 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:59.876336  4959 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:00:59.878813  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:59.946215  1526 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:00:59.881754  4930 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:59.878181  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:00:59.882474  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:59.883823  5027 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:59.884037  4950 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:59.886919  4911 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:00:59.890683  4777 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.618s	user 0.011s	sys 0.002s
I20260709 13:00:59.838297  4876 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.894168  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:59.958428  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:00:59.899197  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.958846  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:59.959154  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.899689  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:00:59.959986  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:59.960330  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.960543  1628 raft_consensus.cc:2376] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7. Re-sending same reply.
I20260709 13:00:59.953169  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:59.961690  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:00:59.903748  1707 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:00:59.824836  4756 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:59.813025  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:59.908821  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:00:59.909528  4749 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:00:59.910629  4887 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:00:59.936986  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:59.937927  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 83688.6us
I20260709 13:00:59.939961  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:00:59.947311  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:00:59.947898  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.977962  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.978479  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.957773  4876 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:00:59.811937  4831 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.122s
I20260709 13:00:59.952621  4951 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:59.975071  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:00:59.949716  1706 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:59.953687  1633 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:00:59.955054  1701 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:59.964071  1705 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:59.962818  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:00:59.893261  1458 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 270) took 17195 ms (client timeout 15000 ms). Trace:
I20260709 13:00:59.934051  4844 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:00:59.981419  1708 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:59.993333  2347 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:59.994737  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:00:59.995361  1635 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:59.986121  1709 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:00:59.963728  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:59.957239  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:59.998378  1707 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:00:59.933367  2348 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:00:59.970804  4950 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:00.005240  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2394) took 1819 ms (client timeout 2138 ms). Trace:
I20260709 13:00:59.993717  2349 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:01:00.011795  1553 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:00:59.967543  4930 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.972779  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 614) took 1063 ms. Trace:
I20260709 13:01:00.016753  1631 rpcz_store.cc:276] 0709 13:00:58.909689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.298208 (+388519us) service_pool.cc:224] Handling call
0709 13:00:59.298615 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.318398 (+ 19783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.318435 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.328154 (+  9719us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.502542 (+174388us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:59.502581 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.502649 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:59.972551 (+469902us) raft_consensus.cc:1685] finished
0709 13:00:59.972576 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.972718 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16350,"raft.queue_time_us":23069,"raft.run_cpu_time_us":594,"raft.run_wall_time_us":71626,"thread_start_us":150510,"threads_started":1,"wal-append.queue_time_us":524523}
I20260709 13:00:59.973156  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:00:59.974442  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:00.019498  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.976466  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:00:59.978991  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.986905  4951 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.943939  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:00:59.967298  4958 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.987244  2351 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:00:59.982801  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:59.982434  2352 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:59.983189  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:00:59.991360  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:00.027426  1628 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 4.
I20260709 13:00:59.965327  5017 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:00.009459  1632 rpcz_store.cc:269] 0709 13:00:58.168528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.683510 (+514982us) service_pool.cc:224] Handling call
0709 13:00:59.988076 (+1304566us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400349ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:00:59.988175 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2490169600}
I20260709 13:01:00.015467  2350 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:00:59.957295  4777 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.023516  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:00.031589  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 62093.4us
I20260709 13:00:59.972458  1558 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:00.024315  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:00.005024  4844 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:00.024014  1557 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:01:00.026178  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:00.056764  2353 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:00.057742  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:00.057276  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:00.022496  2345 raft_consensus.cc:2410] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:00.060532  1631 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:00.062450  1553 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
W20260709 13:01:00.061560  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 152815us
I20260709 13:01:00.060809  1633 raft_consensus.cc:2368] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6.
I20260709 13:01:00.059194  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:00.034663  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:00:59.981752  4975 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:01:00.064556  1706 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:00.065053  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:00.027621  4911 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:00.069211  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 112065us
I20260709 13:01:00.065315  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:00.065732  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:00.069952  1628 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.010474  4860 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:00.066394  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:00.072515  2348 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:01:00.067328  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 121189us
I20260709 13:01:00.067134  1634 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:00.049942  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:00.066407  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 108509us
I20260709 13:01:00.031177  4962 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:00.070039  2350 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:01:00.058822  5027 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:00.013787  4997 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.031939  4951 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:00.084065  1561 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:00.113152  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:00.124075  1527 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:00.124296  1634 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:00.031052  4930 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.122973  1705 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:00.121551  1702 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:01:00.129594  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:00.130084  1559 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:00.112170  4831 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.145851  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:00.066073  5023 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
W20260709 13:01:00.150676  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 155751us
I20260709 13:01:00.152340  1707 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:01:00.056098  5017 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.153366  1633 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:00.121322  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:00.034137  4958 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.128388  4860 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.169653  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:00.176152  1561 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:00.182128  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:00.182337  1631 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:00.184051  1705 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:01:00.172722  1628 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:00.116655  1459 catalog_manager.cc:5697] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 4, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.134929  5022 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.080612  4911 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.190079  1552 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
W20260709 13:01:00.195001  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.191955  1552 pending_rounds.cc:85] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 5
I20260709 13:01:00.209022  1552 pending_rounds.cc:107] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:01:00.209379  1552 raft_consensus.cc:2889] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:00.147002  4756 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:01:00.211591  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:00.146605  4950 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:01:00.227366  4911 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:00.228256  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 259767us
W20260709 13:01:00.230140  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 212414us
W20260709 13:01:00.234776  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 117992us
I20260709 13:01:00.146828  5030 consensus_queue.cc:1059] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:01:00.240837  1704 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:00.240450  1701 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:00.246451  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:00.153802  4985 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.238s
W20260709 13:01:00.244194  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 78321.9us
I20260709 13:01:00.251111  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:00:59.990654  1458 rpcz_store.cc:269] 0709 13:00:42.698166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.785617 (+ 87451us) service_pool.cc:224] Handling call
0709 13:00:59.893016 (+17107399us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:55.081425 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:55.120750 (+ 39325us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:55.120826 (+    76us) write_op.cc:435] Acquired schema lock
0709 13:00:55.120858 (+    32us) tablet.cc:662] Decoding operations
0709 13:00:55.123346 (+  2488us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:55.123471 (+   125us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:55.123514 (+    43us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:00:55.144766 (+ 21252us) tablet.cc:701] Row locks acquired
0709 13:00:55.144821 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:00:55.145039 (+   218us) write_op.cc:270] Start()
0709 13:00:55.145215 (+   176us) write_op.cc:276] Timestamp: P: 1783602055144962 usec, L: 0
0709 13:00:55.145240 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:00:55.295114 (+149874us) log.cc:844] Serialized 6646 byte log entry
0709 13:00:56.570051 (+1274937us) op_driver.cc:464] REPLICATION: finished
0709 13:00:57.661746 (+1091695us) write_op.cc:301] APPLY: starting
0709 13:00:57.705282 (+ 43536us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:57.724767 (+ 19485us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:57.724971 (+   204us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:58.292907 (+567936us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:58.328248 (+ 35341us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:00:58.328329 (+    81us) write_op.cc:312] APPLY: finished
0709 13:00:58.329379 (+  1050us) log.cc:844] Serialized 113 byte log entry
0709 13:00:58.562956 (+233577us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:58.563456 (+   500us) write_op.cc:454] Released schema lock
0709 13:00:58.563702 (+   246us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:58.690078 (+126376us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:58.690122 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":1017776,"apply.run_cpu_time_us":17185,"apply.run_wall_time_us":1057765,"num_ops":16,"prepare.queue_time_us":191390,"prepare.run_cpu_time_us":53358,"prepare.run_wall_time_us":709248,"raft.queue_time_us":497035,"raft.run_cpu_time_us":65759,"raft.run_wall_time_us":476213,"replication_time_us":1424585,"thread_start_us":1297354,"threads_started":5,"wal-append.queue_time_us":1084727,"wal-append.run_cpu_time_us":35881,"wal-append.run_wall_time_us":2371134}]]}
I20260709 13:01:00.263863  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:00.264134  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.270603  2317 leader_election.cc:400] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7.
I20260709 13:01:00.271149  2317 leader_election.cc:403] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:01:00.241593  4835 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.685s	user 0.007s	sys 0.003s
I20260709 13:01:00.274035  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:00.275489  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:00.276098  1701 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:00.274417  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:00.238389  5043 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.283555  1702 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.204245  5022 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:00.285056  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:00.272401  4767 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.041s	user 0.019s	sys 0.001s
I20260709 13:01:00.291672  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:00.189129  4860 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.306550  1631 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:01:00.309688  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45881.8us
W20260709 13:01:00.320075  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 73750.1us
I20260709 13:01:00.317361  1459 catalog_manager.cc:5697] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 2 to 4, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.322911  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:00.325536  1561 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:00.290570  4911 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:00.336961  1527 leader_election.cc:400] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6.
I20260709 13:01:00.338505  1633 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
W20260709 13:01:00.337266  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 62946.5us
I20260709 13:01:00.339008  1527 leader_election.cc:403] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:01:00.339403  1675 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:00.341276  1674 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:00.342509  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.345316  1633 pending_rounds.cc:85] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 4
I20260709 13:01:00.345669  1633 pending_rounds.cc:107] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:01:00.345942  1633 raft_consensus.cc:2889] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:00.347692  5050 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:00.349968  5036 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:01:00.314261  4756 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:01:00.343708  5032 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
W20260709 13:01:00.373354  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 87052.2us
I20260709 13:01:00.340855  4911 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.345690  4835 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:00.310624  4767 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.351958  4777 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:00.399652  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 58460.3us
I20260709 13:01:00.381636  4911 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:00.401731  1602 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:00.402879  4975 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:00.403419  4975 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.403851  4975 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:00.379995  4982 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:00.398695  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:00.360731  4950 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.408784  1561 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:00.380976  4997 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:00.368485  5018 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.824s	user 0.011s	sys 0.000s
I20260709 13:01:00.410399  5018 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.411453  1707 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:01:00.417197  1602 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:00.398336  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:00.418361  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 81248.4us
I20260709 13:01:00.418865  1555 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:01:00.420840  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 211094us
W20260709 13:01:00.383008  5033 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
W20260709 13:01:00.425163  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.445135  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:00.380512  5049 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:00.446111  1709 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:00.418032  4952 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.360s	user 0.013s	sys 0.002s
I20260709 13:01:00.380206  5048 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:00.447965  1556 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:00.432924  5031 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.305s	user 0.001s	sys 0.000s
W20260709 13:01:00.453194  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 497424us
W20260709 13:01:00.457300  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 121217us
W20260709 13:01:00.444432  5035 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:01:00.456388  1630 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:00.460213  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:00.433296  4777 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:00.444464  5050 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:00.422026  4835 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:00.431516  4950 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.451371  5049 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:00.468410  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:00.471580  1631 pending_rounds.cc:85] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 4
I20260709 13:01:00.473950  1631 pending_rounds.cc:107] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:01:00.474166  1631 raft_consensus.cc:2889] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:00.475677  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:00.467332   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:00.444480  4959 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:01:00.483273  1709 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:00.484786  5063 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:00.485869  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:00.486128  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:00.486582  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.487470  2317 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:00.487541  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:00.496469  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:00.500144  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.512928  2352 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:01:00.513921  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.514503  1602 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:00.515211  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46876.1us
I20260709 13:01:00.516583  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:00.464620  4777 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.468924  5048 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:00.466260  5050 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:00.523023  1708 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.000s
I20260709 13:01:00.533598  1702 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:00.533895  1708 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:01:00.503295  1459 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 3 to 5, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:01:00.469183  4958 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:00.535434  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:00.525413  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:00.535265  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:00.476835  2344 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEARNER]: Advancing to term 7
I20260709 13:01:00.476369  5049 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:00.543423  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 73738.2us
I20260709 13:01:00.543462  1459 catalog_manager.cc:5697] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:00.508723  5018 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:00.548436  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.529140  5062 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:00.531311  4777 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:00.551494  1459 catalog_manager.cc:5697] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 3 to 4, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:00.555907  4756 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:00.537161  5050 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:00.461328  5054 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:01:00.520728  5034 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7.
I20260709 13:01:00.559257  2317 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:00.508358  4997 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 13:01:00.563155  5038 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:01:00.571504  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2211) took 1004 ms. Trace:
I20260709 13:01:00.549832  4416 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:01:00.569665  1635 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.004s
I20260709 13:01:00.574383  1635 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:00.553771  4861 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.483006  4950 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.590534  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:00.574100  1459 catalog_manager.cc:5697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:00.581804  5037 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 13:01:00.573513  1636 rpcz_store.cc:276] 0709 13:00:59.566890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.570129 (+  3239us) service_pool.cc:224] Handling call
0709 13:00:59.570443 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.570905 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.570934 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.571369 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.952035 (+380666us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:59.952074 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.952147 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:00.571278 (+619131us) raft_consensus.cc:1685] finished
0709 13:01:00.571301 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.571425 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1063,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":380300,"threads_started":1,"wal-append.queue_time_us":487292}
I20260709 13:01:00.585873  4860 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:00.586544  4756 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.587553  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:00.600409  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 70621.2us
I20260709 13:01:00.570639  5062 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:00.608008  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:00.613850  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 478967us
I20260709 13:01:00.618088  4861 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:00.610296  4835 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:00.621866  5072 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:01:00.623569  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:00.617930  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 58750us
I20260709 13:01:00.564508  4975 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:00.628022  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 111535us
I20260709 13:01:00.629146  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2325) took 1189 ms. Trace:
I20260709 13:01:00.563167  5046 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:01:00.609508  4416 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:00.646600  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:00.632896  1554 rpcz_store.cc:276] 0709 13:00:59.441739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.594984 (+153245us) service_pool.cc:224] Handling call
0709 13:00:59.595312 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:59.595797 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.595832 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:59.640811 (+ 44979us) log.cc:844] Serialized 52 byte log entry
0709 13:00:59.883707 (+242896us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b4800002958. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:01:00.064530 (+180823us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:00.064574 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.064644 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:00.624870 (+560226us) raft_consensus.cc:1685] finished
0709 13:01:00.624897 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.629056 (+  4159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34644,"raft.run_cpu_time_us":2709,"raft.run_wall_time_us":246496,"spinlock_wait_cycles":29802880,"thread_start_us":253650,"threads_started":1,"wal-append.queue_time_us":527402}
W20260709 13:01:00.647459  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 84547.7us
W20260709 13:01:00.641830  1706 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.001s
I20260709 13:01:00.612949  4767 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:00.648300  1706 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:01:00.650954  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:00.646263  5051 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.201s	user 0.001s	sys 0.001s
I20260709 13:01:00.638862  4861 raft_consensus.cc:2764] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:01:00.662110  5065 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:01:00.613613  4951 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:00.665225  4831 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.671906  5075 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:01:00.673913  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 50437.1us
I20260709 13:01:00.660993  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:00.661886  5061 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:00.664713  5062 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.652271  5048 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.645632  5059 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:00.696827  4767 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:00.697177  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:00.698645  5063 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6.
I20260709 13:01:00.698909  4975 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:00.710479  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:00.732231  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 85714.5us
W20260709 13:01:00.734503  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 119704us
I20260709 13:01:00.737778  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:00.737908  1554 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:00.738497  1560 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:00.657207  5070 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:00.712635  4982 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.712046  4951 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:00.710325  5022 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:00.735957  5075 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:00.758513  4958 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:00.779733  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:00.784905  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:00.787420  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 126558us
I20260709 13:01:00.774662  2344 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:00.776372  5072 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:00.798650  2318 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:00.803174  5059 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:00.801278  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 104210us
I20260709 13:01:00.805225  5059 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:00.807893  5076 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:00.806787  5085 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:00.809430  5076 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:00.811981  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:00.816280  5057 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.813553  5087 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:01:00.820648  1527 connection.cc:256] RPC call timeout handler was delayed by -1.3134905200000162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 13:01:00.813068  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 56610.1us
I20260709 13:01:00.799887  4958 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:01:00.822209  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:00.824054  4958 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:00.824324  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:00.833207  1554 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:01:00.833410  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:00.771613  5064 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:01:00.835107  1554 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:00.838186  5083 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:01:00.839030  5079 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEARNER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:00.841603  5071 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.844313  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:00.848141  5078 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:01:00.849658  5064 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:00.850081  5064 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:00.850337  5083 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:00.851274  4975 raft_consensus.cc:2764] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:01:00.851248  5083 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:00.845079  4950 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:00.853798  5083 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.855261  5083 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:00.856240  4844 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.739s	user 0.003s	sys 0.005s
I20260709 13:01:00.856660  4844 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.848830  4911 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.859383  5022 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:00.859959  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:00.860270  5022 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:00.861009  5046 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:00.861460  5018 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:00.861881  1557 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:01:00.862236  1557 pending_rounds.cc:85] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 9
I20260709 13:01:00.862432  5046 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:00.859380  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:00.862251  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:00.863355  5046 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:00.863405  1555 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:00.863479  1552 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:00.863879  5046 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:00.862510  1557 pending_rounds.cc:107] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:01:00.864765  1557 raft_consensus.cc:2889] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:00.850561  4958 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:00.874260  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:00.885807  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:00.801226  5072 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:00.865203  4959 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:00.890646  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:00.891170  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.774482  4416 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:00.861902  5018 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:00.899595  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:00.904394  5068 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:00.904464  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:00.905074  5068 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.905509  5068 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.906558  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:00.906224  5072 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:00.907166  1554 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:00.907747  1555 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:01:00.884176  4911 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:00.909446  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:00.910109  1558 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:01:00.912688  4975 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:00.913041  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:00.913458  4975 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:00.914702  4975 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:00.915097  4975 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:00.915663  1675 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:00.915973  4975 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:00.912726  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:00.903330  1552 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 4.
I20260709 13:01:00.919955  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:00.920398  1560 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:00.930269  1558 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
W20260709 13:01:00.931571  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 57413.2us
I20260709 13:01:00.896552  4835 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:00.895525  4959 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.922477  4749 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:00.941640  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:00.946916  4959 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:00.951419  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:00.952075  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:00.923947  4951 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:00.923784  4958 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.954468  1627 raft_consensus.cc:2368] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6.
W20260709 13:01:00.945426  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 158632us
I20260709 13:01:00.955575  1629 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:01:00.888114  5087 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
W20260709 13:01:00.959854  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 47244.6us
I20260709 13:01:00.944130  4861 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:00.970743  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:00.974201  1635 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:00.974732  1631 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:00.974942  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:00.976598  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 65875.5us
I20260709 13:01:00.977329  2317 leader_election.cc:400] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6.
I20260709 13:01:00.977638  2317 leader_election.cc:403] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:01:00.850881  5064 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:00.924150  4994 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:01:00.953733  5081 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:00.955775  4975 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:00.979727  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59961.2us
I20260709 13:01:00.968335  4835 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.955235  5070 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:00.979696  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:00.973830  4861 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.995246  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:00.991750  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:00.978104  4951 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:00.983458  5081 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.015187  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:01.012444  2317 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:01.021644  5084 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:01.022104  5084 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:01.022223  5056 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:01.023007  5056 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:01.002992  4861 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:01.024317  5056 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.026077  5064 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:01.027398  4930 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:00.996057  4835 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.029110  5044 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:01.029539  5044 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:01.029650  5093 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6.
I20260709 13:01:01.030175  4835 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:01.030413  5044 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.031795  5044 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:01.023259  5084 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.016203  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:01.074810  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:01.075922  1629 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
W20260709 13:01:01.074781  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 62450.3us
I20260709 13:01:00.931521  5075 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.038519  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:01.080395  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 97348.6us
I20260709 13:01:01.080542  5034 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:01.081394  4930 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:01.081655  5095 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:01:01.081979  2317 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:01.031821  4951 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.097632  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:01.098315  1629 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.090924  1558 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:00.990897  5054 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:01.083061  5084 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.102142  1560 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:01.009291  4860 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.017884  5081 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:01.113278  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 128013us
I20260709 13:01:00.980975  5018 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.094729  5056 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:01.132131  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.121155  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:01.129357  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:01.008111  5087 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 13:01:01.133658  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:01.134011  5102 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.134498  5102 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.134855  5102 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.137791  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:01.130615  1552 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:01.143810  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:01.144488  4997 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:01.144775  2350 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:01.145200  4997 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:01.146456  4997 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.104707  4887 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:01.153465  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:01.164690  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:01.118407  5093 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.165535  2352 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:01.118268  4860 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:01.170292  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:01.185165  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:01.185989  2348 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:01.150817  5102 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:01.175860  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.123661  4844 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:01.169418  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:01.129575  5072 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:01.104871  4930 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.156700  4887 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:01.157042  5054 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:01.189553  5074 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.245s	user 0.002s	sys 0.000s
W20260709 13:01:01.199994  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 99764.4us
W20260709 13:01:01.206686  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 190601us
I20260709 13:01:01.208186  1558 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:01.154131  4911 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:01:01.210951  1674 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:01.189036  4860 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.249308  1629 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:01.249310  1674 leader_election.cc:400] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6.
I20260709 13:01:01.250864  1674 leader_election.cc:403] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:01:01.184705  5022 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:01.253650  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.253791  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:01.255544  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:01.199193  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:01.255510  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 104713us
I20260709 13:01:01.259614  1558 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:01:01.260380  1676 leader_election.cc:336] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.528s (SENT)
I20260709 13:01:01.256425  2348 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:01:01.260207  1555 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:01.263818  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:01.210748  5054 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.268585  1558 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:01.269898  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:01.270531  1629 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:01.213660  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:01.284034  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:01.188846  5093 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.295589  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:01.302176  2353 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:01.272504  5054 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:01.301590  2346 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:01.280097  5034 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:01.158290  5070 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.303815  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:01.307940  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.312650  1558 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:01.315413  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:01.262933  5022 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.322969  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:01.304034  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:01.330484  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:01.334323  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:01.336503  2351 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:01:01.347549  1629 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:01.347882  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:01.343701  2344 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:01.356681  1555 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:01.326628  4876 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:01.367275  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:01.361306  2318 connection.cc:256] RPC call timeout handler was delayed by -1.1388342680000392s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
I20260709 13:01:01.302838  4887 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.378437  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:01.377020  2346 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:01.372452  5108 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:01:01.358645  5105 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:01.355840  5118 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.368140  5072 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:01.407109  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:01.410566  1555 raft_consensus.cc:2376] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6. Re-sending same reply.
W20260709 13:01:01.394409  4962 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.282s	user 0.015s	sys 0.000s
I20260709 13:01:01.401453  5105 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.443674  1552 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:01.445586  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:01.451187  1553 raft_consensus.cc:2376] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6. Re-sending same reply.
W20260709 13:01:01.453114  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:01.454682  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 170718us
W20260709 13:01:01.457581  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1802) took 1831 ms (client timeout 1949 ms). Trace:
W20260709 13:01:01.458254  1559 rpcz_store.cc:269] 0709 13:00:59.611949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.849252 (+237303us) service_pool.cc:224] Handling call
0709 13:01:01.442764 (+1593512us) spinlock_profiling.cc:243] Waited 46.8 ms on lock 0x7b54003048d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:01.442967 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107565952}
I20260709 13:01:01.422834  5103 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.422966  4930 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:01.415999  5112 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:01.421698  5102 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:01.356802  4749 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.472167  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:01.424860  5104 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6.
I20260709 13:01:01.443100  5109 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:01.491621  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 626) took 1192 ms. Trace:
I20260709 13:01:01.459857  4887 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:01.439500  5105 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:01.467651  5049 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.913s	user 0.013s	sys 0.000s
W20260709 13:01:01.518039  5110 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:01:01.518282  5072 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.504236  1633 rpcz_store.cc:276] 0709 13:01:00.279997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.309418 (+ 29421us) service_pool.cc:224] Handling call
0709 13:01:00.336992 (+ 27574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:00.346477 (+  9485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.471771 (+1125294us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b5400196754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:01.472061 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2447232}
I20260709 13:01:01.471022  5106 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.205s
I20260709 13:01:01.525722  5093 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:01.467012  5103 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:01.504595  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.531256  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:01.473237  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:01.537308  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 189524us
I20260709 13:01:01.538098  1559 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:01.489534  4930 raft_consensus.cc:2764] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:01:01.548367  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 76272.9us
I20260709 13:01:01.552522  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:01.531960  4844 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.517798  5049 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.560732  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:01.539296  5103 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.526468  5109 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.405433  5034 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.578063  5109 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:01.544787  5123 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:01.578712  4835 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:01:01.581867  5064 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.002s
I20260709 13:01:01.582610  4860 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:01.589457  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:01.641875  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 52523.2us
W20260709 13:01:01.590402  4861 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.566s	user 0.004s	sys 0.008s
W20260709 13:01:01.590560  5046 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.707s	user 0.013s	sys 0.000s
I20260709 13:01:01.590867  4844 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:01.592435  5018 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:01.644627  4959 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.003s
W20260709 13:01:01.596531  5111 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:01:01.604795  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2336) took 1234 ms. Trace:
I20260709 13:01:01.608351  1628 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:01.675653  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 624) took 1396 ms. Trace:
I20260709 13:01:01.676172  1628 rpcz_store.cc:276] 0709 13:01:00.279296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.607798 (+1328502us) service_pool.cc:224] Handling call
0709 13:01:01.608096 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.675338 (+ 67242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.675583 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:01.542397  5022 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:01.609282  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2347) took 1341 ms. Trace:
W20260709 13:01:01.558014  5050 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.987s	user 0.005s	sys 0.008s
W20260709 13:01:01.601001  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2315) took 2170 ms (client timeout 2517 ms). Trace:
I20260709 13:01:01.631084  2346 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:01.631605  1703 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:01.632418  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:01.638695  1709 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:01.645409  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 114239us
W20260709 13:01:01.646221  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:01.678519  1556 rpcz_store.cc:269] 0709 13:00:59.430086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.430276 (+   190us) service_pool.cc:224] Handling call
0709 13:01:01.600631 (+2170355us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b54003304d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:01.600864 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2238179072}
I20260709 13:01:01.645725  5121 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:01.635082  5123 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:01.595629  5120 log.cc:927] Time spent T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 13:01:01.643676  5068 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.738s	user 0.011s	sys 0.002s
I20260709 13:01:01.640779  4930 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.677399  1707 rpcz_store.cc:276] 0709 13:01:00.267875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.273740 (+  5865us) service_pool.cc:224] Handling call
0709 13:01:00.274123 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:00.411766 (+137643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.608957 (+1197191us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54002e86d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:01.609179 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315332352}
I20260709 13:01:01.675165  1561 rpcz_store.cc:276] 0709 13:01:00.370766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.370920 (+   154us) service_pool.cc:224] Handling call
0709 13:01:00.371758 (+   838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:00.409111 (+ 37353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.604481 (+1195370us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b54001ce154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:01.604716 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84378496}
W20260709 13:01:01.641496  4777 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.070s	user 0.005s	sys 0.008s
W20260709 13:01:01.684396  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 132010us
I20260709 13:01:01.630844  4962 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:01:01.664361  5046 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:01.688853  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:01.690008  5044 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.004s
I20260709 13:01:01.689895  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:01.705997  1559 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.691921  5049 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:01.681582  4994 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:01.686229  5084 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.566s	user 0.008s	sys 0.004s
W20260709 13:01:01.692408  5056 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.539s	user 0.008s	sys 0.003s
I20260709 13:01:01.706970  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:01.715238  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:01.716117  1556 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:01:01.692459  4975 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.691s	user 0.008s	sys 0.003s
I20260709 13:01:01.724296  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:01.654922  4861 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.687239  5050 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.701736  5123 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.725396  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:01.733247  1701 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:01.734865  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:01.738018  1706 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:01.717576  4958 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:01.683537  4959 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.751176  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:01.755050  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:01.758366  1628 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:01.757649  1705 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:01.704881  4777 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:01.760830  2347 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.006s	sys 0.017s
W20260709 13:01:01.761453  2347 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.006s	sys 0.017s
I20260709 13:01:01.771090  1559 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:01.736541  5121 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.706454  5068 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:01.775164  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 49845.2us
I20260709 13:01:01.791520  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:01.717729  4994 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:01.801620  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43769.3us
I20260709 13:01:01.757086  4844 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:01.759946  4975 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:01.816522  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:01.830287  1633 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:01:01.762768  5022 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:01:01.803227  5096 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:01.850034  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.865114  4861 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:01.859594  4887 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:01.866497  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:01.876693  5132 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:01:01.816679  4975 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:01.878531  5140 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:01:01.837455  5044 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.787997  5121 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.817864  5049 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.065s
W20260709 13:01:01.880903  5020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.433s	user 0.009s	sys 0.018s
I20260709 13:01:01.833071  5133 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 13:01:01.926250  4835 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:01.926654  5131 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:01:01.919991  4860 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:01.867204  5096 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:01.903703  4887 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:01.927855  5141 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:01:01.928220  5046 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:01.929085  5050 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:01.950996  4835 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.951937  5096 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.950287  4887 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:01.953377  5096 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:01.945546  5020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.501s	user 0.010s	sys 0.018s
W20260709 13:01:01.959721  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.930851  5068 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:01.932432  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 141037us
W20260709 13:01:01.937572  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 71191.9us
I20260709 13:01:01.967675  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:01.970616  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:01.904639  4975 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.996086  1527 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:01.942552  5027 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:02.008595  5113 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.013773  4958 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:02.016652  5054 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.691s	user 0.018s	sys 0.000s
I20260709 13:01:02.021337  1631 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:02.021728  5119 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:01:02.022929  5070 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:02.023348  5070 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:02.023518  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:02.024574  5070 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:01.867749  4844 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.028200  5070 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.029664  4835 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:02.025614  1631 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:01:02.025969  5136 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:01:02.026152  5054 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:02.041159  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 177716us
W20260709 13:01:02.049227  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44128.9us
I20260709 13:01:01.987213  5020 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:02.050340  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:02.001673  5050 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:02.053213  5105 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.506s	user 0.002s	sys 0.012s
I20260709 13:01:02.085742  5050 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:02.090157  5050 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.090092  5105 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:02.095696  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45510.7us
I20260709 13:01:02.034291  4844 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:02.101346  4958 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:02.102293  4958 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:02.103801  4958 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.104920  5054 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:02.106185  5097 log.cc:927] Time spent T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.136s	user 0.007s	sys 0.000s
W20260709 13:01:02.108892  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.111249  1631 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:02.106513  4777 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:01.994316  5075 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.088s
W20260709 13:01:02.119921  1601 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.949s (SENT) [suppressed 23 similar messages]
I20260709 13:01:02.082684  5027 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.134084  5126 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.134259  5105 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:02.134812  5126 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.009492  5113 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.086229  5151 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.144698  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1002) took 1002 ms. Trace:
I20260709 13:01:02.149240  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 628) took 1032 ms. Trace:
I20260709 13:01:02.149866  1628 rpcz_store.cc:276] 0709 13:01:01.116510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.955430 (+838920us) service_pool.cc:224] Handling call
0709 13:01:01.955896 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.956139 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:01.956509 (+   370us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.148729 (+192220us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:02.148817 (+    88us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:02.148882 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.148975 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.149145 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7692,"prepare.queue_time_us":2597,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":47,"raft.queue_time_us":16562,"thread_start_us":175402,"threads_started":1}
I20260709 13:01:02.146373  2347 rpcz_store.cc:276] 0709 13:01:01.142579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.142770 (+   191us) service_pool.cc:224] Handling call
0709 13:01:01.143064 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:01.143430 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.143455 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:01.143859 (+   404us) log.cc:844] Serialized 26 byte log entry
0709 13:01:01.792774 (+648915us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:01.792837 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.792935 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:02.144391 (+351456us) raft_consensus.cc:1685] finished
0709 13:01:02.144425 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.144616 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21301,"raft.queue_time_us":121349,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":9672,"thread_start_us":565854,"threads_started":1,"wal-append.queue_time_us":660310}
I20260709 13:01:02.164770  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:02.169345  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 199063us
I20260709 13:01:02.087879  5020 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.153420  4777 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:02.157442  5150 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.236419  5126 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:02.150470  5075 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:01:02.237378  1628 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:02.165866  5017 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:02.141814  5027 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.234313  4749 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:02.265102  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:02.274374  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:02.274230  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:02.278542  1559 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:02.247241  5081 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.091s	user 0.017s	sys 0.000s
I20260709 13:01:02.244527  5071 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:02.285879  1628 raft_consensus.cc:2955] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:02.281441  4749 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.297873  4749 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:02.297761  5071 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.300171  5071 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.296661  4962 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.243371  5151 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.295440  5027 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.303354  5113 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.304044  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2428) took 1225 ms. Trace:
I20260709 13:01:02.310770  1635 rpcz_store.cc:276] 0709 13:01:01.078894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.859822 (+780928us) service_pool.cc:224] Handling call
0709 13:01:01.888062 (+ 28240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:01.888884 (+   822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.888929 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:01.889806 (+   877us) log.cc:844] Serialized 26 byte log entry
0709 13:01:02.168359 (+278553us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:02.168398 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.168466 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:02.300865 (+132399us) raft_consensus.cc:1685] finished
0709 13:01:02.300891 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.303959 (+  3068us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10808,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":4138,"thread_start_us":231309,"threads_started":1,"wal-append.queue_time_us":389994}
I20260709 13:01:02.246145  5150 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:02.314464  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1765) took 2675 ms (client timeout 3351 ms). Trace:
W20260709 13:01:02.311810  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 46801.2us
I20260709 13:01:02.316622  1527 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:02.315361  1703 rpcz_store.cc:269] 0709 13:00:59.632819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.635453 (+  2634us) service_pool.cc:224] Handling call
0709 13:01:02.307879 (+2672426us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b54001e9454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:02.308026 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4137144064}
I20260709 13:01:02.322535  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:02.324266  5123 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.004s
I20260709 13:01:02.295504  5020 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:02.326437  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:02.327662  1555 raft_consensus.cc:2410] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:01:02.328577  5071 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:02.324903  5123 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.359941  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:02.361209  1555 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:02.307453  4962 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.367851  1631 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:01:02.371078  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 251202us
I20260709 13:01:02.322712  5161 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:02.308403  5081 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.404903  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:02.406136  1555 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:02.367498  5043 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:02.414934  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:02.417192  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:02.420919  2348 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:02.425033  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:02.427726  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:02.431140  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 106237us
I20260709 13:01:02.432974  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:02.408437  5068 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:02.438848  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:02.441942  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.443710  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2429) took 1337 ms. Trace:
I20260709 13:01:02.413201  5043 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:02.449141  1630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2214) took 2706 ms (client timeout 2155 ms). Trace:
W20260709 13:01:02.449687  1630 rpcz_store.cc:269] 0709 13:00:59.736841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.811535 (+ 74694us) service_pool.cc:224] Handling call
0709 13:01:02.443180 (+2631645us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b540010d6d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:02.443339 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1387401472}
I20260709 13:01:02.423396  4860 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:02.444311  1632 rpcz_store.cc:276] 0709 13:01:01.105967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.867346 (+761379us) service_pool.cc:224] Handling call
0709 13:01:01.876733 (+  9387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:01.877419 (+   686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.877450 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:01.878243 (+   793us) log.cc:844] Serialized 52 byte log entry
0709 13:01:02.074231 (+195988us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:02.074291 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.075858 (+  1567us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:02.440938 (+365080us) raft_consensus.cc:1685] finished
0709 13:01:02.440954 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.443541 (+  2587us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":198,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":318,"thread_start_us":168586,"threads_started":1,"wal-append.queue_time_us":292114}
W20260709 13:01:02.440786  5018 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.006s
I20260709 13:01:02.433933  5044 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:02.448923  5084 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.446974  5161 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:02.473294  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:02.473047  5027 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.477608  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:02.478329  5159 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:01:02.479208  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:02.469744  5084 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:02.490134  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.463043  5044 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.490824  5044 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.491690  4860 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.492507  4860 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.493664  5126 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.494218  5126 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.495882  4860 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:02.497597  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 67413us
I20260709 13:01:02.498194  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 473) took 1346 ms. Trace:
I20260709 13:01:02.500643  1527 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:02.503237  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:02.499627  1552 rpcz_store.cc:276] 0709 13:01:01.151894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.152383 (+   489us) service_pool.cc:224] Handling call
0709 13:01:01.443378 (+290995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.449872 (+  6494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.497957 (+1048085us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54003048d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:02.498123 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":630557568}
I20260709 13:01:02.506857  1555 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:02.506726  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:02.507750  1627 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.496323  5126 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:02.519624  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:02.520582  1635 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:02.458803  4958 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:01:02.448537  5068 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.526788  5068 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.518671  1632 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:02.471052  5161 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:02.527519  5161 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:02.511525  5164 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:01:02.488859  5084 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:02.517580  4975 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.512s	user 0.004s	sys 0.009s
I20260709 13:01:02.544106  1527 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:02.514168  5070 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.553689  5173 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:02.554275  5081 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.556176  5126 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.557716  5151 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.558077  5123 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:02.559792  4994 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:02.560770  5134 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.691s	user 0.003s	sys 0.022s
I20260709 13:01:02.560442  5043 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.561434  4416 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:02.564394  5030 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.566274  5150 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:02.567215  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:02.592312  5034 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.578954  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:02.579291  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:02.621837  1560 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.579455  5050 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.580008  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:02.581776  4962 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:02.585908  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:02.586634  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:02.628799  1552 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:02.628590  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:02.588302  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:02.582453  4749 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.630889  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:02.631789  1708 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:02.589768  4958 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:02.606860  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.633567  1702 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:02.607174  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.613909  4982 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.547806  5068 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:02.610728  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:02.639410  1701 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:02.617648  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.618652  1635 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:02.640007  1707 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:01:02.531543  5103 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.891s	user 0.012s	sys 0.004s
I20260709 13:01:02.620746  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.623263  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:02.624006  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 124943us
I20260709 13:01:02.642659  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:02.578980  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:02.643497  1636 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:02.629426  1556 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
W20260709 13:01:02.589591  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 56826.2us
I20260709 13:01:02.603919  5048 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.477645  5027 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.634531  1703 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:02.648463  1708 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:02.611724  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:02.618450  4959 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.649266  1553 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:02.615612  5023 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:02.575395  4975 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.592124  5070 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.635759  5043 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.649205  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.661963  1706 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:02.602353  5126 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.663516  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:02.664642  1707 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:02.639065  5068 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.660758  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:02.670866  1635 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:02.617025  5151 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:01:02.666301  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.672420  1701 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:02.653226  1560 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:02.623870  5030 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.655819  5123 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:02.677620  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.676079  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:02.680788  1701 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:02.681174  1560 raft_consensus.cc:2376] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7. Re-sending same reply.
I20260709 13:01:02.687078  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:02.688091  1635 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.627426  5034 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.695369  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2423) took 1718 ms. Trace:
I20260709 13:01:02.702227  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:02.703016  1554 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:02.703933  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:02.704991  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:02.706017  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.642755  4958 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.675117  5068 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:02.646144  4982 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.647842  5103 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.735178  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:02.736476  1704 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:02.657015  5048 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.725646  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 633) took 1027 ms. Trace:
I20260709 13:01:02.778890  1631 rpcz_store.cc:276] 0709 13:01:01.698449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.136394 (+437945us) service_pool.cc:224] Handling call
0709 13:01:02.136863 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.372858 (+235995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.725308 (+352450us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54001b7454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:02.725574 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530377088}
I20260709 13:01:02.698345  1629 rpcz_store.cc:276] 0709 13:01:00.976645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.348669 (+372024us) service_pool.cc:224] Handling call
0709 13:01:01.368794 (+ 20125us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:01.490018 (+121224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.490089 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:01.500671 (+ 10582us) log.cc:844] Serialized 26 byte log entry
0709 13:01:01.582425 (+ 81754us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:01.582477 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.582557 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:02.550328 (+967771us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:02.676170 (+125842us) raft_consensus.cc:1685] finished
0709 13:01:02.676191 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.695297 (+ 19106us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121863,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":15756,"spinlock_wait_cycles":84560256,"thread_start_us":144649,"threads_started":2,"wal-append.queue_time_us":231565}
I20260709 13:01:02.730015  1636 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:02.733372  1708 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:02.781697  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.737501  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.782775  1706 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:02.708251  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:02.662417  5023 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.694370  4959 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:02.745923  4962 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.746563  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.791015  1704 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:02.792433  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:02.790905  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:02.748782  5034 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.749811  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.750325  4975 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:02.751744  5142 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.708s	user 0.001s	sys 0.010s
I20260709 13:01:02.754984  5050 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.813175  1635 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:02.836192  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 862) took 1212 ms. Trace:
I20260709 13:01:02.755277  5121 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:02.711971  5123 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:02.847105  5123 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:02.758908  5128 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.069s	user 0.010s	sys 0.005s
I20260709 13:01:02.757117  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:02.851738  4911 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.852116  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:02.852790  1629 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.852562  4911 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.854314  1558 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:01:02.854749  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.854950  4911 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.855073  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1001) took 1713 ms. Trace:
I20260709 13:01:02.759251  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2424) took 1778 ms. Trace:
I20260709 13:01:02.855616  4911 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:02.856034  5126 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.760663  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 570) took 1489 ms. Trace:
I20260709 13:01:02.762964  5027 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.856983  5172 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:02.766146  1556 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:02.856163  1633 rpcz_store.cc:276] 0709 13:01:00.981003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.519962 (+538959us) service_pool.cc:224] Handling call
0709 13:01:01.520517 (+   555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:01.834858 (+314341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.758976 (+924118us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b5400198054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:02.759187 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706968960}
I20260709 13:01:02.768662  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:02.858956  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:02.859323  1556 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:02.859606  1633 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:02.860014  2353 raft_consensus.cc:2410] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:02.860533  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:02.769827  2347 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:02.769984  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:02.861933  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 71116.9us
I20260709 13:01:02.862500  4958 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.863554  1525 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:02.863977  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:02.770671  5084 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:02.773389  5103 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.776027  5044 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.776886  1632 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:02.777101  1555 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:01:02.814082  5174 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:01:02.733268  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:02.783530  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 75354.7us
I20260709 13:01:02.894003  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:02.855530  2349 rpcz_store.cc:276] 0709 13:01:01.142092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.142237 (+   145us) service_pool.cc:224] Handling call
0709 13:01:01.142581 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:01.325964 (+183383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.326011 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:01.345562 (+ 19551us) log.cc:844] Serialized 26 byte log entry
0709 13:01:01.540179 (+194617us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:01.540246 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.482887 (+942641us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b5400762d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:02.482947 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:02.749283 (+266336us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:02.854842 (+105559us) raft_consensus.cc:1685] finished
0709 13:01:02.854867 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.854999 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8578,"raft.queue_time_us":78,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":20653,"spinlock_wait_cycles":416052992,"thread_start_us":143402,"threads_started":1,"wal-append.queue_time_us":252239}
I20260709 13:01:02.786985  1707 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:02.788219  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:02.897789  1629 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:02.899502  1702 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:02.743939  1554 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:02.694793  5081 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:02.901942  5081 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.795387  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:02.906148  1631 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:02.801704  1703 raft_consensus.cc:2410] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:01:02.797910  4959 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:02.836828  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.834241  5030 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:02.910987  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 199471us
I20260709 13:01:02.911326  1705 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:02.836614  2350 rpcz_store.cc:276] 0709 13:01:01.624017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.625818 (+  1801us) service_pool.cc:224] Handling call
0709 13:01:01.633280 (+  7462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:01.634524 (+  1244us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:01.634835 (+   311us) log.cc:844] Serialized 14 byte log entry
0709 13:01:01.928047 (+293212us) spinlock_profiling.cc:243] Waited 5.65 ms on lock 0x7b18003452e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda2b5ae58f 00007fda245ed571 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd
0709 13:01:01.934715 (+  6668us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:01.945168 (+ 10453us) log.cc:844] Serialized 26 byte log entry
0709 13:01:01.945546 (+   378us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:01.945677 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.945870 (+   193us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:01.952867 (+  6997us) spinlock_profiling.cc:243] Waited 9.96 ms on lock 0x7b18003452e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1
0709 13:01:01.953413 (+   546us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b18003452e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:02.750993 (+797580us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:02.835939 (+ 84946us) raft_consensus.cc:1685] finished
0709 13:01:02.835965 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.836124 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":187412,"prepare.queue_time_us":2993,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":183991,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":11708,"spinlock_wait_cycles":52691200,"thread_start_us":112736,"threads_started":1,"wal-append.queue_time_us":266850}
I20260709 13:01:02.856640  4911 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:02.767975  2345 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:02.769659  5070 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.855974  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.914918  1706 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:02.725029  5017 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:01:02.916173  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:02.854772  1558 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 5
I20260709 13:01:02.867817  2317 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:02.865430  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:02.917315  1558 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.6
I20260709 13:01:02.793532  5054 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:02.869935  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:02.781559  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:02.920789  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 53111.6us
I20260709 13:01:02.921289  1708 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.884248  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:02.911856  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:02.856914  2348 rpcz_store.cc:276] 0709 13:01:01.271694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.271863 (+   169us) service_pool.cc:224] Handling call
0709 13:01:01.272409 (+   546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.421673 (+1149264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.760352 (+338679us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400756554. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:02.760558 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2640380672}
I20260709 13:01:02.869974  4994 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.918292  1630 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:02.918932  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:02.889629  5121 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:02.917855  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52541us
I20260709 13:01:02.933769  5121 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.934237  5121 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.935352  5044 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:02.937414  1558 raft_consensus.cc:2917] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Aborting config change with OpId 6.6: Aborted: Op aborted by new leader
I20260709 13:01:02.926224  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:02.930529  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:02.798934  4962 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.942482  1558 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:02.946209  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:02.947712  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:02.946276  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:02.949725  1635 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:02.905076  5081 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:02.955570  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:02.957844  1627 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:02.958319  1708 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:02.915707  4911 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:02.958964  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:02.958931  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 239069us
I20260709 13:01:02.960409  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:02.961839  1627 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:02.920485  5054 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.964986  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:02.939322  5126 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:01:02.972235  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.972540  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:02.972831  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.972946  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:02.973279  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.982864  1602 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:02.984130  1602 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:02.969844  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.939607  5018 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:02.939105  4975 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:02.933792  4982 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:02.940996  5103 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:02.973423  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:02.967581  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:02.990109  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:02.965545  5044 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:02.970890  5050 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:02.973901  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:03.014181  1702 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.986047  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:03.015579  2350 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:02.986246  1708 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:02.945828  5151 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:02.949725  5084 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:03.021687  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:03.023175  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:03.023399  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:02.945946  5119 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.946058  4994 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:02.990337  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:02.994262  5103 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.917390  5161 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.998726  4911 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.978186  5059 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:01:02.973752  1627 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:03.145535  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:03.146711  1627 raft_consensus.cc:2410] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:01:03.147792  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:03.148586  1627 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:02.999634  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:02.967480  5027 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:02.971913  5054 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.002888  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:03.174068  1706 raft_consensus.cc:2376] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7. Re-sending same reply.
W20260709 13:01:03.006861  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.973698  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:02.971169  5175 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.980187  5179 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:02.938472  5170 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:01:02.980762  5101 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.987191  5018 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.957734  5061 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:01:02.990108  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:02.990476  5109 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:02.992277  5171 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:03.031488  1628 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:02.957406  4958 raft_consensus.cc:2764] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:01:03.033036  5119 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:02.992158  5182 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:03.000758  5044 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:02.960750  4749 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:03.008800  4777 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:03.042469  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:03.221388  1629 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:03.042801  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:03.044126  5196 raft_consensus.cc:670] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:03.013952  5050 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.994786  1705 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:03.040758  1702 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:03.203148  5171 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.151723  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:03.152064  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:03.187148  5070 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 13:01:03.155876  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.235909  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:03.160437  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:03.236930  1628 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:01:03.163157  5173 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:03.164510  5084 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:03.167789  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2224) took 3257 ms. Trace:
I20260709 13:01:03.154261  4959 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:03.154348  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:03.244231  1627 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:03.026523  5029 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.176066  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:03.042119  5103 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.245311  2349 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:03.228551  5050 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.180788  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:03.183760  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 210114us
I20260709 13:01:03.177924  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:03.189260  4860 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:03.187386  5018 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.218843  2353 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:03.186080  5197 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.192842  5017 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.193650  5048 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:03.150240  2345 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.186412  5030 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:01:03.201653  5198 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.153466  1701 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:03.241405  5173 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:03.233671  5195 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:03.266554  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:03.182061  5175 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.243865  1634 rpcz_store.cc:276] 0709 13:00:59.910530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.134187 (+223657us) service_pool.cc:224] Handling call
0709 13:01:00.160001 (+ 25814us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:00.179564 (+ 19563us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:00.376482 (+196918us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.444752 (+2068270us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b1800459ca0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:01:02.505851 (+ 61099us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:02.525456 (+ 19605us) log.cc:844] Serialized 52 byte log entry
0709 13:01:02.525672 (+   216us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:02.902259 (+376587us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.919531 (+ 17272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.161398 (+241867us) spinlock_profiling.cc:243] Waited 7.31 ms on lock 0x7b5400197b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:03.167261 (+  5863us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:03.167401 (+   140us) raft_consensus.cc:1685] finished
0709 13:01:03.167423 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:03.167695 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49612,"prepare.queue_time_us":446266,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":110,"raft.queue_time_us":208875,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":30365,"spinlock_wait_cycles":21827072,"thread_start_us":570892,"threads_started":3,"wal-append.queue_time_us":2164787}
I20260709 13:01:03.220758  4777 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:03.237267  1632 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:03.242976  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:03.246021  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:03.249086  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:03.275766  1628 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:03.276137  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:03.185392  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 192292us
I20260709 13:01:03.188860  5101 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:03.228367  5130 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.367s	user 0.010s	sys 0.020s
I20260709 13:01:03.258178  1627 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:03.232364  5171 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.261760  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2354) took 2739 ms. Trace:
W20260709 13:01:03.232764  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 81151.2us
I20260709 13:01:03.312799  1709 rpcz_store.cc:276] 0709 13:01:00.518812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.527744 (+  8932us) service_pool.cc:224] Handling call
0709 13:01:00.540565 (+ 12821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.688925 (+1148360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.257277 (+1568352us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400078354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:03.257517 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2524856064}
W20260709 13:01:03.264828  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 86983us
I20260709 13:01:03.217516  5044 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.271549  5175 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:03.258282  5161 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:03.337949  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 61774.6us
I20260709 13:01:03.339877  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:03.341374  1629 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:03.253366  5029 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.268253  5198 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.282646  4975 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:03.278157  4777 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:03.285933  5169 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.674s	user 0.006s	sys 0.006s
I20260709 13:01:03.286854  1602 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:03.262187  4962 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:03.288105  5172 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:03.288655  5205 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:03.292325  1635 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:03.292917  2350 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:03.262198  5017 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.481035  5209 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.481734  5209 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.483681  5017 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:03.298709  5050 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.484264  5209 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:03.300836  5194 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:01:03.487579  1628 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:03.301919  4860 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:03.489013  4860 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:03.490547  4860 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:03.304461  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.309998  2345 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:03.311020  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:03.315500  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:03.316284  1634 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:03.313380  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:03.323976  1632 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:03.268586  5195 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:03.327872  5130 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.459s	user 0.010s	sys 0.020s
I20260709 13:01:03.317119  5154 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.350423  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:03.369215  4975 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:03.348244  5199 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:03.352989  5179 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.349367  5109 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:01:03.352118  5084 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.509214  1627 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:01:03.510372  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:03.511116  1702 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
W20260709 13:01:03.512310  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 196862us
I20260709 13:01:03.351483  4958 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:01:03.349195  5175 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:03.259971  5197 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:03.474092  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:03.520995  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 207701us
I20260709 13:01:03.520267  1701 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:03.367535  5173 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.513597  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:03.468766  5068 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.678s	user 0.013s	sys 0.007s
I20260709 13:01:03.530707  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:03.540498  1709 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:03.541661  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:03.546836  1701 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:03.544763  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:03.479842  5103 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.495409  5050 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:03.554811  5068 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.614984  5197 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:03.615864  1706 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:03.616377  5068 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:03.611733  5061 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:01:03.616981  2345 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:01:03.617388  2345 pending_rounds.cc:85] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 6
I20260709 13:01:03.617638  2345 pending_rounds.cc:107] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:01:03.617583  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:03.618086  2345 raft_consensus.cc:2889] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:03.618367  1709 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:03.522145  5154 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.619225  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:03.620024  2345 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:03.620112  2350 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:03.613178  5048 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:03.627532  5048 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:03.628290  5048 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:03.631510  5121 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.697s	user 0.003s	sys 0.014s
I20260709 13:01:03.632217  5121 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.521140  4959 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:03.612927  5207 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:01:03.613567  5101 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:03.612440  5029 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:03.529844  5175 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.520087  5179 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.612978  2348 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:03.543000  5205 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:01:03.625910  5056 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:03.651629  1634 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:03.652240  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 365499us
I20260709 13:01:03.633652  1708 raft_consensus.cc:2368] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8.
I20260709 13:01:03.636924  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:03.646764  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2435) took 1146 ms. Trace:
I20260709 13:01:03.654673  1630 rpcz_store.cc:276] 0709 13:01:02.499119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.512312 (+ 13193us) service_pool.cc:224] Handling call
0709 13:01:02.528728 (+ 16416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.932785 (+404057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.645132 (+712347us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b54003b92d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:03.645429 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":895272320}
I20260709 13:01:03.641271  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:03.662654  1709 raft_consensus.cc:2410] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:01:03.577963  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:03.664266  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.517453  5195 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:03.600080  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45820.1us
I20260709 13:01:03.646958  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2357) took 1398 ms. Trace:
I20260709 13:01:03.666504  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:03.666119  1561 rpcz_store.cc:276] 0709 13:01:02.248688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.248817 (+   129us) service_pool.cc:224] Handling call
0709 13:01:02.249713 (+   896us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.277635 (+ 27922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:02.277687 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:02.282976 (+  5289us) log.cc:844] Serialized 26 byte log entry
0709 13:01:02.435179 (+152203us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:02.436835 (+  1656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.436923 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:03.285192 (+848269us) spinlock_profiling.cc:243] Waited 7.97 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:03.646658 (+361466us) raft_consensus.cc:1685] finished
0709 13:01:03.646683 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:03.646895 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52075,"raft.queue_time_us":31222,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":25998,"spinlock_wait_cycles":18341632,"thread_start_us":118238,"threads_started":1,"wal-append.queue_time_us":215966}
I20260709 13:01:03.668879  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:03.641948  5109 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:03.673082  5101 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:03.675415  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:03.542582  5133 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:03.681658  2345 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:03.493289  5018 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:03.685465  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.686810  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:03.692557  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:03.694783  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:03.643203  5175 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.555078  5150 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:03.708408  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 358027us
I20260709 13:01:03.711015  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:03.643486  5103 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:03.645578  4975 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:03.725562  1634 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:03.640763  5121 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:03.545516  5161 consensus_queue.cc:1059] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.145s
W20260709 13:01:03.734236  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54748.8us
I20260709 13:01:03.724540  5056 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.730911  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:03.651057  5050 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:03.542721  5203 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.176s
I20260709 13:01:03.680586  5109 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.742046  1630 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:01:03.647182  4982 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:01:03.756291  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:03.547850  5017 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.179s
I20260709 13:01:03.706146  5173 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.706020  5179 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:03.752768  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:03.757669  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:03.754405  1634 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:03.725514  5150 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.759586  2348 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:03.761003  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:03.766891  1674 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:03.768422  2349 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.000s
I20260709 13:01:03.768957  2349 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:01:03.769680  2350 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:03.794816  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:03.760149  5109 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.807906  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:03.791697  5182 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.563s	user 0.012s	sys 0.000s
I20260709 13:01:03.766669  5150 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.820711  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:03.787097  5101 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:03.765133  5161 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:03.732880  4975 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.824563  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 640) took 1035 ms. Trace:
I20260709 13:01:03.793236  5133 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:03.755579  5050 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.814167  5197 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:03.789069  5198 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:03.752184  5056 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.836300  1627 rpcz_store.cc:276] 0709 13:01:02.789158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.510082 (+720924us) service_pool.cc:224] Handling call
0709 13:01:03.534903 (+ 24821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:03.819595 (+284692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:03.819645 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:03.824129 (+  4484us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:03.824205 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.824311 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:03.824481 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14891,"raft.queue_time_us":231857,"spinlock_wait_cycles":15348096,"thread_start_us":268945,"threads_started":1}
I20260709 13:01:03.846732  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:03.863940  2345 raft_consensus.cc:2410] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:03.830472  5044 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.822274  5182 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:03.877521  5205 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:03.878749  5205 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:03.879890  5205 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:03.880515  5023 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:03.797868  5020 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:01:03.881775  5103 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:03.881989  5020 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:03.882283  5103 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:03.882622  5020 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:03.883450  5103 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.883826  5020 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.885131  5103 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:03.824925  5217 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:03.889585  5217 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:03.889794  4975 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:03.890358  4975 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.839560  5101 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:03.890769  5217 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.892112  5101 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.835958  5154 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:03.894930  4777 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.535s	user 0.014s	sys 0.000s
I20260709 13:01:03.876410  5044 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:03.810472  5179 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:01:03.895624  4777 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.897730  5179 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:01:03.899083  5171 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.564s	user 0.013s	sys 0.000s
I20260709 13:01:03.885983  1628 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:03.899513  5171 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.885681  5201 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:03.880960  5205 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:03.901615  5201 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:03.902900  5201 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.904832  5201 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:03.886744  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:03.850241  5145 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.877630  5182 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.840051  4962 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:03.926308  4962 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.926956  4962 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.896001  5044 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:03.907004  5182 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.887404  5020 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.888456  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:03.934232  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.934919  5084 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.845276  5215 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:03.935662  5215 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:03.935544  5084 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:03.936180  5215 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.936786  5205 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:03.862353  5133 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:03.922432  5018 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:03.840289  5161 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:03.924937  5054 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.740s	user 0.009s	sys 0.003s
I20260709 13:01:03.939043  5133 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.939939  5054 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.941570  1631 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:03.948671  5034 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:03.949378  5034 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:03.952703  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 583) took 1255 ms. Trace:
I20260709 13:01:03.959522  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 646) took 1032 ms. Trace:
I20260709 13:01:03.960142  1635 rpcz_store.cc:276] 0709 13:01:02.927333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.968704 (+ 41371us) service_pool.cc:224] Handling call
0709 13:01:03.959203 (+990499us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400197b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:03.959429 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238243712}
I20260709 13:01:03.966722  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1798) took 1010 ms. Trace:
I20260709 13:01:03.967382  1706 rpcz_store.cc:276] 0709 13:01:02.956657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.187700 (+231043us) service_pool.cc:224] Handling call
0709 13:01:03.188073 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:03.616248 (+428175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.966330 (+350082us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54002a76d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:03.966631 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":982957440}
W20260709 13:01:03.980029  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 91634.7us
I20260709 13:01:03.953302  1707 rpcz_store.cc:276] 0709 13:01:02.697708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.698235 (+   527us) service_pool.cc:224] Handling call
0709 13:01:02.702072 (+  3837us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:02.897326 (+195254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.952304 (+1054978us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b540014d2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:03.952614 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174103296}
I20260709 13:01:03.985666  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:03.929075  5044 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:03.995528  2318 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:03.996089  5103 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.016129  2318 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:03.906080  5145 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.030917  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:04.006287  5050 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:03.890848  4975 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:03.973560  5054 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:04.015038  5103 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:04.063699  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:04.025640  5161 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:04.038514  5205 raft_consensus.cc:2764] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:01:04.071405  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 343028us
I20260709 13:01:04.043685  5203 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:04.053557  4777 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:04.092278  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:04.096997  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 315412us
I20260709 13:01:04.066952  5044 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.074456  5070 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.081564  5029 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.120714  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.068353  5209 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:04.072044  5056 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.077766  5216 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:04.147217  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 339391us
I20260709 13:01:04.019819  5034 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.104202  5054 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:04.147020  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 379089us
I20260709 13:01:04.149271  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:04.115172  5182 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.159047  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:04.168610  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.149369  5084 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:04.206441  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.206703  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.214636  1635 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:04.202488  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.174479  4958 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:04.211103  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.161278  5018 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.176002  5034 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:04.215811  1630 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:04.136376  5205 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 13:01:04.217542  1629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.000s	sys 0.022s
I20260709 13:01:04.206547  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:04.227258  1634 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:04.227787  1629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.000s	sys 0.022s
I20260709 13:01:04.150453  5029 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:04.207213  5048 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.578s	user 0.011s	sys 0.004s
I20260709 13:01:04.234951  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:04.242719  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 171239us
W20260709 13:01:04.221696  4860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.012s	sys 0.012s
I20260709 13:01:04.245735  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.174005  5023 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:01:04.155794  5216 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.161592  5054 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.251579  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:04.253456  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1010) took 1485 ms. Trace:
I20260709 13:01:04.262552  1560 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:04.197111  5210 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:01:04.257336  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:04.268381  1702 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:04.268734  1558 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:04.262656  1561 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.201941  5219 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:01:04.268754  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.233354  5217 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:04.217185  4958 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:04.272047  5224 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:01:04.272343  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.298776  1559 raft_consensus.cc:2410] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:04.274139  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:04.248658  4860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.014s	sys 0.013s
I20260709 13:01:04.276206  1554 raft_consensus.cc:2410] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:04.246830  4777 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.273250  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.283532  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.263058  2351 rpcz_store.cc:276] 0709 13:01:02.767549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.767795 (+   246us) service_pool.cc:224] Handling call
0709 13:01:02.768145 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:02.909756 (+141611us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:02.989128 (+ 79372us) log.cc:844] Serialized 14 byte log entry
0709 13:01:03.149826 (+160698us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:03.149880 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:03.149931 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:04.253150 (+1103219us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400686d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:04.253222 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:04.253380 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":135040,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":53689,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":154,"spinlock_wait_cycles":325041408,"thread_start_us":236920,"threads_started":2}
I20260709 13:01:04.288878  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:04.306833  1636 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.305377  1555 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:04.245350  5048 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.326990  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.336391  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.337509  1560 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:01:04.297060  4958 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:04.294382  5103 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:04.294544  5145 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:04.320984  5121 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:04.335497  5223 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:01:04.370862  1561 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:04.319326  5229 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:04.334233  4975 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:04.380676  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2243) took 1485 ms. Trace:
I20260709 13:01:04.301229  5217 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.381245  1632 rpcz_store.cc:276] 0709 13:01:02.895354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.511708 (+616354us) service_pool.cc:224] Handling call
0709 13:01:03.517982 (+  6274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:03.535793 (+ 17811us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:03.535842 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:03.545272 (+  9430us) log.cc:844] Serialized 26 byte log entry
0709 13:01:03.797092 (+251820us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:03.797155 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.797279 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:04.380415 (+583136us) raft_consensus.cc:1685] finished
0709 13:01:04.380440 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:04.380602 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123576,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":79,"thread_start_us":168129,"threads_started":1,"wal-append.queue_time_us":429549}
I20260709 13:01:04.383643  5217 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.390048  4749 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:04.390631  4749 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:04.391237  5215 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.391637  4749 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.392791  4749 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:04.394407  5232 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:01:04.395167  5215 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:04.395557  5225 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:01:04.403177  5048 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:04.403689  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2365) took 1698 ms. Trace:
I20260709 13:01:04.404083  5048 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:04.404273  1703 rpcz_store.cc:276] 0709 13:01:02.705588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.705713 (+   125us) service_pool.cc:224] Handling call
0709 13:01:04.403338 (+1697625us) spinlock_profiling.cc:243] Waited 94.4 ms on lock 0x7b54001e9e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:04.403539 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217216768}
I20260709 13:01:04.318298  4777 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.407891  4777 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.397073  5068 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:04.410991  5068 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:04.412137  5068 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.415131  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.415917  5216 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:04.416723  1703 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:04.418448  5029 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.419139  4962 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.424753  5119 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:04.431316  5020 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.010s
I20260709 13:01:04.372629  5121 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.434032  5121 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:04.434238  5184 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.524s	user 0.006s	sys 0.000s
W20260709 13:01:04.413234  5175 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.680s	user 0.012s	sys 0.000s
I20260709 13:01:04.442319  1636 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:04.443825  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:04.440661  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 205769us
I20260709 13:01:04.447824  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2360) took 2119 ms. Trace:
I20260709 13:01:04.449493  1674 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:04.451476  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:04.404985  5048 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.448263  1557 rpcz_store.cc:276] 0709 13:01:02.322532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.326640 (+  4108us) service_pool.cc:224] Handling call
0709 13:01:02.327217 (+   577us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.448547 (+121330us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:02.620870 (+172323us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.948883 (+328013us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.971141 (+ 22258us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.971347 (+   206us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:02.997765 (+ 26418us) log.cc:844] Serialized 26 byte log entry
0709 13:01:03.011550 (+ 13785us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:03.011616 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.011726 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:03.169952 (+158226us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b18003314e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:04.433636 (+1263684us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:04.441267 (+  7631us) raft_consensus.cc:1685] finished
0709 13:01:04.441289 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:04.441428 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87221,"prepare.queue_time_us":62238,"prepare.run_cpu_time_us":224,"prepare.run_wall_time_us":13870,"raft.queue_time_us":58987,"raft.run_cpu_time_us":583,"raft.run_wall_time_us":36700,"spinlock_wait_cycles":62121344,"thread_start_us":531041,"threads_started":3,"wal-append.queue_time_us":262557}
W20260709 13:01:04.448999  5173 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.681s	user 0.010s	sys 0.004s
I20260709 13:01:04.438103  5068 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:04.439592  5070 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.466393  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:04.452121  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:04.460691  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:04.441637  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 116322us
I20260709 13:01:04.448410  4962 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.470417  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:04.473948  4416 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:04.474411  4416 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:04.475689  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:04.452546  5109 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.626s	user 0.010s	sys 0.003s
I20260709 13:01:04.482189  4416 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.483362  4416 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.465726  5202 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.485838  5202 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.487987  5202 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.489683  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:04.461611  5133 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:04.491436  2349 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:01:04.494042  2349 pending_rounds.cc:85] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 6
I20260709 13:01:04.494374  2349 pending_rounds.cc:107] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:01:04.494659  2349 raft_consensus.cc:2889] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:04.457443  5020 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:04.456593  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:04.457541  5198 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:04.457667  5210 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:04.472903  5150 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.637s	user 0.014s	sys 0.000s
I20260709 13:01:04.565361  5150 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.568041  5150 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:04.569970  5212 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.932s	user 0.001s	sys 0.004s
W20260709 13:01:04.571803  5211 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.936s	user 0.004s	sys 0.000s
I20260709 13:01:04.573532  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1016) took 1295 ms. Trace:
I20260709 13:01:04.438246  5229 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.510490  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:04.515177  5196 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:04.516340  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.516726  5043 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:04.475301  5123 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:04.502499  5103 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.526150  5054 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.526373  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.528122  5020 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:04.492859  5109 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.532250  4959 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:04.450194  5175 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.573717  5228 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:01:04.626699  2350 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:04.628334  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:04.632009  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:04.540567  1676 leader_election.cc:336] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.968s (SENT)
W20260709 13:01:04.633023  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 92486.3us
I20260709 13:01:04.542210  2346 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:01:04.542889  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:04.543088  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:04.544786  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:04.635872  2348 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:04.545969  5198 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.547580  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.550287  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.637287  1705 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 5: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:04.639219  5075 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.639721  5075 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:04.641785  5240 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.169s	user 0.005s	sys 0.002s
I20260709 13:01:04.643571  5198 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.645344  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:04.646040  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:04.554684  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 639) took 1778 ms. Trace:
I20260709 13:01:04.551517  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:04.604104  1634 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:04.648092  1674 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:04.650722  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:04.652318  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 109319us
I20260709 13:01:04.633024  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:04.493846  5133 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.662142  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:04.681581  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:04.682484  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 48172.7us
I20260709 13:01:04.624337  2353 rpcz_store.cc:276] 0709 13:01:03.278167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.278349 (+   182us) service_pool.cc:224] Handling call
0709 13:01:03.293610 (+ 15261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:03.294098 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:03.294190 (+    92us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:03.295403 (+  1213us) log.cc:844] Serialized 564 byte log entry
0709 13:01:03.351208 (+ 55805us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:03.351252 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.351326 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:04.571019 (+1219693us) spinlock_profiling.cc:243] Waited 38.4 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:04.573148 (+  2129us) raft_consensus.cc:1685] finished
0709 13:01:04.573178 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:04.573458 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88397568,"thread_start_us":52405,"threads_started":1,"wal-append.queue_time_us":340272}
I20260709 13:01:04.679064  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.679757  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:04.646868  1629 rpcz_store.cc:276] 0709 13:01:02.777828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.343243 (+565415us) service_pool.cc:224] Handling call
0709 13:01:03.356444 (+ 13201us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:03.651065 (+294621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:03.651104 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:03.651757 (+   653us) log.cc:844] Serialized 52 byte log entry
0709 13:01:04.228241 (+576484us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:04.319000 (+ 90759us) log.cc:844] Serialized 14 byte log entry
0709 13:01:04.472572 (+153572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:04.472657 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:04.554451 (+ 81794us) raft_consensus.cc:1685] finished
0709 13:01:04.554471 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:04.554624 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5792,"prepare.queue_time_us":185745,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":78934,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":34751,"thread_start_us":941849,"threads_started":4,"wal-append.queue_time_us":699509}
I20260709 13:01:04.684753  1705 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.686731  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:04.686859  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.688103  1707 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:04.689319  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:04.689953  1702 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.696626  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.578016  4994 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:04.628820  4911 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.644450  5043 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:04.625416  5196 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:04.578612  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:04.628286  4959 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:04.688493  5133 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.700799  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:04.705817  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.711212  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:04.712567  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 64541.8us
I20260709 13:01:04.700665  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:04.702134  1701 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:04.713896  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.660175  5150 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:04.579836  5121 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.732793  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.718041  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.725536  1634 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:04.733770  1701 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:01:04.725070  1704 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:04.728953  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.733922  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:04.734189  1702 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:04.734563  1628 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:04.692044  5241 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.721822  1629 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:04.714928  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:04.691756  5201 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:04.735083  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:04.743010  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:04.619951  4958 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.736181  1675 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:04.648020  5210 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.740023  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.712031  5170 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:04.713610  5075 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:04.747117  4959 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.717736  5041 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:04.750082  1702 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:04.728127  5199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.005s	sys 0.018s
I20260709 13:01:04.752504  5133 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.701771  5173 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.754010  2347 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:04.754477  4911 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:04.820899  1702 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:04.754904  5229 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:04.711745  4416 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.695029  5239 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:01:04.728505  5198 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:01:04.756822  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.759891  5175 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.731340  5205 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.761298  5121 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:04.834914  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.835822  1561 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:04.762979  5109 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:04.713785  5023 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:01:04.763865  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:04.764039  5034 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.002s
I20260709 13:01:04.765853  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.853765  1628 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:04.851416  1709 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:04.700405  5048 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:04.771745  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 51404.6us
I20260709 13:01:04.856241  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:04.795715  1704 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:04.736014  4962 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.094s
W20260709 13:01:04.772405  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 49772us
I20260709 13:01:04.857378  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.858415  1553 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:04.804365  1706 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:01:04.860419  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 177716us
W20260709 13:01:04.860752  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 127682us
I20260709 13:01:04.807395  1707 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:04.861124  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:04.773514  1557 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:04.809399  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 73338.1us
I20260709 13:01:04.862172  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.862342  1559 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:04.862790  1555 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:04.863557  1560 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:04.696094  5203 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:01:04.704751  5244 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 13:01:04.818329  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 75438.2us
W20260709 13:01:04.817932  5199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.198s	user 0.005s	sys 0.018s
I20260709 13:01:04.874859  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:04.755438  5020 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.875710  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.876504  1552 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:04.850580  5154 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.851245  5044 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.826083  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:04.826817  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:04.889195  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.740413  4860 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.809032  4975 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.891396  4975 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.714514  4994 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.110s
I20260709 13:01:04.772063  5201 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:04.894856  5201 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:04.809708  5056 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:04.824002  4959 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.818480  5170 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.897912  5056 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:04.898954  5056 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.899590  5170 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:04.843498  5195 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.136s	user 0.017s	sys 0.000s
I20260709 13:01:04.824316  4911 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.741770  5150 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.902484  4911 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:04.902391  5195 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.904084  5150 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.818293  5230 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.828075  5173 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.905682  1710 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:04.906288  5195 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:04.824211  5041 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.808848  4982 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:01:04.846414  5133 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:04.918015  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:04.825544  5081 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.919013  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.919713  1552 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:04.848635  5210 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.828317  4416 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:04.850190  1702 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:04.851439  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 150835us
I20260709 13:01:04.924162  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:04.771005  5247 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:04.868978  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 154112us
I20260709 13:01:04.867389  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:04.878820  5199 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:04.874426  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:04.910413  5197 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.909832  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:04.930611  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56249.6us
I20260709 13:01:04.911247  1634 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:04.911363  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.931391  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:04.931625  1555 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:01:04.912815  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:04.914337  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:04.932785  1557 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:04.932866  2351 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:04.906585  5173 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:04.862004  5198 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:04.922890  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.923435  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:04.925796  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:04.827888  5043 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.886543  5034 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.824827  1705 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:04.950950  2318 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:04.951761  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.951980  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:04.952358  1705 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:04.952855  1703 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 13:01:04.930711  1553 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:04.910816  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:04.934267  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 66980.5us
I20260709 13:01:04.935117  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:04.942620  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.947337  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:04.949311  2353 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:01:04.955232  1706 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:04.921898  5251 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:01:04.950099  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.922899  4994 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.968971  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:04.956641  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:04.969005  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:04.959566  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.976167  1555 raft_consensus.cc:2410] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:04.962706  5262 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:04.963469  5103 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.963949  5203 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:04.964603  5075 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:04.931128  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:04.915491  5044 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.966543  5041 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:04.935390  5068 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:04.966246  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:04.971583  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:04.968957  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:04.963258  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:05.000969  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:05.001287  2352 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:04.970327  1559 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:05.002576  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.002851  1709 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:04.949051  5109 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:05.005260  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.005645  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:04.986820  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:04.949230  5170 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:04.968216  5198 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:04.948783  5175 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.915735  5154 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:04.992653  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:04.985582  5103 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.995103  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:04.956184  5034 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:04.982038  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:05.003126  1559 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.925062  5081 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.026281  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:05.009521  1676 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:04.958778  5259 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:04.985324  5145 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.999567  4958 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:04.958635  5251 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:04.991525  4975 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:05.058185  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 46438.7us
I20260709 13:01:04.993398  1706 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:05.041177  1553 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:05.042588  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:05.042189  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.043102  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:05.004567  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.066800  1557 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:05.041837  2348 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:05.011564  5241 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.014370  5041 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:05.060406  5154 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:05.021589  5175 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.014127  4994 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:05.025223  5170 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:05.022897  5029 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.088495  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:05.089247  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:05.089349  1553 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:05.064985  1709 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:05.067634  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:05.043781  5034 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.040671  5229 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:05.057039  5145 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.088043  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.044765  5103 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.120411  2345 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:05.090914  1706 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:05.092971  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:05.093444  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:05.128780  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:05.098242  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:05.098542  5266 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:01:05.099534  5209 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:05.101864  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:05.091789  5056 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:05.025563  5203 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:01:05.061759  4982 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:05.108982  5170 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:05.038822  5243 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 13:01:05.044363  5215 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:01:05.114907  5197 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:05.116060  5222 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.961s	user 0.004s	sys 0.000s
W20260709 13:01:05.118273  5254 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:01:05.023859  4911 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:05.121714  2347 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 873) took 1941 ms (client timeout 2219 ms). Trace:
I20260709 13:01:05.147588  5056 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.124794  5145 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:05.117192  5258 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:01:05.055968  5259 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:05.130162  1559 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:05.164366  5197 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:05.172315  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.173044  1559 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:05.173972  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.174669  1559 raft_consensus.cc:2376] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8. Re-sending same reply.
I20260709 13:01:05.131501  1675 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:05.144025  5023 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:01:05.086971  5175 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.136229  1557 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:05.135517  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:05.146180  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:05.179584  1558 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:05.164294  5084 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:05.179890  1552 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:05.133029  5029 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:05.134675  5209 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:05.151330  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 145252us
W20260709 13:01:05.166499  2347 rpcz_store.cc:269] 0709 13:01:03.180294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.180513 (+   219us) service_pool.cc:224] Handling call
0709 13:01:05.121516 (+1941003us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400478954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:05.121661 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3463720832}
I20260709 13:01:05.169749  5056 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.211840  5070 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:05.123997  5251 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:05.178615  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47207.6us
I20260709 13:01:05.222078  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:05.222716  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:05.223718  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:05.131851  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.225749  1553 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:05.180349  1556 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:05.151752  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:05.180593  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:05.227523  1559 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:05.227626  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:05.228341  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.229324  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:05.181248  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:05.130692  5262 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:05.172677  2344 raft_consensus.cc:2410] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:05.122484  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1795) took 2166 ms. Trace:
I20260709 13:01:05.185657  4911 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:05.191267  5271 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:01:05.192093  5260 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:01:05.092162  5199 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.194604  5196 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.196388  5034 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:05.197432  5265 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 13:01:05.262372  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.263055  1555 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:05.199026  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:05.201570  5109 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:05.204013  5247 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:05.207885  5043 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:05.218538  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2366) took 1711 ms. Trace:
I20260709 13:01:05.232146  1632 raft_consensus.cc:2368] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6.
I20260709 13:01:05.228999  2317 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:05.233794  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:05.236305  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:05.237970  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.232721  1708 rpcz_store.cc:276] 0709 13:01:02.955933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.023070 (+ 67137us) service_pool.cc:224] Handling call
0709 13:01:05.122283 (+2099213us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b54001e9454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:05.122428 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1400980352}
I20260709 13:01:05.239027  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.183365  5195 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:05.240347  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:05.241431  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.192948  5259 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.243247  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:05.248848  2352 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:01:05.183229  5029 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:05.247778  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:05.214308  5084 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.325757  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:05.326661  2349 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:05.317914  5023 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.327832  5023 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:05.330470  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:05.248059  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.331694  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:05.332451  1632 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:05.332510  1525 connection.cc:256] RPC call timeout handler was delayed by -1.0845986860000494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 13:01:05.264555  1553 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:05.245040  5262 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.272639  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:05.307703  1557 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:05.338619  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 98373.2us
I20260709 13:01:05.339949  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:05.340739  1553 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:05.341171  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:05.288779  1555 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:05.342806  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:05.242460  4951 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:05.307808  1710 rpcz_store.cc:276] 0709 13:01:03.506881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.507042 (+   161us) service_pool.cc:224] Handling call
0709 13:01:03.507444 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:04.906088 (+1398644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.205468 (+299380us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400247054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:05.218469 (+ 13001us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3215169792}
W20260709 13:01:05.309429  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 80519.3us
W20260709 13:01:05.310308  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 74101.5us
I20260709 13:01:05.275383  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:05.345649  1629 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:05.312062  5101 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.312551  5070 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.312988  5277 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:05.236976  4982 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:01:05.314893  5103 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.307742  2347 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:05.242214  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:05.318713  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:05.322430  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.361961  1559 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:01:05.349710  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:05.362926  1632 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:05.322471  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:05.363879  2346 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:05.324590  2352 pending_rounds.cc:85] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 6
I20260709 13:01:05.252362  5251 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:05.239929  5198 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:05.335891  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:05.232203  5201 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.337232  4749 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.344233  5179 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:05.348994  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:05.349309  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:05.316879  5195 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.277383  5145 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.276513  5056 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.356231  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:05.358567  1557 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:05.400501  2350 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:01:05.364814  2352 pending_rounds.cc:107] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:01:05.337260  5029 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:05.367394  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:05.375542  5198 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.344274  5109 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.348505  5050 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:05.436693  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 91457.2us
I20260709 13:01:05.469166  1552 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:05.337040  5041 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:05.469940  5041 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:05.472711  5041 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:05.380532  1553 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:05.344656  4951 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:05.350005  5247 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:05.346025  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:05.384200  1634 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:05.384709  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:05.351758  5277 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.254273  5197 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.367820  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:05.491506  1629 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:05.429445  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 62141us
I20260709 13:01:05.401311  5195 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.377259  5064 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:05.401710  1557 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:05.495318  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:05.406893  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.449430  2352 raft_consensus.cc:2889] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:05.430074  4777 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:05.430624  5271 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:01:05.430435  5275 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:01:05.355206  5101 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.480546  4951 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:05.482139  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 136153us
I20260709 13:01:05.520255  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:05.523135  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:05.492190  5109 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:05.434237  5276 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 13:01:05.484125  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.392454  5023 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:05.435611  5198 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.492858  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:05.493110  2346 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:05.499089  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:05.500684  1635 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:05.509555  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:05.386848  4749 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:05.511770  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.512694  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2248) took 2563 ms. Trace:
I20260709 13:01:05.478129  1710 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:01:05.528419  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:05.528399  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:05.573720  1631 rpcz_store.cc:276] 0709 13:01:02.949036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.644113 (+695077us) service_pool.cc:224] Handling call
0709 13:01:03.644566 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:03.942056 (+297490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.512399 (+1570343us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54001397d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:05.512636 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":682316672}
W20260709 13:01:05.574608  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 46281.9us
I20260709 13:01:05.574875  5023 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.575623  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:05.576713  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:05.577384  5020 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:05.577859  5020 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:05.578231  5020 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:05.491636  5277 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:05.579878  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51555.5us
W20260709 13:01:05.586180  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 93358.8us
I20260709 13:01:05.589145  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:05.527220  5197 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:05.591987  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2466) took 1135 ms. Trace:
I20260709 13:01:05.618499  1559 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:05.521142  5247 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:05.605223  1628 rpcz_store.cc:276] 0709 13:01:04.456078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.857030 (+400952us) service_pool.cc:224] Handling call
0709 13:01:04.876292 (+ 19262us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:04.909846 (+ 33554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:04.909881 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:04.924546 (+ 14665us) log.cc:844] Serialized 52 byte log entry
0709 13:01:04.997665 (+ 73119us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:05.045737 (+ 48072us) log.cc:844] Serialized 14 byte log entry
0709 13:01:05.055541 (+  9804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.055620 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:05.102469 (+ 46849us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b18003de640. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:05.581961 (+479492us) raft_consensus.cc:1685] finished
0709 13:01:05.591724 (+  9763us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:05.591913 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2626,"prepare.queue_time_us":47892,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":5284,"raft.queue_time_us":8850,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":6151,"spinlock_wait_cycles":18128768,"thread_start_us":38178,"threads_started":1,"wal-append.queue_time_us":150131}
I20260709 13:01:05.529394  5241 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.531306  5215 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.643649  5171 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:05.648222  4749 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:05.649901  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:05.650991  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61940.8us
I20260709 13:01:05.674567  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:05.584950  5262 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:05.583451  5109 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:05.590123  5023 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:05.676180  5023 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:05.676710  5023 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:05.592689  4777 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:05.660307  5041 raft_consensus.cc:2764] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:01:05.696743  5154 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:05.700700  5154 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.702486  5154 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.703091  5233 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:05.703526  5233 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:05.704294  5233 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:05.705237  5233 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:05.705554  5171 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:05.708950  1632 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:05.726082  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:05.726891  2346 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.661463  5288 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:01:05.664348  5241 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.732842  5241 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:05.706557  5103 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:05.709416  5172 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 13:01:05.734194  5209 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.545s	user 0.007s	sys 0.002s
I20260709 13:01:05.725062  5201 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:05.746613  4911 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:05.706924  4749 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:01:05.729084  5056 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:05.751520  4911 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:05.786605  5029 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.788946  5029 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:05.790285  5213 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 2.163s	user 0.008s	sys 0.008s
I20260709 13:01:05.747357  5209 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:05.792737  2346 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:05.803900  4911 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.805625  4911 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:05.814088  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 598) took 1472 ms. Trace:
I20260709 13:01:05.814568  1703 rpcz_store.cc:276] 0709 13:01:04.342195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.765127 (+422932us) service_pool.cc:224] Handling call
0709 13:01:04.952606 (+187479us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:04.953354 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.813869 (+860515us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400336b50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:05.814021 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430259456}
I20260709 13:01:05.822642  5044 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:05.825934  5298 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:01:05.829430  1710 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:05.831707  5020 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.779264  5101 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:05.784919  4951 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:05.843792  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2456) took 1671 ms. Trace:
I20260709 13:01:05.844801  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:05.847741  1602 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:05.844213  1627 rpcz_store.cc:276] 0709 13:01:04.172102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.218604 (+ 46502us) service_pool.cc:224] Handling call
0709 13:01:04.242236 (+ 23632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:04.382267 (+140031us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:04.693381 (+311114us) log.cc:844] Serialized 14 byte log entry
0709 13:01:04.929385 (+236004us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:04.929988 (+   603us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:04.930251 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.099111 (+168860us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b48000318d8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:01:05.843530 (+744419us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54003cdcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:05.843599 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:05.843726 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64454,"prepare.queue_time_us":32008,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":33376,"raft.run_cpu_time_us":1102,"raft.run_wall_time_us":163385,"spinlock_wait_cycles":351294592,"thread_start_us":479895,"threads_started":2,"wal-append.queue_time_us":361482}
I20260709 13:01:05.858316  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:05.843724  5068 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.862006  5068 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:05.863140  5198 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.864599  5068 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:05.865923  5199 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 13:01:05.866400  5199 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:01:05.866744  5043 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.544s	user 0.007s	sys 0.003s
I20260709 13:01:05.867522  5103 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:05.867756  4777 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:05.867667  5154 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.868335  4777 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:05.866895  5247 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:05.853703  2344 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.024s	sys 0.032s
W20260709 13:01:05.874737  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 176420us
I20260709 13:01:05.842563  5044 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.884627  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:05.895676  1634 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:01:05.898861  1525 leader_election.cc:400] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 5 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6.
I20260709 13:01:05.866823  5277 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:05.900283  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:05.901727  5277 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:05.880918  1560 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:05.865409  5198 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:05.902756  5023 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:05.897538  2344 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.025s	sys 0.032s
I20260709 13:01:05.903084  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:05.899899  1631 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:05.906730  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:05.910406  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:05.914016  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:05.916077  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:05.917670  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:05.922227  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:05.923206  1703 raft_consensus.cc:2410] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:01:05.929823  1555 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.007s	sys 0.019s
I20260709 13:01:05.844363  5195 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:05.874367  5103 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:05.931883  1555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.007s	sys 0.019s
I20260709 13:01:05.869272  4777 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:05.920648  5170 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.007s	sys 0.019s
I20260709 13:01:05.913836  5020 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:05.914186  5029 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:05.919587  4749 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.914741  5154 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:05.919737  5056 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.919486  5172 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.920964  5023 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:06.010092  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 76446.9us
I20260709 13:01:05.914458  5044 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:06.015786  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:05.956763  5170 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.007s	sys 0.020s
I20260709 13:01:06.021888  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2378) took 1188 ms. Trace:
I20260709 13:01:06.005630  4777 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.962182  5043 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.941494  5103 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:06.048012  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.023393  4749 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:06.015175  5029 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:01:06.022436  1554 rpcz_store.cc:276] 0709 13:01:04.833720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.849433 (+ 15713us) service_pool.cc:224] Handling call
0709 13:01:04.849792 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:04.850166 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:04.854173 (+  4007us) log.cc:844] Serialized 14 byte log entry
0709 13:01:05.851246 (+997073us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b180034eac0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:01:06.018517 (+167271us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:06.018644 (+   127us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:06.018739 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.021712 (+  2973us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.021831 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3472,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":183,"spinlock_wait_cycles":4313984,"thread_start_us":166735,"threads_started":1}
W20260709 13:01:06.047231  5251 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.670s	user 0.000s	sys 0.011s
I20260709 13:01:06.081723  1636 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:06.085803  5029 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:06.086265  5029 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:06.087061  5029 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:06.088302  5029 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:06.089788  4749 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:06.079695  5175 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.864s	user 0.013s	sys 0.006s
I20260709 13:01:06.092372  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1840) took 1170 ms. Trace:
I20260709 13:01:06.038826  5170 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:01:06.092749  1555 rpcz_store.cc:276] 0709 13:01:04.922334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.342266 (+419932us) service_pool.cc:224] Handling call
0709 13:01:05.342590 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:05.342908 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:05.342934 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:05.349798 (+  6864us) log.cc:844] Serialized 26 byte log entry
0709 13:01:05.936012 (+586214us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:05.936063 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.936125 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:06.092154 (+156029us) raft_consensus.cc:1685] finished
0709 13:01:06.092173 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.092318 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1609,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":6523,"thread_start_us":536354,"threads_started":1,"wal-append.queue_time_us":650682}
I20260709 13:01:06.096658  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.074175  5209 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:06.097309  1559 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:01:06.097820  1555 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:06.098018  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:06.115835  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:06.115914  1558 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:06.122547  5044 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:06.092885  4951 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:06.123624  5154 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:06.090775  5175 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:06.124164  5305 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:01:06.124212  5154 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.131186  5154 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:06.134500  5299 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.339s	user 0.001s	sys 0.005s
I20260709 13:01:06.139298  5195 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:06.142638  5109 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.142777  5081 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:06.143190  5109 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.145632  5199 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:06.146231  5199 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:06.146602  5044 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:06.147156  5044 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.148247  5199 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:06.149318  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 50371.9us
I20260709 13:01:06.150254  5239 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:06.150635  5239 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:06.151818  5064 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:06.152190  5064 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:06.152971  5064 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.153851  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:06.154273  5064 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:06.154409  1627 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:06.155325  5239 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.156531  5277 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:06.148298  4416 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:06.090833  4749 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:06.157905  5239 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:06.161679  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1833) took 1321 ms. Trace:
I20260709 13:01:06.162154  1560 rpcz_store.cc:276] 0709 13:01:04.840610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.864260 (+ 23650us) service_pool.cc:224] Handling call
0709 13:01:06.161500 (+1297240us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b54002526d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:06.161621 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2104127872}
I20260709 13:01:06.130180  5103 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:06.163821  5145 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:06.167060  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:06.162549  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61712.2us
I20260709 13:01:06.167783  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:06.149235  5199 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.169181  2349 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:06.169591  5199 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:06.168713  5282 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.738s	user 0.003s	sys 0.010s
I20260709 13:01:06.172765  2351 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:06.078696  5304 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.110s
W20260709 13:01:06.173301  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.174727  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.222050  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:06.223172  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:06.223909  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:06.211227  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.178988  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:06.197624  1560 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:01:06.241801  2349 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:06.217466  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:06.179927  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:06.246212  1703 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:06.196745  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:06.248317  1708 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:06.248883  2350 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:06.247941  1632 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:06.249855  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:06.251435  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:06.251734  1627 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:06.177608  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:06.253057  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:06.255613  1702 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:06.222342  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:06.257730  1707 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:06.258035  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.258985  1636 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:06.259346  1557 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:06.262374  1557 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:06.263945  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.264557  1557 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:06.266561  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:06.267375  1557 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:06.272086  1636 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:06.269961  1629 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:06.278472  1675 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:06.223802  5296 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.455s	user 0.006s	sys 0.006s
I20260709 13:01:06.284514  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:06.286412  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.286942  1708 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:06.290247  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55037.8us
I20260709 13:01:06.290962  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:06.291400  1557 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:06.292032  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:06.292732  1702 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:06.289614  2318 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:06.300007  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:06.302263  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:06.222095  5196 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:06.309047  1628 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:06.311044  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:06.305428  1632 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:06.190594  5109 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:06.311995  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:06.200830  5044 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:06.179453  5203 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.134s
I20260709 13:01:06.289614  5145 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.344405  5145 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:06.326347  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:06.306353  5017 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:06.350880  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:06.348284  1560 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:06.325774  5171 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:06.326500  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:06.361425  1561 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:06.320240  1675 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:06.363330  1675 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:06.356361  1707 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:06.359488  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:06.368625  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:06.322122  5070 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:06.372288  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.321576  4982 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:06.433493  1557 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:06.308096  5205 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.065s
W20260709 13:01:06.449615  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 86354.7us
I20260709 13:01:06.436249  1708 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:06.308617  4777 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:01:06.372579  5154 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:06.373531  5171 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:06.412958  5044 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.489298  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:06.495154  1560 raft_consensus.cc:2376] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7. Re-sending same reply.
I20260709 13:01:06.439491  5199 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:06.378153  5070 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.482069  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:06.488938  5195 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 13:01:06.503648  5319 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:01:06.483665  5172 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:06.466270  5017 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:06.526078  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.529860  1560 raft_consensus.cc:2376] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10. Re-sending same reply.
I20260709 13:01:06.530941  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.464876  5018 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:06.518711  5199 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.525833  5133 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:06.524454  4982 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:01:06.500847  5171 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:06.527670  5233 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:01:06.512774  5044 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:06.518131  5070 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:06.579954  5175 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:06.619112  2345 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:06.625029  5322 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
W20260709 13:01:06.602003  5308 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:01:06.604650  5289 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.835s	user 0.000s	sys 0.003s
I20260709 13:01:06.605818  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.636191  2351 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:06.637835  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:06.638968  2349 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:06.582284  5018 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:06.640484  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:06.641592  2346 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:06.641147  5018 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:06.642874  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:06.645179  2352 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:06.634509  1559 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:01:06.523866  1675 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:06.648627  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:06.649216  1559 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:06.649582  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:06.650177  2350 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:06.648181  1558 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
W20260709 13:01:06.654021  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 130230us
I20260709 13:01:06.655730  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:06.656179  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:06.611610  5070 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.614425  5170 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:06.632285  1552 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.032s	sys 0.062s
I20260709 13:01:06.552132  5199 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:06.685226  1559 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:06.664095  1556 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:06.648447  5205 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.687991  1559 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:06.653527  1561 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:06.650646  5043 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:06.681767  1552 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.037s	sys 0.062s
I20260709 13:01:06.670789  5244 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:06.743650  1628 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:06.672745  5170 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:06.695721  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 656) took 1497 ms (client timeout 1631 ms). Trace:
W20260709 13:01:06.748541  1635 rpcz_store.cc:269] 0709 13:01:05.198637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.227181 (+ 28544us) service_pool.cc:224] Handling call
0709 13:01:06.695545 (+1468364us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540017af54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:06.695667 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322621824}
I20260709 13:01:06.682839  2347 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:06.695528  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:06.676877  4982 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:06.693192  1561 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:01:06.747591  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2458) took 2568 ms. Trace:
I20260709 13:01:06.746749  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:06.754798  1631 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:06.751375  1633 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:01:06.750491  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 55096.2us
I20260709 13:01:06.752159  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:06.741844  5239 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.583s	user 0.013s	sys 0.002s
I20260709 13:01:06.756798  1675 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:06.754462  1630 rpcz_store.cc:276] 0709 13:01:04.179202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.226826 (+ 47624us) service_pool.cc:224] Handling call
0709 13:01:04.236761 (+  9935us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:04.303540 (+ 66779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:04.303595 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:05.323370 (+1019775us) spinlock_profiling.cc:243] Waited 6.83 ms on lock 0x7b2c00070338. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:01:05.339482 (+ 16112us) log.cc:844] Serialized 26 byte log entry
0709 13:01:05.529295 (+189813us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:05.529347 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.529442 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:06.603885 (+1074443us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:06.747340 (+143455us) raft_consensus.cc:1685] finished
0709 13:01:06.747369 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.747522 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66006,"raft.queue_time_us":67046,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":25431,"spinlock_wait_cycles":19449856,"thread_start_us":149860,"threads_started":1,"wal-append.queue_time_us":392942}
I20260709 13:01:06.662864  5171 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:06.763679  5171 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:06.764426  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:06.765167  1628 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:06.671383  5233 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:01:06.742794  5017 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:06.766482  1560 raft_consensus.cc:2410] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:06.672271  5020 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:01:06.765173  5070 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:06.770076  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.767866  5203 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:01:06.744947  5244 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.800500  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:06.741987  4911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.007s	sys 0.007s
I20260709 13:01:06.773892  4994 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:06.770987  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:06.768086  5195 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:06.776392  5199 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:06.741604  5205 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.780982  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:06.806035  1627 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:06.765228  5133 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:06.790218  1636 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:06.790581  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:06.792047  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:06.807375  1561 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:06.759779  5317 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:01:06.798344  1635 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:01:06.796301  5029 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.708s	user 0.012s	sys 0.001s
I20260709 13:01:06.799266  5244 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:06.807538  1629 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:06.822175  5233 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.822861  5233 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.823324  1561 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:06.824712  2318 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:06.811368  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:06.828650  1635 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:06.807262  5203 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:06.831385  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:06.834122  1627 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:06.837524  1527 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:06.809379  5195 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:06.812356  4994 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:06.839226  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:06.844038  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:06.832386  5103 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:06.845940  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:06.848105  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 479554us
I20260709 13:01:06.849538  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:06.838433  5034 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:06.839172  5044 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:06.854887  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:06.848609  5233 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:06.840758  5317 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:01:06.855814  5195 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.859397  4994 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:06.861104  5315 log.cc:927] Time spent T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.576s	user 0.002s	sys 0.000s
I20260709 13:01:06.861449  5331 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:06.862097  5324 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 13:01:06.846915  5330 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:01:06.862777  5056 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:06.869179  5320 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.492s	user 0.001s	sys 0.000s
I20260709 13:01:06.843881  5029 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:06.844769  5304 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:06.849102  5064 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:06.875965  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:06.873140  5171 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:06.870471  5103 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:06.902907  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.904094  1555 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:06.908782  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.910257  1555 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:06.911785  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.913607  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.914770  1555 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:06.884917  5171 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:06.882825  4982 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:06.918597  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.883394  5064 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:06.883119  5304 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:06.920179  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:06.924990  1555 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
W20260709 13:01:06.925376  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 173314us
I20260709 13:01:06.928574  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:06.929100  1554 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:06.929322  1561 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:06.879866  4777 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:01:06.904709  5044 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:06.931919  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:06.936311  2347 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:01:06.929586  5171 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:06.940652  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:06.940901  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:06.941391  2347 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:06.941431  2350 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:06.941615  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:06.922935  5154 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:06.937940  1676 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:06.948062  5342 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:06.948470  5342 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.948757  4911 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:06.949463  4911 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.951602  4911 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:06.932163  5064 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.952682  5244 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.925616  5334 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:06.935840  5333 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:01:06.945847  5044 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.939024  5203 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:06.948827  5342 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:06.892513  5103 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.960681  2350 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:06.938866  5316 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:06.939281  5070 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.932130  5304 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.946267  5277 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:06.966325  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 200162us
I20260709 13:01:06.989249  5084 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:06.957234  5119 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:06.982573  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:06.993336  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 92483.5us
I20260709 13:01:06.967288  5133 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:06.999538  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59361.1us
I20260709 13:01:06.988667  5048 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:06.959673  5154 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.006372  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:06.920467  4982 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:07.010170  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:07.008374  1601 leader_election.cc:336] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 3.351s (SENT)
I20260709 13:01:06.999764  5316 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:07.017089  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:07.007416  5346 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.993964  5171 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.009202  5133 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.018715  5346 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.020725  5346 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:07.006758  5070 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:06.999315  5230 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:01:06.989825  5334 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.022976  5070 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:07.023488  5334 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.005808  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 600) took 1770 ms. Trace:
I20260709 13:01:07.025794  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:07.024605  2344 rpcz_store.cc:276] 0709 13:01:05.234925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.328893 (+ 93968us) service_pool.cc:224] Handling call
0709 13:01:05.329268 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:05.329687 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:05.904161 (+574474us) log.cc:844] Serialized 14 byte log entry
0709 13:01:06.078434 (+174273us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:06.133911 (+ 55477us) log.cc:844] Serialized 26 byte log entry
0709 13:01:06.134075 (+   164us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:06.134355 (+   280us) log.cc:844] Serialized 14 byte log entry
0709 13:01:06.134580 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.134663 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:06.860456 (+725793us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:07.002846 (+142390us) raft_consensus.cc:1685] finished
0709 13:01:07.002873 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:07.005736 (+  2863us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6761,"prepare.queue_time_us":29451,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":23296,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":6507,"spinlock_wait_cycles":46996224,"thread_start_us":721567,"threads_started":2,"wal-append.queue_time_us":380502}
I20260709 13:01:07.028319  5316 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.010640  5277 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.030076  5215 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:07.031482  5277 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:06.995541  5203 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.031492  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:07.033602  5203 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:07.034489  5215 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:07.034915  5244 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.034916  5215 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:07.035658  5215 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:07.045969  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:07.046754  1552 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
W20260709 13:01:07.048430  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.050928  2318 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:07.046108  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:07.052773  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:07.053403  1552 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:07.029369  5316 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:07.055310  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 72817.8us
I20260709 13:01:07.083833  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:07.084383  1561 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:07.087903  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:07.092626  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:07.096987  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46173us
W20260709 13:01:07.098562  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 80960.5us
I20260709 13:01:07.082619  5335 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:07.053151  5334 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.104743  2318 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:07.106510  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:07.107146  2345 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:07.107417  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 101109us
I20260709 13:01:07.107844  5346 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:07.108477  5346 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:07.109428  5346 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.111021  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:07.112711  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.015347  5119 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.117156  2345 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:07.090649  5199 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:07.096957  5068 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:07.086365  5341 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:07.016521  5048 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:07.094566  5217 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:07.101939  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:07.121582  5325 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.682s	user 0.004s	sys 0.000s
I20260709 13:01:07.101177  5203 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:07.126902  2318 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:07.125973  1674 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:07.095191  5109 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:01:07.140467  2318 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:07.121367  5070 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:07.095075  5175 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:01:07.122457  5344 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:07.097681  5327 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:01:07.140579  5020 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:07.118566  5334 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.140398  5203 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.142673  5175 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:07.141208  5070 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:07.137651  5217 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.173362  5348 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:07.211123  5348 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.183182  5203 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:07.211609  5348 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:07.218470  5048 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:07.218900  5044 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.184197  5277 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:07.181895  5344 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.218994  5323 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.201197  5175 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.204289  5070 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:07.222816  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.227402  5044 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:07.210305  5337 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:01:07.208544  5145 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:07.235380  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:07.236222  1560 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:07.237232  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:07.239509  1560 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:07.219136  5348 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.229176  5175 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:07.228287  5304 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.228140  5064 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:07.259464  1560 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:07.232481  5277 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:07.262200  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:07.204710  4994 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:01:07.228399  5323 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.264061  5154 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.248072  5044 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.297945  5119 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:07.273892  5357 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:07.262424  5304 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.232605  5344 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.280893  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:07.280673  5327 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.357594  5327 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.264426  5064 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.361763  5172 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:07.362303  5172 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:07.363606  5064 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.363348  5172 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.365109  5172 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:07.287513  5259 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:07.310243  5277 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.402072  5259 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:07.402343  4416 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:07.402865  4416 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:07.403059  5259 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.404165  5259 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:07.314040  5203 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:07.314465  5351 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 13:01:07.314072  5350 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:01:07.315155  5344 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:07.330296  5217 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:07.331944  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 455492us
I20260709 13:01:07.332906  5361 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:07.333400  5056 raft_consensus.cc:3055] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:07.342767  5323 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:07.283147  5355 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:07.348654  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 510) took 1174 ms. Trace:
I20260709 13:01:07.350812  5044 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.511809  1558 rpcz_store.cc:276] 0709 13:01:06.174196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.174408 (+   212us) service_pool.cc:224] Handling call
0709 13:01:07.348388 (+1173980us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b5400311854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:07.348567 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":926674816}
I20260709 13:01:07.352428  5348 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:07.285804  5247 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.355654  5205 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:07.356436  5198 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:07.356534  5334 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:07.366326  5342 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.368314  5215 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.555012  5215 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:07.369930  5363 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:01:07.370415  1560 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:07.379197  5304 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:07.301282  5345 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
W20260709 13:01:07.387481  5302 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.046s	user 0.001s	sys 0.004s
I20260709 13:01:07.394678  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:07.558013  2344 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:07.395431  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:07.396829  1554 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:01:07.398928  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 100305us
W20260709 13:01:07.308142  5353 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 13:01:07.332521  5354 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:01:07.277561  5199 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:07.564278  5199 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.416235  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:07.416947  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:07.565771  1708 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:07.565938  1707 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:07.566843  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:07.567674  5342 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.567891  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:07.569139  1707 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:07.417357  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:07.361220  5327 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.417640  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:07.417769  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:07.420218  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:07.420539  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 281584us
I20260709 13:01:07.403838  4416 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.571596  4416 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:07.572779  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:07.425693  1701 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:07.574250  5346 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:07.575229  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:07.576654  5346 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:07.427162  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:07.513187  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:07.577435  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:07.581075  5109 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:07.527796  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:07.532112  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:07.582123  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.582044  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:07.583328  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.583390  5109 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.583911  5109 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:07.584590  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:07.577621  5323 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:01:07.585673  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:07.578322  5327 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:01:07.586241  1627 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:01:07.587042  5323 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:07.577648  5343 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:07.587605  5323 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:07.587726  5343 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.588183  5343 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:07.588475  5323 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.420934  5197 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:07.589711  5197 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.591723  5197 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:07.497359  5230 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.592461  5215 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:07.592346  5230 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.550499  5247 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.546878  5198 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:07.593535  1630 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:07.594221  5304 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:07.594086  5198 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.594517  5230 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.594591  5304 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:07.595142  5304 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:07.595507  5198 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:07.595642  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:07.597638  1704 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:07.597688  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:07.577852  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:07.598492  5203 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:07.599164  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:07.599715  1627 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:01:07.600262  5247 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:07.600589  1630 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:07.425096  5364 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.312412  5357 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.601773  5364 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.602650  1628 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:07.603461  5357 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.603791  5364 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:07.497349  5133 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.604584  5133 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:07.605343  5316 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.000s
I20260709 13:01:07.500034  5056 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:07.606201  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:07.606788  1633 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:07.606600  5056 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.607712  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:07.607852  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:07.608394  1708 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:07.612082  1557 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:07.592926  5215 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:07.613229  5070 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.504299  5361 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:07.615252  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.617110  1705 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:07.619688  1628 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:07.620077  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:07.538055  4777 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.630031  4777 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.632249  4777 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:07.543085  5355 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.607939  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:07.636876  1631 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:01:07.638375  5348 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:07.638813  5348 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:07.639222  5348 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:07.643992  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:07.644475  2318 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:07.606657  5133 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.628306  2348 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:07.629359  5323 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:07.644598  1633 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:07.607779  5056 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:07.624641  1557 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:07.647957  1459 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 272) took 28655 ms (client timeout 15000 ms). Trace:
I20260709 13:01:07.685402  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:07.686161  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.014s	sys 0.006s
W20260709 13:01:07.686663  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.014s	sys 0.006s
W20260709 13:01:07.687250  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 594646us
W20260709 13:01:07.688149  1552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.006s	sys 0.014s
W20260709 13:01:07.688599  1552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.006s	sys 0.014s
W20260709 13:01:07.648308  1676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.009s	sys 0.007s
I20260709 13:01:07.650338  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:07.651885  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:07.655702  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:07.659245  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:07.690567  2352 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:07.662427  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:07.664947  2349 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:07.667160  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:07.692253  2351 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:01:07.668570  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.013s	sys 0.006s
W20260709 13:01:07.692938  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.013s	sys 0.006s
I20260709 13:01:07.692893  2317 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:07.694895  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:07.669947  2346 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:01:07.695608  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:07.671344  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:07.581460  5341 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:07.587010  5327 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
W20260709 13:01:07.689496  1676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.010s	sys 0.007s
W20260709 13:01:07.708986  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 590314us
I20260709 13:01:07.709916  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:07.701395  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:07.699328  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 49046.9us
I20260709 13:01:07.709973  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:07.677685  1458 catalog_manager.cc:5697] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 5 to 6, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:07.716595  1458 catalog_manager.cc:5697] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 4 to 7, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:07.718561  1458 catalog_manager.cc:5697] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 3 to 4, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:07.719291  1557 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
W20260709 13:01:07.696975  5068 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.013s	sys 0.014s
I20260709 13:01:07.681538  4911 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:07.719957  1458 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 3 to 5, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 5 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:07.729411  1458 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:07.730894  1458 catalog_manager.cc:5697] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 6 to 7, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:07.732254  1458 catalog_manager.cc:5697] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:07.737684  2350 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:07.739020  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:07.739554  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:07.727658  2318 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:07.741746  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:07.721751  5018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.011s	sys 0.015s
I20260709 13:01:07.745450  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:07.760259  1705 raft_consensus.cc:2410] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:07.746395  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:07.762360  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:07.706840  5075 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:07.698771  5171 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.684979  5133 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:01:07.752604  1633 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:07.776412  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:07.777038  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:07.778091  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:07.720474  5068 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.013s	sys 0.014s
I20260709 13:01:07.753932  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:07.613628  5170 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.754398  5357 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:07.754766  1706 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:07.758693  5196 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:07.763099  1631 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:01:07.699486  5017 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.785885  5017 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.788757  5017 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.713047  4777 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:07.789505  4777 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:07.789795  2344 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:07.790062  4777 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:07.791805  5335 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.021s	sys 0.008s
W20260709 13:01:07.792178  5335 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.021s	sys 0.008s
I20260709 13:01:07.792414  5335 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:07.721171  5175 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.793300  5335 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.794222  5361 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.794629  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:07.794801  5361 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:07.795390  1631 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:07.682967  5203 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:01:07.795722  5361 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:07.712491  5103 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:07.721107  4994 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:07.799074  4994 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.799544  4994 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:07.681767  5230 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:01:07.802037  5355 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.721115  5119 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:07.803573  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:07.803968  5119 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:07.805199  5119 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.747000  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:07.808861  2353 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:07.771697  5084 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:01:07.682651  1459 rpcz_store.cc:269] 0709 13:00:38.992246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.996542 (+  4296us) service_pool.cc:224] Handling call
0709 13:01:07.647859 (+28651317us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:00.802156 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:00.803150 (+   994us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:00.803206 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:01:00.803228 (+    22us) tablet.cc:662] Decoding operations
0709 13:01:00.803993 (+   765us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:00.804071 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:00.804088 (+    17us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:01:00.804611 (+   523us) tablet.cc:701] Row locks acquired
0709 13:01:00.804658 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:01:00.804841 (+   183us) write_op.cc:270] Start()
0709 13:01:00.805004 (+   163us) write_op.cc:276] Timestamp: P: 1783602060804776 usec, L: 0
0709 13:01:00.805028 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:01:00.828986 (+ 23958us) log.cc:844] Serialized 2472 byte log entry
0709 13:01:01.750737 (+921751us) op_driver.cc:464] REPLICATION: finished
0709 13:01:03.179788 (+1429051us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda245d00ac 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2b110ef0
0709 13:01:03.261542 (+ 81754us) write_op.cc:301] APPLY: starting
0709 13:01:03.277542 (+ 16000us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:03.289996 (+ 12454us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:03.290047 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:03.293268 (+  3221us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:03.293654 (+   386us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:01:03.293713 (+    59us) write_op.cc:312] APPLY: finished
0709 13:01:03.294103 (+   390us) log.cc:844] Serialized 53 byte log entry
0709 13:01:03.525507 (+231404us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:03.541743 (+ 16236us) write_op.cc:454] Released schema lock
0709 13:01:03.551207 (+  9464us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:03.686853 (+135646us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:03.706403 (+ 19550us) write_op.cc:454] Released schema lock
0709 13:01:04.569300 (+862897us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
Metrics: {"mutex_wait_us":117428,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":1503421,"apply.run_cpu_time_us":61221,"apply.run_wall_time_us":391800,"num_ops":6,"prepare.queue_time_us":102154,"prepare.run_cpu_time_us":15251,"prepare.run_wall_time_us":153411,"raft.queue_time_us":428775,"raft.run_cpu_time_us":35753,"raft.run_wall_time_us":1813510,"replication_time_us":932141,"spinlock_wait_cycles":199606912,"thread_start_us":2322147,"threads_started":5,"wal-append.queue_time_us":488182,"wal-append.run_cpu_time_us":49486,"wal-append.run_wall_time_us":3515135}]]}
I20260709 13:01:07.810139  5084 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:07.773561  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:07.811240  5084 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.812646  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:07.736701  5327 consensus_queue.cc:1243] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer 95fc6a4cf5874b7eb91280dffcd3b3fb log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:01:07.759651  5018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.012s	sys 0.016s
I20260709 13:01:07.727826  5346 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:07.824759  1554 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:01:07.825275  1554 pending_rounds.cc:85] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 6
I20260709 13:01:07.825551  1554 pending_rounds.cc:107] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:01:07.825819  1554 raft_consensus.cc:2889] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:07.778789  1708 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 6: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:07.829267  5109 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:07.832542  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:07.833125  2352 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:01:07.842159  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:07.754261  5043 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:01:07.779232  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:07.779945  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:07.780190  2345 raft_consensus.cc:2410] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:07.740442  4911 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:07.779628  1458 catalog_manager.cc:5697] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 2 to 5, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:07.795396  5175 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:07.775816  5171 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:07.709830  5215 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:01:07.782829  2348 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:07.699733  5340 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.766485  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:07.813239  1631 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:07.763545  5217 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.763372  5247 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:08.075624  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.815249  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:07.816504  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:07.817451  1556 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:07.871883  5109 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:07.785295  5068 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:07.727195  5172 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:01:07.852936  1633 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:07.858410  1458 catalog_manager.cc:5697] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 3 to 4, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 4 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:07.838409  5327 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:08.057351  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 329765us
W20260709 13:01:08.103006  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.108462  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:08.086802  5196 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.848635  5346 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:08.151618  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:08.151909  5346 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.152966  5017 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:08.084479  5340 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.153895  5017 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:08.154301  5017 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.154632  5017 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.154891  5244 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:08.155256  5244 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:08.155428  5340 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:08.156034  5244 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.148933  5196 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.157788  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:08.158356  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44874.2us
I20260709 13:01:07.861353  4911 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.158977  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:07.838168  5355 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:01:07.845129  5365 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:08.159941  5365 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:08.160143  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:08.098685  5327 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:08.160511  5018 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.160811  5365 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.161502  5018 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.161669  5327 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:08.073356  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 200511us
I20260709 13:01:08.066594  2349 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:07.814883  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:08.163403  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 348595us
I20260709 13:01:08.164232  5198 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:08.164655  5198 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.164997  5198 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:07.817579  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:08.101296  5070 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:08.166062  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 348588us
I20260709 13:01:08.167826  5230 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:08.168255  5230 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.168637  5230 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.168318  5172 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.169296  5172 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.066804  5372 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:08.072410  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.082487  1558 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:08.076663  2351 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:01:08.095844  1633 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:08.088641  1458 catalog_manager.cc:5697] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:01:08.179453  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:08.180228  1633 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:08.179718  1458 catalog_manager.cc:5697] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 2 to 6, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:08.122098  2348 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:08.085855  5217 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.149606  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:08.150092  5171 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.081748  5247 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.159060  5196 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.157224  5244 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.172147  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:08.073591  2352 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
W20260709 13:01:08.086061  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:08.181958  1630 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.020s	sys 0.025s
W20260709 13:01:08.207347  1630 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.020s	sys 0.025s
I20260709 13:01:08.207253  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:08.088032  5103 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.081640  5373 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:08.264555  5373 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:08.265708  5373 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.096453  5043 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.187441  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:08.188680  2353 raft_consensus.cc:2410] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:08.078744  5328 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:08.150940  5277 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:08.192899  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:08.269286  1631 raft_consensus.cc:2376] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8. Re-sending same reply.
I20260709 13:01:08.195201  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:08.196106  2348 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:01:08.270274  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:08.270278  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:08.271090  1631 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:08.271224  1633 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:08.272145  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:08.274401  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:08.275151  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.022s	sys 0.044s
W20260709 13:01:08.275509  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.022s	sys 0.044s
W20260709 13:01:08.275822  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 517525us
W20260709 13:01:08.198396  1634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.005s	sys 0.016s
I20260709 13:01:08.200403  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:08.185992  5199 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.013s	sys 0.013s
I20260709 13:01:08.189410  1458 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 2 to 6, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:08.171234  5172 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:08.189956  5348 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.001s
W20260709 13:01:08.275936  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 80812.3us
I20260709 13:01:08.273212  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:08.277369  1558 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:08.200868  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:08.277796  1634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.005s	sys 0.017s
I20260709 13:01:08.278633  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:08.202466  5247 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.286814  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:08.295862  2317 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:08.197499  5316 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:01:08.292402  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.002s	sys 0.031s
W20260709 13:01:08.329875  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.002s	sys 0.032s
W20260709 13:01:08.330295  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 791625us
W20260709 13:01:08.289659  5199 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.013s	sys 0.013s
W20260709 13:01:08.297492  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.267807  4749 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.332404  1708 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:08.332911  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:08.333228  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:08.350826  1704 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:08.350903  1706 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:08.352279   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:08.333379  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:08.354468  1707 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:08.298485  1631 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:08.356087  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:08.337926  1527 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:08.290633  1458 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 6 to 7, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: HEALTHY } } }
I20260709 13:01:08.358212  1458 catalog_manager.cc:5697] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 2 to 6, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:08.360859  1458 catalog_manager.cc:5697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 6 to 9, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 9 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:08.289826  5233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.000s	sys 0.021s
I20260709 13:01:08.275233  5328 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:08.261664  5196 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:08.275733  5119 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.365605  1458 catalog_manager.cc:5697] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:01:08.201704  5370 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 13:01:08.362931  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:08.292626  5171 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:08.519042  5171 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.519443  5171 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.290383  5215 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:01:08.329375  5205 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.008s	sys 0.020s
W20260709 13:01:08.522207  5205 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.009s	sys 0.021s
I20260709 13:01:08.387630  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:08.524748  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:08.388211  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 229226us
I20260709 13:01:08.190202  5372 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:08.339659  5335 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.005s	sys 0.020s
I20260709 13:01:08.335431  5172 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.296175  5203 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:08.518141  5277 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.530706  1628 raft_consensus.cc:2410] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:08.577400  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:08.578706  1628 raft_consensus.cc:2410] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:08.580972  1628 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:01:08.412711  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 240618us
I20260709 13:01:08.536041  1558 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:08.349861  5133 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:08.540840  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:08.583338  1627 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:08.584515  1627 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:08.288622  5043 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.587303  5043 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.588603  5043 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:08.553323  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.589154  5043 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.595324  5043 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.596091  5043 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:08.598625  1628 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:08.600075  1628 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:08.601882  1628 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:08.603579  1628 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:08.407290  5364 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:08.294453  5103 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.335963  5348 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:08.557893  5335 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.005s	sys 0.021s
I20260709 13:01:08.540650  5145 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:08.334797  5068 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.368495  5316 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:08.388310  4749 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:08.633313  4994 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.833s	user 0.010s	sys 0.000s
I20260709 13:01:08.418635  5196 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.509542  5119 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.634203  5233 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:08.654874  5323 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.972s	user 0.013s	sys 0.000s
I20260709 13:01:08.619871  5172 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.651700  5018 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:08.662747  5376 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.473s	user 0.000s	sys 0.001s
I20260709 13:01:08.758800  5044 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:08.759238  5044 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:08.760332  5172 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.760056  5044 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.761153  5196 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.629570  5328 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.761704  5196 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.762375  5328 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:08.660742  5372 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.685732  5364 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:08.746136  5419 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:08.765738  5364 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.770951  5412 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:08.747320  5119 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.747656  5233 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.748288  5380 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:08.751422  4777 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.961s	user 0.009s	sys 0.001s
W20260709 13:01:08.751616  5304 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.156s	user 0.005s	sys 0.005s
I20260709 13:01:08.753188  4749 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.754901  5316 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.755244  5403 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:08.762209  5196 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:08.660352  5084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.005s	sys 0.022s
W20260709 13:01:08.771457  5382 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.402s	user 0.001s	sys 0.000s
I20260709 13:01:08.771463  5412 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.727622  5145 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.781916  5170 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:08.782642  5375 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.509s	user 0.001s	sys 0.000s
W20260709 13:01:08.771389  5377 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.486s	user 0.000s	sys 0.001s
W20260709 13:01:08.727535  5198 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.561s	user 0.013s	sys 0.008s
I20260709 13:01:08.777251  5103 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.743888  5217 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.853711  5198 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.854210  5391 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.854862  5391 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:08.856148  5198 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:08.857369  5348 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:08.859875  5230 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.691s	user 0.010s	sys 0.001s
I20260709 13:01:08.821415  5380 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.862419  5145 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.864522  5196 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:08.864166  5171 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.872131  5373 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:08.860386  5230 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.831410  5403 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:08.843531  5412 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:08.824934  5304 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:08.893159  5343 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.305s	user 0.013s	sys 0.000s
I20260709 13:01:08.903646  5364 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:08.907537  5379 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.348s	user 0.002s	sys 0.000s
W20260709 13:01:08.908738  5381 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.380s	user 0.000s	sys 0.001s
I20260709 13:01:08.909806  5327 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.931368  5364 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:08.922570  5343 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:08.933907  5343 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:08.898202  5403 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:08.934975  5403 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:08.897323  5357 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.076s	user 0.006s	sys 0.007s
I20260709 13:01:08.936650  5357 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:08.947088  5412 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:08.985157  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:08.985217  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:08.987246  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.988103  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2417) took 1860 ms. Trace:
I20260709 13:01:08.679747  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:08.656615  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 328892us
I20260709 13:01:08.656119  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 668) took 1018 ms. Trace:
I20260709 13:01:08.993162  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:08.658710  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 320870us
I20260709 13:01:08.993908  1629 rpcz_store.cc:276] 0709 13:01:07.637830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.637993 (+   163us) service_pool.cc:224] Handling call
0709 13:01:07.638331 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.638529 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:08.079026 (+440497us) log.cc:844] Serialized 14 byte log entry
0709 13:01:08.655540 (+576514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:08.655725 (+   185us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:08.655792 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.655883 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.656045 (+   162us) inbound_call.cc:177] Queueing success response
0709 13:01:08.910711 (+254666us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b18003b08c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240
0709 13:01:08.911783 (+  1072us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b18003b08c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219
0709 13:01:08.912713 (+   930us) spinlock_profiling.cc:243] Waited 8.88 ms on lock 0x7b18003b08c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda2b5ae58f 00007fda245ed571 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd
Metrics: {"mutex_wait_us":38621,"prepare.queue_time_us":914083,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":257052,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":46009,"spinlock_wait_cycles":43941376,"thread_start_us":1003164,"threads_started":3,"wal-append.queue_time_us":559109}
I20260709 13:01:08.990995  1632 raft_consensus.cc:2410] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:08.999670  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:08.988672  1559 rpcz_store.cc:276] 0709 13:01:07.127974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.334347 (+206373us) service_pool.cc:224] Handling call
0709 13:01:07.334733 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.335100 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.335139 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:07.335660 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:01:07.730143 (+394483us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:07.730180 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.730249 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:08.987349 (+1257100us) raft_consensus.cc:1685] finished
0709 13:01:08.987412 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.987836 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2035,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":394061,"threads_started":1,"wal-append.queue_time_us":565876}
I20260709 13:01:09.002501  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2494) took 1380 ms. Trace:
I20260709 13:01:09.003865  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 667) took 1380 ms. Trace:
I20260709 13:01:09.005084  1636 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:09.007265  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.008178  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2504) took 1181 ms. Trace:
I20260709 13:01:09.020666  1629 rpcz_store.cc:276] 0709 13:01:07.826959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.006584 (+1179625us) service_pool.cc:224] Handling call
0709 13:01:09.007219 (+   635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.007709 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.007761 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:09.007839 (+    78us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:09.007902 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.007956 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.008084 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:08.757289  4416 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.185s	user 0.009s	sys 0.000s
W20260709 13:01:08.776268  4911 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.617s	user 0.007s	sys 0.003s
I20260709 13:01:08.735338  4994 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.146802  1705 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:08.798035  5044 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:09.064803  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:09.083427  2317 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.012s	sys 0.016s
I20260709 13:01:09.100652  5109 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:08.892016  5277 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:09.135514  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 98138.1us
I20260709 13:01:09.136197  4911 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.137547  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:09.019507  1634 rpcz_store.cc:276] 0709 13:01:07.621329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.621521 (+   192us) service_pool.cc:224] Handling call
0709 13:01:07.628856 (+  7335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.629206 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.629231 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:07.639792 (+ 10561us) log.cc:844] Serialized 26 byte log entry
0709 13:01:08.297875 (+658083us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:08.297929 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.298100 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:09.001713 (+703613us) raft_consensus.cc:1685] finished
0709 13:01:09.001758 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.002155 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":313,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":62,"thread_start_us":657682,"threads_started":1,"wal-append.queue_time_us":988993}
W20260709 13:01:09.165146  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 165733us
W20260709 13:01:09.217329  2317 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.012s	sys 0.023s
I20260709 13:01:09.019941  1635 rpcz_store.cc:276] 0709 13:01:07.623696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.623825 (+   129us) service_pool.cc:224] Handling call
0709 13:01:07.628361 (+  4536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.628690 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.628717 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:07.629192 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:01:08.072478 (+443286us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:08.072518 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.072582 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:09.003266 (+930684us) raft_consensus.cc:1685] finished
0709 13:01:09.003310 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.003647 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":576,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":297625,"threads_started":1,"wal-append.queue_time_us":1111943}
W20260709 13:01:09.163661  5056 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.478s	user 0.013s	sys 0.000s
W20260709 13:01:09.185339  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 663) took 1604 ms (client timeout 1790 ms). Trace:
I20260709 13:01:09.158141  5171 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.147764  5230 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:09.168757  5304 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:09.152379  5361 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:09.230921  4749 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.160660  5405 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.232144  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:09.234781  4911 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:09.137571  5145 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 13:01:09.239223  5233 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:09.240121  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:09.240886  5406 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:09.224217  5217 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:09.241829  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 878914us
I20260709 13:01:09.243093  2351 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:09.230816  5109 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:09.227747  1632 rpcz_store.cc:269] 0709 13:01:07.580558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.607618 (+ 27060us) service_pool.cc:224] Handling call
0709 13:01:09.184915 (+1577297us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54003b9cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:09.185205 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3175500544}
I20260709 13:01:09.183926  5380 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.185776  5415 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:09.253968  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.024s	sys 0.034s
I20260709 13:01:09.160301  5412 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.162380  5391 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.181231  5328 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:09.206360  4416 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.151743  5355 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:09.261307  1633 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
W20260709 13:01:09.295868  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.024s	sys 0.034s
I20260709 13:01:09.264761  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:09.152123  5364 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.145s
I20260709 13:01:08.821852  4777 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.318928  1634 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:09.315204  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 722737us
I20260709 13:01:09.324222  1635 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
W20260709 13:01:09.328893  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.309983  4994 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:09.327661  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2492) took 1603 ms. Trace:
I20260709 13:01:09.287318  4749 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:09.331190  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 194024us
W20260709 13:01:09.347823  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.001s	sys 0.015s
W20260709 13:01:09.348503  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.001s	sys 0.015s
W20260709 13:01:09.349557  5367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.049s	sys 0.070s
W20260709 13:01:09.349964  5017 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.195s	user 0.007s	sys 0.013s
W20260709 13:01:09.350265  5367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.049s	sys 0.071s
W20260709 13:01:09.355997  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:09.357280  5203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.006s	sys 0.014s
W20260709 13:01:09.359508  5203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.007s	sys 0.015s
I20260709 13:01:09.359963  5203 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:09.311726  5405 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.361945  5203 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:09.367105  5247 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.019s	user 0.006s	sys 0.011s
I20260709 13:01:09.367934  5018 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.367713  5247 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.368767  5018 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.369226  1628 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:09.369606  5380 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:09.370066  5380 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:09.370498  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 120649us
I20260709 13:01:09.371671  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:09.372001  5360 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:09.373982  5355 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:09.374509  5355 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.374927  5355 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.376652  5217 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.346073  5328 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.380545  5328 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.370534  5380 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:09.382686  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2500) took 1698 ms. Trace:
I20260709 13:01:09.383172  1627 rpcz_store.cc:276] 0709 13:01:07.683935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.585248 (+901313us) service_pool.cc:224] Handling call
0709 13:01:08.590390 (+  5142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:08.590634 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:08.990475 (+399841us) log.cc:844] Serialized 14 byte log entry
0709 13:01:09.341066 (+350591us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:09.351870 (+ 10804us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:09.358236 (+  6366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.382163 (+ 23927us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.382431 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63920,"prepare.queue_time_us":322075,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":80860,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":9048,"thread_start_us":665118,"threads_started":2}
I20260709 13:01:09.357328  5199 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:09.386059  5199 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.387048  5018 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:09.388684  5199 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.372982  5360 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.335938  5415 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.390195  5145 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:09.391287  5415 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:09.393595  4777 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.394430  5412 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:09.332696  5406 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.395087  5412 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:09.398074  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44904.9us
I20260709 13:01:09.400144  5412 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:09.400576  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.395740  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:09.406853  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:09.406140  1709 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.002s	sys 0.013s
W20260709 13:01:09.411720  2319 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.010s	sys 0.004s
W20260709 13:01:09.418197  1709 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.002s	sys 0.014s
I20260709 13:01:09.395560  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:09.333513  1630 rpcz_store.cc:276] 0709 13:01:07.620775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.622159 (+  1384us) service_pool.cc:224] Handling call
0709 13:01:07.641284 (+ 19125us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.641763 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:08.207767 (+566004us) log.cc:844] Serialized 14 byte log entry
0709 13:01:08.989787 (+782020us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:08.991170 (+  1383us) log.cc:844] Serialized 26 byte log entry
0709 13:01:08.991429 (+   259us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:08.991509 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.991692 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:09.188457 (+196765us) raft_consensus.cc:1685] finished
0709 13:01:09.188491 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.223846 (+ 35355us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":980450,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":204505,"raft.run_cpu_time_us":629,"raft.run_wall_time_us":6723,"thread_start_us":1314968,"threads_started":3,"wal-append.queue_time_us":696881}
I20260709 13:01:09.396359  5360 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:09.420799  2319 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.011s	sys 0.004s
I20260709 13:01:09.435341  5420 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:09.436435  5420 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.350672  5017 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:09.417641  5378 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.016s	sys 0.027s
I20260709 13:01:09.351601  4416 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.422864  5199 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.441041  5420 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:09.443423  5405 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:09.391319  5145 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.451939  5199 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.445254  1675 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:09.424535  5365 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.016s	sys 0.053s
I20260709 13:01:09.453495  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:09.426522  5215 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.024s	sys 0.021s
I20260709 13:01:09.458355  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:09.454595  2316 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.003s	sys 0.011s
W20260709 13:01:09.454670  2318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.007s	sys 0.012s
W20260709 13:01:09.460888  2318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.008s	sys 0.012s
W20260709 13:01:09.477448  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.479048  4416 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.480727  5430 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:09.457098  5433 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.482086  5433 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.483063  4749 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:09.483409  1559 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:01:09.476610  2316 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.003s	sys 0.012s
I20260709 13:01:09.484458  5433 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:09.485265  5433 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:09.460440  5378 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.017s	sys 0.028s
I20260709 13:01:09.494715  5109 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:09.495648  5423 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.496425  5423 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.497298  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:09.498265  1634 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
W20260709 13:01:09.509968  5244 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.249s	user 0.015s	sys 0.005s
I20260709 13:01:09.446969  5328 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
W20260709 13:01:09.515053  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61761.4us
I20260709 13:01:09.504446  5367 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:09.495013  5170 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.646s	user 0.013s	sys 0.005s
I20260709 13:01:09.499704  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:09.488060  5215 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.024s	sys 0.021s
W20260709 13:01:09.476059  5365 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.017s	sys 0.055s
I20260709 13:01:09.487656  5233 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:09.522007  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 610) took 1822 ms. Trace:
I20260709 13:01:09.489578  5428 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.526353  1627 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:01:09.528445  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51685.2us
I20260709 13:01:09.519729  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:09.516657  5367 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:09.536196  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:09.516765  4749 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.528748  5423 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:09.548009  5043 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.520706  5199 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:09.539605  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:09.549630  1556 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:09.550002  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 908) took 1370 ms. Trace:
I20260709 13:01:09.550338  1526 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:09.548871  5380 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.550442  2351 rpcz_store.cc:276] 0709 13:01:08.179803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.180597 (+   794us) service_pool.cc:224] Handling call
0709 13:01:08.208395 (+ 27798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.304004 (+1095609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.549604 (+245600us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54007bff54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:09.549899 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2378167680}
I20260709 13:01:09.538406  5367 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.554913  1559 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:09.555316  1556 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:09.557286  2348 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:01:09.559480  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:09.560448  2352 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:09.561002  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:09.540230  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:09.526002  1709 rpcz_store.cc:276] 0709 13:01:07.694401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.694541 (+   140us) service_pool.cc:224] Handling call
0709 13:01:07.720974 (+ 26433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.721182 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:08.197975 (+476793us) log.cc:844] Serialized 14 byte log entry
0709 13:01:09.446839 (+1248864us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:09.453751 (+  6912us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:09.474031 (+ 20280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.486790 (+ 12759us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.517067 (+ 30277us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":88327,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":139752,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":1696326,"threads_started":3,"wal-append.queue_time_us":656347}
I20260709 13:01:09.541395  2347 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:09.520792  5017 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:09.538785  1634 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:09.537923  5428 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:09.571374  5403 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.636s	user 0.011s	sys 0.004s
I20260709 13:01:09.571684  5361 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:09.572292  5361 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.572713  5438 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:09.573652  5428 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.537708  5233 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.573556  5438 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.562253  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:09.562356  4749 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.563704  5433 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:09.565146  5406 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.517100  5215 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:09.568156  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2286) took 1380 ms. Trace:
W20260709 13:01:09.542120  5327 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.616s	user 0.000s	sys 0.019s
I20260709 13:01:09.570263  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:09.563746  5217 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:09.577862  1629 rpcz_store.cc:276] 0709 13:01:08.187378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.098387 (+911009us) service_pool.cc:224] Handling call
0709 13:01:09.186183 (+ 87796us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:09.299461 (+113278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.299514 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:09.325834 (+ 26320us) log.cc:844] Serialized 52 byte log entry
0709 13:01:09.460843 (+135009us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:09.460895 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.460968 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:09.567839 (+106871us) raft_consensus.cc:1685] finished
0709 13:01:09.567866 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.568054 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87623,"raft.queue_time_us":101456,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":31685,"spinlock_wait_cycles":28544,"thread_start_us":78946,"threads_started":1,"wal-append.queue_time_us":193271}
I20260709 13:01:09.559520  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:09.590667  1635 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:09.591079  2346 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:01:09.592005  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 197035us
I20260709 13:01:09.594874  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:09.597004  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.640529  1557 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:09.645727  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:09.548933  5043 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.577113  4777 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:01:09.652793  1629 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:09.659145  1629 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:09.608183  5145 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:09.654032  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:09.660791  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.651075  1634 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:09.641677  5406 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:09.656903  1709 raft_consensus.cc:2376] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8. Re-sending same reply.
I20260709 13:01:09.640693  5380 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.677608  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2503) took 1855 ms. Trace:
I20260709 13:01:09.686415  1631 raft_consensus.cc:2410] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:09.687161  1554 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 13:01:09.691392  5119 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.692358  5119 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:09.698410  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 128311us
I20260709 13:01:09.699597  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:09.699833  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:09.703701  5247 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.704672  1553 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:09.705564  5215 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.707068  1561 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:09.707330  1630 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:09.684070  1628 rpcz_store.cc:276] 0709 13:01:07.822406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.998563 (+1176157us) service_pool.cc:224] Handling call
0709 13:01:08.999586 (+  1023us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:09.369726 (+370140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.677233 (+307507us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b540017a7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:09.677536 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849132800}
I20260709 13:01:09.669677  5406 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:09.682981  5433 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:09.734126  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:09.742690  1634 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:09.742358  1709 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:09.725761  2318 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:09.746227  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:09.726361  5406 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:09.676831  4416 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:09.745832  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:09.752053  1553 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:09.746963  1557 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:09.751524  1629 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.755327  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:09.761796  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44031.7us
I20260709 13:01:09.742107  5433 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.738559  5017 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:09.774092  5373 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.777s	user 0.013s	sys 0.002s
I20260709 13:01:09.774426  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:09.775332  5436 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:09.776470  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:09.762889  5328 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:09.757108  5412 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.778041  5328 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:09.778594  5412 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.779839  5328 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.782200  4416 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.784454  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:09.781898  1556 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.786101  4777 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:09.786293  4416 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:09.786655  4777 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.787158  4777 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.788069  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:09.781944  1561 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:09.786649  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 227311us
I20260709 13:01:09.790354  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:09.791241  1561 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:09.792318  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:09.792732  5444 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:01:09.792959  1561 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:09.794584  5371 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:09.795235  4749 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:09.795396  5371 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:09.795984  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 74893.4us
W20260709 13:01:09.797076  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.802634  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 666) took 2219 ms. Trace:
I20260709 13:01:09.804738  4911 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:09.803233  1636 rpcz_store.cc:276] 0709 13:01:07.583315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.625088 (+ 41773us) service_pool.cc:224] Handling call
0709 13:01:07.626031 (+   943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.083817 (+1457786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.802248 (+718431us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001b6f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:09.802540 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3169753344}
I20260709 13:01:09.782497  2346 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:09.805788  1629 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:09.777456  1630 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:09.811534  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:09.806747  1705 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:09.723544  5355 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:01:09.783059  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 682) took 1595 ms (client timeout 2001 ms). Trace:
I20260709 13:01:09.805260  5435 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.823562  1635 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:09.823376  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:09.824545  1709 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:09.824450  4749 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:09.805953  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:09.768196  5433 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.827265  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:09.881132  1557 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:09.830098  5145 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:09.805459  4911 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:09.781320  5412 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:09.837613  1632 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:09.837309  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:09.764951  5438 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.887439  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:09.838567  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:09.839325  5380 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:09.829628  4416 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:01:09.841173  2347 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:01:09.842824  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:09.843679  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.843501  5435 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:09.816359  1631 rpcz_store.cc:269] 0709 13:01:08.187906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.355764 (+167858us) service_pool.cc:224] Handling call
0709 13:01:09.782776 (+1427012us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400139cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:09.782991 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2870170880}
I20260709 13:01:09.844961  5103 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:09.848819  5043 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:09.850497  5371 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:09.782714  5343 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.854477  5203 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:09.855141  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:09.855548  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:09.856211  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:09.857302  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:09.858566  1629 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:09.859619  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:09.808161  5017 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:09.861927  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:09.864589  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:09.865973  2317 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.790s (SENT) [suppressed 17 similar messages]
I20260709 13:01:09.868963  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:09.843729  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:09.844202  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:09.850495  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:09.820011  1527 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:09.852072  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:09.878314  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:10.006145  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 109899us
W20260709 13:01:10.613627  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 708883us
I20260709 13:01:11.104283  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 622) took 1250 ms. Trace:
W20260709 13:01:10.617355  5454 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.181s	user 0.000s	sys 0.002s
I20260709 13:01:09.933725  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:11.107133  1636 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:11.107897  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2521) took 1293 ms. Trace:
I20260709 13:01:11.108309  1636 rpcz_store.cc:276] 0709 13:01:09.814335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.816027 (+  1692us) service_pool.cc:224] Handling call
0709 13:01:11.107810 (+1291783us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:10.683018  1556 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.897s	user 0.015s	sys 0.002s
I20260709 13:01:11.109993  1556 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:10.689961  1708 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:10.697271  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.112674  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 631) took 1242 ms. Trace:
I20260709 13:01:11.113126  1706 rpcz_store.cc:276] 0709 13:01:09.869919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.696859 (+826940us) service_pool.cc:224] Handling call
0709 13:01:11.112499 (+415640us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:11.114096  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2439) took 1437 ms (client timeout 1632 ms). Trace:
W20260709 13:01:11.115101  1556 rpcz_store.cc:269] 0709 13:01:09.677140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.682867 (+  5727us) service_pool.cc:224] Handling call
0709 13:01:11.114000 (+1431133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.116583  5170 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.117424  5170 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:11.119803  5450 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.670s	user 0.009s	sys 0.003s
I20260709 13:01:10.718482  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:11.121744  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2435) took 1553 ms. Trace:
W20260709 13:01:10.731462  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.122153  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2450) took 1373 ms. Trace:
I20260709 13:01:10.740751  1707 raft_consensus.cc:2410] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:11.122740  1561 rpcz_store.cc:276] 0709 13:01:09.748969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.718083 (+969114us) service_pool.cc:224] Handling call
0709 13:01:11.122070 (+403987us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.124248  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:11.125063  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 621) took 1318 ms. Trace:
I20260709 13:01:10.753050  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2431) took 1267 ms. Trace:
I20260709 13:01:10.773116  1703 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
W20260709 13:01:10.806763  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.126822  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 630) took 1264 ms. Trace:
I20260709 13:01:10.863330  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:09.967489  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.127460  1703 rpcz_store.cc:276] 0709 13:01:09.862229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.868495 (+  6266us) service_pool.cc:224] Handling call
0709 13:01:11.126730 (+1258235us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:11.122746  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.873773  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2516) took 1111 ms. Trace:
I20260709 13:01:10.876150  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 627) took 1015 ms. Trace:
I20260709 13:01:11.130429  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2425) took 1071 ms. Trace:
W20260709 13:01:10.892786  1552 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.170s	user 0.006s	sys 0.042s
W20260709 13:01:11.132078  1552 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.414s	user 0.008s	sys 0.044s
W20260709 13:01:10.914449  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 1.06397e+06us
I20260709 13:01:10.914593  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2429) took 3098 ms. Trace:
W20260709 13:01:10.898573  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.914947  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2451) took 1165 ms. Trace:
W20260709 13:01:10.919245  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.920995  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:10.923537  1705 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:11.135006  1557 rpcz_store.cc:276] 0709 13:01:09.749268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.593046 (+843778us) service_pool.cc:224] Handling call
0709 13:01:10.593549 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.593928 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:10.594392 (+   464us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.914540 (+320148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.914604 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:10.914680 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.914748 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.914861 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":317304,"prepare.queue_time_us":235,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"raft.queue_time_us":318218,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":651,"thread_start_us":2274,"threads_started":1,"wal-append.queue_time_us":2715}
W20260709 13:01:11.136894  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 623) took 1282 ms (client timeout 1682 ms). Trace:
W20260709 13:01:11.137321  1705 rpcz_store.cc:269] 0709 13:01:09.855013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.855187 (+   174us) service_pool.cc:224] Handling call
0709 13:01:11.136825 (+1281638us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.129341  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.138404  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.138586  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2423) took 1252 ms. Trace:
I20260709 13:01:11.138995  1703 rpcz_store.cc:276] 0709 13:01:09.886686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.128959 (+1242273us) service_pool.cc:224] Handling call
0709 13:01:11.138507 (+  9548us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.349983  2348 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
W20260709 13:01:09.943892  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.144091  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:10.352723  2346 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:11.146469  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 914) took 1304 ms. Trace:
I20260709 13:01:11.146814  2346 rpcz_store.cc:276] 0709 13:01:09.842306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.842519 (+   213us) service_pool.cc:224] Handling call
0709 13:01:11.146395 (+1303876us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.147372  5170 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:10.289939  2352 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:01:10.390113  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 535195us
I20260709 13:01:10.753901  4749 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:11.155951  4749 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:09.905097  4911 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:10.542742  4777 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.755s	user 0.012s	sys 0.004s
I20260709 13:01:11.162781  4777 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:10.915078  4416 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:10.359802  5017 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:09.934473  5119 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:10.361147  5103 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:10.916486  5145 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:10.925864  5171 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.181895  5145 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.184947  5145 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.185623  2344 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:11.185089  5171 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:10.389564  5203 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:10.878286  5215 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.889878  5233 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:10.931803  5304 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:10.426693  5328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.016s	sys 0.030s
I20260709 13:01:10.379796  5343 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:10.427253  5355 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:10.387130  5361 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.814s	user 0.015s	sys 0.000s
I20260709 13:01:09.856751  5367 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:09.904127  5380 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.344806  5380 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:10.405329  5406 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.642s	user 0.005s	sys 0.009s
I20260709 13:01:11.355712  5406 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:10.913774  5409 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.359760  5409 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:11.360792  5367 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:11.361315  1561 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:10.021812  5423 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.362820  5423 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:11.363499  5409 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:11.364166  5409 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:11.365017  5423 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:11.364729  5409 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:10.388024  5433 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.000s
I20260709 13:01:10.381896  5435 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:10.287546  5449 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.169s	user 0.001s	sys 0.001s
W20260709 13:01:10.816205  5452 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.421s	user 0.004s	sys 0.000s
W20260709 13:01:10.967854  1560 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.224s	user 0.012s	sys 0.037s
I20260709 13:01:11.339927  5233 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 13:01:11.373652  1560 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.633s	user 0.014s	sys 0.037s
I20260709 13:01:11.374022  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1046) took 1818 ms. Trace:
I20260709 13:01:10.978557  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2447) took 1236 ms. Trace:
I20260709 13:01:11.374765  1553 rpcz_store.cc:276] 0709 13:01:09.737145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.851654 (+114509us) service_pool.cc:224] Handling call
0709 13:01:10.973397 (+1121743us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.376289  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:11.374578  2351 rpcz_store.cc:276] 0709 13:01:09.555468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.559640 (+  4172us) service_pool.cc:224] Handling call
0709 13:01:09.560092 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:09.579492 (+ 19400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.579545 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:09.597678 (+ 18133us) log.cc:844] Serialized 26 byte log entry
0709 13:01:09.714709 (+117031us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:09.714827 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.870618 (+155791us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b5400762d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:09.870692 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:11.373605 (+1502913us) raft_consensus.cc:1685] finished
0709 13:01:11.373639 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.373938 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21142,"raft.queue_time_us":794,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":11306,"spinlock_wait_cycles":35985280,"thread_start_us":115427,"threads_started":1,"wal-append.queue_time_us":519634}
I20260709 13:01:10.982268  1630 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:11.377516  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2519) took 1565 ms. Trace:
I20260709 13:01:11.377928  1630 rpcz_store.cc:276] 0709 13:01:09.811815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.815954 (+  4139us) service_pool.cc:224] Handling call
0709 13:01:11.377417 (+1561463us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.983008  5460 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:11.015465  5461 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:11.369796  5145 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:11.379151  5461 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:11.016040  5463 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.020305  5462 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:09.937052  1633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 673) took 2290 ms (client timeout 2233 ms). Trace:
I20260709 13:01:11.374547  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2438) took 1788 ms. Trace:
W20260709 13:01:11.047483  5453 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.659s	user 0.000s	sys 0.005s
I20260709 13:01:11.047741  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:11.048418  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2508) took 1571 ms. Trace:
I20260709 13:01:11.050452  5464 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:11.380174  5170 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:11.383031  5462 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:11.384531  5464 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:11.385100  1635 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
W20260709 13:01:11.065706  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47310.2us
I20260709 13:01:11.385792  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2522) took 1559 ms. Trace:
I20260709 13:01:11.386162  1635 rpcz_store.cc:276] 0709 13:01:09.826836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.047387 (+1220551us) service_pool.cc:224] Handling call
0709 13:01:11.385717 (+338330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.386543  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:11.104825  1709 rpcz_store.cc:276] 0709 13:01:09.854278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.877677 (+ 23399us) service_pool.cc:224] Handling call
0709 13:01:11.104160 (+1226483us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.059188  5465 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:11.114279  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.387967  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.388058  5464 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:11.388161  5461 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:11.384238  5463 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:11.374491  5406 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:11.389438  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:11.390136  5406 raft_consensus.cc:2764] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:01:11.390642  1709 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.391172  5145 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:11.391662  5463 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:11.130611  1701 rpcz_store.cc:276] 0709 13:01:09.860821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.861229 (+   408us) service_pool.cc:224] Handling call
0709 13:01:10.876063 (+1014834us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.392922  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.393720  1701 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:11.395097  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:10.916085  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 1.09975e+06us
I20260709 13:01:11.132710  1555 rpcz_store.cc:276] 0709 13:01:07.816149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.818347 (+  2198us) service_pool.cc:224] Handling call
0709 13:01:07.818587 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.818755 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:07.845035 (+ 26280us) log.cc:844] Serialized 14 byte log entry
0709 13:01:09.699258 (+1854223us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b1800148600. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:01:10.815241 (+1115983us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:10.913753 (+ 98512us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.913885 (+   132us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:10.914014 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.914136 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.914378 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1055984,"prepare.queue_time_us":25892,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":1056339,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":206,"spinlock_wait_cycles":50539136,"thread_start_us":118500,"threads_started":1,"wal-append.queue_time_us":2548762}
I20260709 13:01:11.126454  1558 rpcz_store.cc:276] 0709 13:01:09.482258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.484558 (+  2300us) service_pool.cc:224] Handling call
0709 13:01:09.485023 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.485429 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:09.666234 (+180805us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.748760 (+1082526us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.748933 (+   173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.748977 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:10.749034 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.749132 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.749313 (+   181us) inbound_call.cc:177] Queueing success response
0709 13:01:10.851125 (+101812us) spinlock_profiling.cc:243] Waited 906 us on lock 0x7b4800002958. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:01:11.046641 (+195516us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
Metrics: {"mutex_wait_us":236930,"prepare.queue_time_us":320311,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":99,"raft.queue_time_us":927427,"raft.run_cpu_time_us":4475,"raft.run_wall_time_us":101256,"spinlock_wait_cycles":63442560,"thread_start_us":265501,"threads_started":2,"wal-append.queue_time_us":670295}
I20260709 13:01:11.139221  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2421) took 1270 ms. Trace:
I20260709 13:01:11.397544  1705 rpcz_store.cc:276] 0709 13:01:09.868930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.138061 (+1269131us) service_pool.cc:224] Handling call
0709 13:01:11.139131 (+  1070us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:10.520833  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 655042us
I20260709 13:01:11.122174  1559 rpcz_store.cc:276] 0709 13:01:09.568469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.569807 (+  1338us) service_pool.cc:224] Handling call
0709 13:01:09.570453 (+   646us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:09.597573 (+ 27120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.597615 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:09.598466 (+   851us) log.cc:844] Serialized 52 byte log entry
0709 13:01:09.743082 (+144616us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:09.743210 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.743303 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:11.118645 (+1375342us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:11.121307 (+  2662us) raft_consensus.cc:1685] finished
0709 13:01:11.121335 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.121655 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10579,"raft.queue_time_us":10531,"raft.run_cpu_time_us":713,"raft.run_wall_time_us":74978,"spinlock_wait_cycles":10950912,"thread_start_us":143162,"threads_started":1,"wal-append.queue_time_us":827706}
I20260709 13:01:11.129700  1634 rpcz_store.cc:276] 0709 13:01:09.762565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.769948 (+  7383us) service_pool.cc:224] Handling call
0709 13:01:09.783147 (+ 13199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.873130 (+1089983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.873176 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.873264 (+    88us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:10.873306 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.873393 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.873594 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":869557,"raft.queue_time_us":869838,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":149}
I20260709 13:01:11.142225  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 916) took 1299 ms. Trace:
I20260709 13:01:11.142797  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:11.144860  1703 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:11.405411  1709 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:10.288213  2347 pending_rounds.cc:85] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 7
I20260709 13:01:11.149008  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 915) took 1306 ms. Trace:
I20260709 13:01:11.959481  2352 rpcz_store.cc:276] 0709 13:01:09.842761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.843837 (+  1076us) service_pool.cc:224] Handling call
0709 13:01:11.148922 (+1305085us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.960891  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:11.962743  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:11.963542  2352 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:11.153179  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:11.966015  2346 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:11.179497  1708 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
W20260709 13:01:11.969915  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 626) took 2113 ms (client timeout 2220 ms). Trace:
W20260709 13:01:11.970525  1708 rpcz_store.cc:269] 0709 13:01:09.856482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.859153 (+  2671us) service_pool.cc:224] Handling call
0709 13:01:11.969794 (+2110641us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.971908  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.972886  1708 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:11.177754  5043 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:11.973964  5043 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:11.189601  4416 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:11.190165  5215 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:11.975771  4416 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:11.978128  5215 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:11.194553  5103 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:11.125716  1707 rpcz_store.cc:276] 0709 13:01:09.807149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.807802 (+   653us) service_pool.cc:224] Handling call
0709 13:01:11.124822 (+1317020us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b540016c954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:11.124993 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1967205248}
I20260709 13:01:11.191991  5171 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:11.213666  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 108671us
I20260709 13:01:11.980110  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.980209  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:11.981988  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.982856  1707 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
W20260709 13:01:11.216569  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 112117us
I20260709 13:01:11.983484  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1832) took 1253 ms. Trace:
I20260709 13:01:11.157095  4749 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:11.983927  1707 rpcz_store.cc:276] 0709 13:01:10.730328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.981605 (+1251277us) service_pool.cc:224] Handling call
0709 13:01:11.983412 (+  1807us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.984740  4749 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:11.220072  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:11.985849  2344 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:11.213699  5017 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:01:11.211035  4777 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:11.346454  1710 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:11.225543  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:11.348994  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:11.992292  2345 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:11.216900  5469 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:11.350749  5476 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:11.303684  2318 connection.cc:256] RPC call timeout handler was delayed by -1.1047947569999224s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
I20260709 13:01:11.226024  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:11.994771  5469 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:01:11.341210  5328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.473s	user 0.020s	sys 0.030s
I20260709 13:01:11.346027  5343 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:11.337877  5473 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.310315  5471 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.940779  1704 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:11.998891  5473 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:11.346796  5361 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:11.366416  2349 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:11.370146  5433 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:11.378309  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:11.379021  5460 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:11.021384  5378 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.235605  5304 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:11.384860  5170 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:11.386809  5462 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:11.084940  5451 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.738s	user 0.005s	sys 0.000s
W20260709 13:01:11.399959  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 83758.9us
I20260709 13:01:11.400552  5233 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:11.131377  1702 rpcz_store.cc:276] 0709 13:01:10.058905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.862948 (+804043us) service_pool.cc:224] Handling call
0709 13:01:11.130356 (+267408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.409019  2348 rpcz_store.cc:276] 0709 13:01:09.843183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.843387 (+   204us) service_pool.cc:224] Handling call
0709 13:01:11.142119 (+1298732us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.415133  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:11.420029  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.421880  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:11.425101  5423 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.426942  5409 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:11.380264  5380 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:10.712651  1448 maintenance_manager.cc:419] P 8f622dabf84e4c0595473d2c01f108d0: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033369
W20260709 13:01:11.210109  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 77132us
I20260709 13:01:11.178433  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 524) took 3003 ms. Trace:
I20260709 13:01:12.554701  1552 rpcz_store.cc:276] 0709 13:01:08.174802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.178436 (+  3634us) service_pool.cc:224] Handling call
0709 13:01:08.178762 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:08.179113 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:08.179139 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:09.540536 (+1361397us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b2c00000658. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:01:09.541538 (+  1002us) log.cc:844] Serialized 52 byte log entry
0709 13:01:09.661977 (+120439us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:11.132678 (+1470701us) log.cc:844] Serialized 14 byte log entry
0709 13:01:11.177546 (+ 44868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.177932 (+   386us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:11.177991 (+    59us) raft_consensus.cc:1685] finished
0709 13:01:11.178116 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.178356 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":140430,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":71,"raft.queue_time_us":42428,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":13313,"spinlock_wait_cycles":26592000,"thread_start_us":1578196,"threads_started":3,"wal-append.queue_time_us":239850,"wal-append.run_cpu_time_us":5712,"wal-append.run_wall_time_us":2095743}
I20260709 13:01:11.377061  1553 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:01:12.559284  1553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1880) took 1817 ms (client timeout 1831 ms). Trace:
W20260709 13:01:12.559746  1553 rpcz_store.cc:269] 0709 13:01:10.741525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.375951 (+634426us) service_pool.cc:224] Handling call
0709 13:01:12.559211 (+1183260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.401078  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:12.562898  1555 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
W20260709 13:01:12.563444  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1876) took 1894 ms (client timeout 1934 ms). Trace:
W20260709 13:01:12.563807  1555 rpcz_store.cc:269] 0709 13:01:10.669247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.400693 (+731446us) service_pool.cc:224] Handling call
0709 13:01:12.563373 (+1162680us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.564802  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:11.149029  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:11.422942  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:11.418080  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:11.381615  1560 rpcz_store.cc:276] 0709 13:01:09.585980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.593664 (+  7684us) service_pool.cc:224] Handling call
0709 13:01:09.594174 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.594655 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:09.672978 (+ 78323us) log.cc:844] Serialized 14 byte log entry
0709 13:01:11.374140 (+1701162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:11.374204 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:11.374262 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.374360 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.374487 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":211417,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":54,"thread_start_us":1710498,"threads_started":2,"wal-append.queue_time_us":749089}
I20260709 13:01:12.582316  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:12.583310  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1888) took 1818 ms (client timeout 2167 ms). Trace:
W20260709 13:01:12.583765  1560 rpcz_store.cc:269] 0709 13:01:10.764792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.581970 (+1817178us) service_pool.cc:224] Handling call
0709 13:01:12.583238 (+  1268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.584987  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:11.371646  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:12.586265  1561 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:11.127846  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.335916  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 115876us
I20260709 13:01:11.388309  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:09.934890  1627 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:12.598424  1560 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:12.599709  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:11.383467  1629 rpcz_store.cc:276] 0709 13:01:09.483740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.599325 (+115585us) service_pool.cc:224] Handling call
0709 13:01:09.631395 (+ 32070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.048068 (+1416673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.048346 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:11.381489  1633 rpcz_store.cc:269] 0709 13:01:07.646918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.272986 (+626068us) service_pool.cc:224] Handling call
0709 13:01:09.892926 (+1619940us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b540017b454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:09.936931 (+ 44005us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1597531904}
I20260709 13:01:11.357714  1628 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:12.110399  1556 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
W20260709 13:01:12.608556  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1890) took 1829 ms (client timeout 1531 ms). Trace:
W20260709 13:01:12.608991  1556 rpcz_store.cc:269] 0709 13:01:10.779447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.125938 (+346491us) service_pool.cc:224] Handling call
0709 13:01:12.608339 (+1482401us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b5400230854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:12.608472 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1611844480}
I20260709 13:01:12.610327  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:12.611224  1556 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:12.611641  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1877) took 1942 ms (client timeout 2424 ms). Trace:
W20260709 13:01:12.227101  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.009s	sys 0.008s
I20260709 13:01:12.266979  1554 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:12.612493  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.264s	user 0.010s	sys 0.009s
W20260709 13:01:12.252709  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2420) took 2384 ms (client timeout 1858 ms). Trace:
W20260709 13:01:12.614576  1710 rpcz_store.cc:269] 0709 13:01:09.868553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.892654 (+ 24101us) service_pool.cc:224] Handling call
0709 13:01:12.252425 (+2359771us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540016bf54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:12.252599 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501184384}
I20260709 13:01:12.616223  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:12.026625  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:12.020424  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1830) took 1290 ms. Trace:
W20260709 13:01:12.618767  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2433) took 1890 ms (client timeout 2134 ms). Trace:
I20260709 13:01:12.052868  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2438) took 1305 ms. Trace:
W20260709 13:01:11.589377  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.617089  1710 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
W20260709 13:01:12.028208  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45021.2us
W20260709 13:01:12.100800  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 60512.5us
W20260709 13:01:12.283461  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.018483  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2430) took 1291 ms. Trace:
I20260709 13:01:12.082892  1701 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
W20260709 13:01:11.999393  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 625) took 2143 ms (client timeout 2060 ms). Trace:
I20260709 13:01:12.021499  1706 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:11.985195  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:11.942782  1731 debug-util.cc:398] Leaking SignalData structure 0x7b0800107080 after lost signal to thread 434
W20260709 13:01:11.887568  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 479929us
W20260709 13:01:12.099992  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 97473us
W20260709 13:01:12.703372  2318 connection.cc:256] RPC call timeout handler was delayed by -1.3902454770000077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
W20260709 13:01:12.703810  2318 connection.cc:256] RPC call timeout handler was delayed by -1.3281454770000209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 13:01:12.704195  2318 connection.cc:256] RPC call timeout handler was delayed by -1.3182454770000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
W20260709 13:01:12.704481  2318 connection.cc:256] RPC call timeout handler was delayed by -1.3182454770000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
W20260709 13:01:12.704747  2318 connection.cc:256] RPC call timeout handler was delayed by -1.2840454770000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
W20260709 13:01:12.705020  2318 connection.cc:256] RPC call timeout handler was delayed by -1.2840454770000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
W20260709 13:01:11.704061  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.092065  2344 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:12.084899  2345 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:12.087291  2347 pending_rounds.cc:107] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:01:12.720697  2347 raft_consensus.cc:2889] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:12.232638  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 627) took 1009 ms. Trace:
I20260709 13:01:12.724037  2349 rpcz_store.cc:276] 0709 13:01:11.222688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.226247 (+  3559us) service_pool.cc:224] Handling call
0709 13:01:11.226596 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.004184 (+777588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.232209 (+228025us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540048ab54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:12.232488 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320396160}
I20260709 13:01:12.005967  2351 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
W20260709 13:01:12.728073  2351 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 635) took 1428 ms (client timeout 1748 ms). Trace:
W20260709 13:01:12.728643  2351 rpcz_store.cc:269] 0709 13:01:11.300278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.378020 (+ 77742us) service_pool.cc:224] Handling call
0709 13:01:12.727971 (+1349951us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.965284  2352 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:12.115432  4416 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:12.721953  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 913) took 2883 ms. Trace:
I20260709 13:01:12.007644  5103 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:01:12.749969  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:11.538615  5145 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:01:12.280367  5170 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:12.038823  5361 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:12.789242  5361 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:12.790053  5361 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:12.791402  5361 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:12.017313  5304 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:12.016673  5380 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:12.013969  5423 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:10.084969  5427 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.301s
I20260709 13:01:12.012531  5233 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:12.824402  5233 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:12.038635  5343 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:12.024354  5469 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:11.540980  5355 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:01:12.013675  5378 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:12.854296  5472 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:12.854859  5472 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:12.856019  5472 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:12.046386  5433 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:12.037894  5435 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:12.859205  5378 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:12.859351  5435 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:12.859992  5435 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:12.047411  5460 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:12.049434  5473 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:11.538221  5461 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:12.866855  5461 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:12.007098  5476 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:12.872257  5476 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:12.033595  5468 consensus_queue.cc:1175] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba0548b7d2a94e8598475a46c279fe0e" responder_term: 7 status { last_received { term: 7 index: 4 } last_committed_idx: 1 last_received_current_leader { term: 7 index: 4 } }
I20260709 13:01:12.042168  5478 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:11.998891  5471 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:12.884121  5471 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:12.507130  1525 connection.cc:256] RPC call timeout handler was delayed by -1.6132142339999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
W20260709 13:01:12.884861  1525 connection.cc:256] RPC call timeout handler was delayed by -1.5770835359999751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
W20260709 13:01:12.885237  1525 connection.cc:256] RPC call timeout handler was delayed by -1.4764142339999466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 13:01:12.885558  1525 connection.cc:256] RPC call timeout handler was delayed by -1.4296142339999847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1829s for a response
W20260709 13:01:12.885850  1525 connection.cc:256] RPC call timeout handler was delayed by -1.3648142339999936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
W20260709 13:01:12.886152  1525 connection.cc:256] RPC call timeout handler was delayed by -1.3403835359999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 13:01:12.886437  1525 connection.cc:256] RPC call timeout handler was delayed by -1.3130603349999888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
W20260709 13:01:12.886719  1525 connection.cc:256] RPC call timeout handler was delayed by -1.1947142340000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
I20260709 13:01:12.507601  5462 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.889259  5462 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:12.892017  5462 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:12.520648  4749 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.893011  4749 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:12.527084  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.536235  5409 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:12.896332  5409 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:12.896656  5505 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.570279  1557 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 5.
I20260709 13:01:12.897789  4749 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:12.897864  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1887) took 2133 ms (client timeout 1627 ms). Trace:
W20260709 13:01:12.898545  1557 rpcz_store.cc:269] 0709 13:01:10.764063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.137442 (+373379us) service_pool.cc:224] Handling call
0709 13:01:12.897769 (+1760327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.898944  5476 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.899653  5476 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:12.899900  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:12.900980  1557 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:12.580991  1559 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:12.898980  5409 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:12.902369  5461 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:12.903532  5472 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:12.561021  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:12.906479  1553 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
W20260709 13:01:12.907214  1553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1894) took 2120 ms (client timeout 2031 ms). Trace:
W20260709 13:01:12.907673  1553 rpcz_store.cc:269] 0709 13:01:10.786747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.560719 (+1773972us) service_pool.cc:224] Handling call
0709 13:01:12.907041 (+346322us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.908923  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:12.597841  1555 raft_consensus.cc:2368] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9.
W20260709 13:01:12.909936  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1891) took 2128 ms (client timeout 2093 ms). Trace:
W20260709 13:01:12.910372  1555 rpcz_store.cc:269] 0709 13:01:10.781078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.564502 (+1783424us) service_pool.cc:224] Handling call
0709 13:01:12.909848 (+345346us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.911602  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:12.912715  1555 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:12.600561  1552 raft_consensus.cc:2410] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:01:12.913894  1552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1892) took 2128 ms (client timeout 2204 ms). Trace:
W20260709 13:01:12.914292  1552 rpcz_store.cc:269] 0709 13:01:10.785138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.599400 (+1814262us) service_pool.cc:224] Handling call
0709 13:01:12.913830 (+314430us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:12.600584  1627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2513) took 3075 ms (client timeout 1632 ms). Trace:
I20260709 13:01:12.915366  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:12.915642  1627 rpcz_store.cc:269] 0709 13:01:09.525967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.536969 (+ 11002us) service_pool.cc:224] Handling call
0709 13:01:12.600306 (+3063337us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54002da0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:12.600470 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":846844160}
W20260709 13:01:12.599004  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1882) took 1856 ms (client timeout 2204 ms). Trace:
W20260709 13:01:12.916757  1560 rpcz_store.cc:269] 0709 13:01:10.742109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.584556 (+1842447us) service_pool.cc:224] Handling call
0709 13:01:12.598937 (+ 14381us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.917829  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:12.919075  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:12.919662  1560 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.920188  5476 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:12.611958  1556 rpcz_store.cc:269] 0709 13:01:10.669571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.609932 (+1940361us) service_pool.cc:224] Handling call
0709 13:01:12.611586 (+  1654us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:12.915393  1458 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 272) took 12563 ms (client timeout 15000 ms). Trace:
I20260709 13:01:12.922351  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:12.923199  1556 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:12.924170  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:12.617872  1561 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
W20260709 13:01:12.619228  1702 rpcz_store.cc:269] 0709 13:01:10.728030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.022696 (+1294666us) service_pool.cc:224] Handling call
0709 13:01:12.618682 (+595986us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:12.925789  1561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1879) took 2255 ms (client timeout 1833 ms). Trace:
W20260709 13:01:12.926179  1561 rpcz_store.cc:269] 0709 13:01:10.670515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.371290 (+700775us) service_pool.cc:224] Handling call
0709 13:01:12.925705 (+1554415us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.926270  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:12.927295  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1849) took 1693 ms (client timeout 1852 ms). Trace:
I20260709 13:01:12.927254  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:12.927866  1702 rpcz_store.cc:269] 0709 13:01:11.234200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.926003 (+1691803us) service_pool.cc:224] Handling call
0709 13:01:12.927237 (+  1234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.928208  1561 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:12.928833  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:12.613157  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2440) took 2923 ms (client timeout 2191 ms). Trace:
W20260709 13:01:12.929633  1554 rpcz_store.cc:269] 0709 13:01:09.689997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.717469 (+ 27472us) service_pool.cc:224] Handling call
0709 13:01:12.612889 (+2895420us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b54003205d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:12.613042 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5760329984}
I20260709 13:01:12.930877  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:12.931495  1554 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:12.931659  5505 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:12.932420  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:12.933077  1554 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:12.619871  1708 rpcz_store.cc:276] 0709 13:01:10.729355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.971533 (+1242178us) service_pool.cc:224] Handling call
0709 13:01:12.034585 (+ 63052us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.934285  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:12.934438  5505 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:12.935190  1708 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:12.935703  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1831) took 2205 ms (client timeout 2424 ms). Trace:
W20260709 13:01:12.936153  1708 rpcz_store.cc:269] 0709 13:01:10.730007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.933961 (+2203954us) service_pool.cc:224] Handling call
0709 13:01:12.935635 (+  1674us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.618963  1703 rpcz_store.cc:276] 0709 13:01:10.729714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.419635 (+689921us) service_pool.cc:224] Handling call
0709 13:01:12.020332 (+600697us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:12.037142  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 53018.8us
W20260709 13:01:12.624902  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1835) took 1392 ms (client timeout 1831 ms). Trace:
W20260709 13:01:12.940527  1710 rpcz_store.cc:269] 0709 13:01:11.228991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.615806 (+1386815us) service_pool.cc:224] Handling call
0709 13:01:12.620977 (+  5171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.629997  1628 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:12.941588  1553 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
W20260709 13:01:12.942241  1553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1884) took 2181 ms (client timeout 2840 ms). Trace:
W20260709 13:01:12.942857  1628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2523) took 2138 ms (client timeout 1915 ms). Trace:
I20260709 13:01:12.944200  5343 consensus_queue.cc:1059] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:01:12.655817  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 1.2675e+06us
W20260709 13:01:12.944829  1628 rpcz_store.cc:269] 0709 13:01:10.804469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.920355 (+115886us) service_pool.cc:224] Handling call
0709 13:01:12.942667 (+2022312us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540017c854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:12.942780 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508388736}
I20260709 13:01:12.945665  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:12.937309  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:12.655756  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 61304.3us
W20260709 13:01:12.664647  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1834) took 1933 ms (client timeout 1833 ms). Trace:
I20260709 13:01:12.677227  1526 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:12.946697  1708 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
W20260709 13:01:12.949600  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2443) took 1596 ms (client timeout 1858 ms). Trace:
W20260709 13:01:12.949972  1708 rpcz_store.cc:269] 0709 13:01:11.353299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.937021 (+1583722us) service_pool.cc:224] Handling call
0709 13:01:12.949515 (+ 12494us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:12.676178  1704 rpcz_store.cc:269] 0709 13:01:09.855928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.856777 (+   849us) service_pool.cc:224] Handling call
0709 13:01:11.999213 (+2142436us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.951660  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:12.952289  1708 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:12.952557  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:12.953516  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2435) took 2224 ms (client timeout 2649 ms). Trace:
I20260709 13:01:12.953653  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:12.682189  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2424) took 2731 ms (client timeout 2063 ms). Trace:
W20260709 13:01:12.688584  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50276.3us
I20260709 13:01:12.954602  1554 raft_consensus.cc:2410] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:12.955545  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:12.956094  1554 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:12.956544  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1878) took 2286 ms. Trace:
I20260709 13:01:12.956588  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:12.956944  1554 rpcz_store.cc:276] 0709 13:01:10.670065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.955260 (+2285195us) service_pool.cc:224] Handling call
0709 13:01:12.956491 (+  1231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.707592  5494 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:12.708624  5492 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.970693  5492 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:12.682875  5488 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:12.974068  5488 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:12.976807  5488 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:12.689462  5490 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.714368  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:12.977865  2346 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:12.977716  5490 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:12.979315  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:12.980055  2346 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.980485  5490 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:12.981176  5490 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:12.953907  1704 rpcz_store.cc:269] 0709 13:01:10.728571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.952252 (+2223681us) service_pool.cc:224] Handling call
0709 13:01:12.953415 (+  1163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.982888  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:12.983533  1704 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:01:12.985074  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1841) took 1753 ms (client timeout 2167 ms). Trace:
W20260709 13:01:12.714433  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 72004.3us
I20260709 13:01:12.715310  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:12.986793  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:12.987475  2348 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
W20260709 13:01:12.718464  2345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 628) took 1495 ms (client timeout 1915 ms). Trace:
W20260709 13:01:12.988404  2345 rpcz_store.cc:269] 0709 13:01:11.223096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.226890 (+  3794us) service_pool.cc:224] Handling call
0709 13:01:12.718396 (+1491506us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.988644  1552 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:12.925364  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:12.989568  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:12.990127  1556 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
W20260709 13:01:12.719209  2344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 624) took 1501 ms (client timeout 1776 ms). Trace:
I20260709 13:01:12.725903  5495 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:12.197052  5043 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:12.730445  2317 connection.cc:256] RPC call timeout handler was delayed by -2.8218933800000059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2001s for a response
I20260709 13:01:12.732270  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 630) took 1508 ms. Trace:
I20260709 13:01:12.738106  4416 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:13.013479  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51885.6us
I20260709 13:01:12.747210  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:12.752247  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:12.774165  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50313us
I20260709 13:01:13.015323  2352 rpcz_store.cc:276] 0709 13:01:11.223878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.964314 (+740436us) service_pool.cc:224] Handling call
0709 13:01:11.964833 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.731782 (+766949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.732168 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:12.782459  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 77035.5us
I20260709 13:01:12.815129  5423 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:12.821566  5501 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.821561  1455 catalog_manager.cc:5697] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 4 to 6, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:01:12.826182  5233 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.841717  5355 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.860918  5435 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:12.863634  5487 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:12.881131  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:13.025915  2318 connection.cc:256] RPC call timeout handler was delayed by -1.1427454770000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1957s for a response
I20260709 13:01:12.896791  1558 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:12.547869  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.021060  1675 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.027117  1709 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:01:12.938097  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.000232  5495 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:12.943763  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:12.929401  1702 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:12.952764  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2442) took 1615 ms (client timeout 1981 ms). Trace:
W20260709 13:01:13.038887  1708 rpcz_store.cc:269] 0709 13:01:11.337939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.951259 (+1613320us) service_pool.cc:224] Handling call
0709 13:01:12.952710 (+  1451us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:12.684592  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1840) took 1453 ms (client timeout 1627 ms). Trace:
W20260709 13:01:12.694165  1731 debug-util.cc:398] Leaking SignalData structure 0x7b080003fdc0 after lost signal to thread 1493
W20260709 13:01:12.942628  1553 rpcz_store.cc:269] 0709 13:01:10.761270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.908576 (+2147306us) service_pool.cc:224] Handling call
0709 13:01:12.942145 (+ 33569us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.041026  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2452) took 2373 ms. Trace:
I20260709 13:01:12.947644  5469 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:12.948755  1701 rpcz_store.cc:269] 0709 13:01:10.730939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.396324 (+665385us) service_pool.cc:224] Handling call
0709 13:01:12.664533 (+1268209us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:12.954365  1706 rpcz_store.cc:269] 0709 13:01:09.950817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.113946 (+1163129us) service_pool.cc:224] Handling call
0709 13:01:12.681985 (+1568039us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b540016c454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:12.682116 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455197952}
I20260709 13:01:12.948607  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:13.047561  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 99015.2us
I20260709 13:01:13.048362  1602 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:12.961977  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 284853us
I20260709 13:01:12.962546  5409 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:12.963104  5478 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:12.922297  1458 rpcz_store.cc:269] 0709 13:01:00.451714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.462036 (+ 10322us) service_pool.cc:224] Handling call
0709 13:01:12.915150 (+12453114us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:09.441932 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:09.443171 (+  1239us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:09.443232 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:01:09.443268 (+    36us) tablet.cc:662] Decoding operations
0709 13:01:09.517322 (+ 74054us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:09.526623 (+  9301us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:09.527608 (+   985us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:01:09.689797 (+162189us) tablet.cc:701] Row locks acquired
0709 13:01:09.689855 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:01:09.690111 (+   256us) write_op.cc:270] Start()
0709 13:01:09.690307 (+   196us) write_op.cc:276] Timestamp: P: 1783602069690026 usec, L: 0
0709 13:01:09.690331 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:01:09.691334 (+  1003us) log.cc:844] Serialized 6400 byte log entry
0709 13:01:11.082722 (+1391388us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:12.089152 (+1006430us) op_driver.cc:464] REPLICATION: finished
0709 13:01:12.117306 (+ 28154us) write_op.cc:301] APPLY: starting
0709 13:01:12.117649 (+   343us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:12.118232 (+   583us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:12.118294 (+    62us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:12.120938 (+  2644us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:12.122619 (+  1681us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:01:12.122683 (+    64us) write_op.cc:312] APPLY: finished
0709 13:01:12.123282 (+   599us) log.cc:844] Serialized 113 byte log entry
0709 13:01:12.124217 (+   935us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:12.124463 (+   246us) write_op.cc:454] Released schema lock
0709 13:01:12.124563 (+   100us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:12.125084 (+   521us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:12.125108 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7508,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":27729,"apply.run_cpu_time_us":5618,"apply.run_wall_time_us":7783,"num_ops":16,"prepare.queue_time_us":458797,"prepare.run_cpu_time_us":16749,"prepare.run_wall_time_us":310660,"raft.queue_time_us":77257,"raft.run_cpu_time_us":13047,"raft.run_wall_time_us":27443,"replication_time_us":2397483,"spinlock_wait_cycles":48385920,"thread_start_us":465185,"threads_started":4,"wal-append.queue_time_us":330458}]]}
I20260709 13:01:12.714368  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:12.990803  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1896) took 2020 ms (client timeout 2436 ms). Trace:
W20260709 13:01:13.054060  1557 rpcz_store.cc:269] 0709 13:01:10.881448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.899533 (+2018085us) service_pool.cc:224] Handling call
0709 13:01:12.901427 (+  1894us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.929163  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:13.054476  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1847) took 1820 ms (client timeout 2048 ms). Trace:
W20260709 13:01:12.987504  1704 rpcz_store.cc:269] 0709 13:01:11.231508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.982543 (+1751035us) service_pool.cc:224] Handling call
0709 13:01:12.983985 (+  1442us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:13.054857  1710 rpcz_store.cc:269] 0709 13:01:11.233663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.943468 (+1709805us) service_pool.cc:224] Handling call
0709 13:01:13.054399 (+110931us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.993427  2346 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:12.994232  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:13.056105  2348 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:12.947438  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:12.988179  1676 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:12.994077  2344 rpcz_store.cc:269] 0709 13:01:11.215734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.217530 (+  1796us) service_pool.cc:224] Handling call
0709 13:01:12.716698 (+1499168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.000556  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:12.991807  5490 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:13.068277  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2454) took 2399 ms. Trace:
I20260709 13:01:13.068164  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.068785  1553 rpcz_store.cc:276] 0709 13:01:10.668684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.057176 (+2388492us) service_pool.cc:224] Handling call
0709 13:01:13.067782 (+ 10606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.067965 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.067987 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.068025 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:13.068065 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.068115 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.068219 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.069682  1629 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:13.046557  5469 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:12.734956  2347 rpcz_store.cc:276] 0709 13:01:09.838235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.840365 (+  2130us) service_pool.cc:224] Handling call
0709 13:01:09.840771 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.721556 (+2880785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.721849 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.081701  1560 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
W20260709 13:01:13.026523  2318 connection.cc:256] RPC call timeout handler was delayed by -1.1427454770000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1957s for a response
I20260709 13:01:12.649336  1705 rpcz_store.cc:276] 0709 13:01:10.727301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.413699 (+686398us) service_pool.cc:224] Handling call
0709 13:01:12.018402 (+604703us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:13.082481  2318 connection.cc:256] RPC call timeout handler was delayed by -1.050045477000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
W20260709 13:01:13.027690  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2439) took 1817 ms (client timeout 1715 ms). Trace:
I20260709 13:01:13.021816  5355 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:13.084579  5435 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:13.084843  1557 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:01:13.085561  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1886) took 2322 ms. Trace:
I20260709 13:01:13.085883  1557 rpcz_store.cc:276] 0709 13:01:10.763063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.070727 (+2307664us) service_pool.cc:224] Handling call
0709 13:01:13.084607 (+ 13880us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.085221 (+   614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.085498 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.086359  5355 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:13.026335  2353 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:13.087404  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:13.088531  1634 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:13.040151  1707 rpcz_store.cc:269] 0709 13:01:11.230712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.984872 (+754160us) service_pool.cc:224] Handling call
0709 13:01:12.684471 (+699599us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.958441  5427 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:13.091157  5427 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:13.042346  1731 debug-util.cc:398] Leaking SignalData structure 0x7b080018adc0 after lost signal to thread 1585
W20260709 13:01:13.091739  1731 debug-util.cc:398] Leaking SignalData structure 0x7b08001c00c0 after lost signal to thread 1586
W20260709 13:01:13.093114  1731 debug-util.cc:398] Leaking SignalData structure 0x7b080025df60 after lost signal to thread 1660
I20260709 13:01:13.091300  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:13.050112  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:13.047343  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1848) took 1813 ms (client timeout 2031 ms). Trace:
W20260709 13:01:13.095451  1703 rpcz_store.cc:269] 0709 13:01:11.233930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.937838 (+1703908us) service_pool.cc:224] Handling call
0709 13:01:13.047271 (+109433us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.096499  5495 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:13.096738  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:13.097174  1554 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:01:13.099110  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.057966  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:13.100198  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2444) took 1744 ms (client timeout 2175 ms). Trace:
W20260709 13:01:13.100579  1710 rpcz_store.cc:269] 0709 13:01:11.356036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.057695 (+1701659us) service_pool.cc:224] Handling call
0709 13:01:13.100129 (+ 42434us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.101404  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:01:13.055608  5505 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:13.102288  1710 raft_consensus.cc:606] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:13.102421  5505 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.102901  5505 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.103080  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43592 (request call id 26) took 1907 ms. Trace:
I20260709 13:01:13.059075  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.103480  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2422) took 3234 ms. Trace:
I20260709 13:01:13.103812  1633 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:13.103950  1701 rpcz_store.cc:276] 0709 13:01:09.869275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.102446 (+3233171us) service_pool.cc:224] Handling call
0709 13:01:13.102766 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.103110 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.103140 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.103189 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:13.103233 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.103293 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.103421 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.063802  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:13.066252  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.025748  5501 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:13.105928  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1837) took 1876 ms (client timeout 2203 ms). Trace:
W20260709 13:01:13.066854  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 119485us
I20260709 13:01:13.106974  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:13.067404  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2455) took 2398 ms. Trace:
I20260709 13:01:13.058586  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:13.107652  1630 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.019958  5492 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:13.108878  5427 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.072089  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1883) took 2320 ms. Trace:
I20260709 13:01:13.109552  1555 rpcz_store.cc:276] 0709 13:01:10.751893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.913465 (+2161572us) service_pool.cc:224] Handling call
0709 13:01:13.072021 (+158556us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.011281  1559 raft_consensus.cc:2410] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:01:13.111312  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1895) took 2232 ms (client timeout 1852 ms). Trace:
I20260709 13:01:13.111858  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1875) took 3347 ms. Trace:
I20260709 13:01:13.112270  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.113000  1629 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:13.066252  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.071791  5490 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:13.113966  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1836) took 1884 ms (client timeout 2204 ms). Trace:
I20260709 13:01:13.053663  5472 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:13.114348  1704 rpcz_store.cc:269] 0709 13:01:11.229355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.055397 (+1826042us) service_pool.cc:224] Handling call
0709 13:01:13.113901 (+ 58504us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46976}
I20260709 13:01:13.114800  5472 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:13.116230  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 632) took 3246 ms. Trace:
I20260709 13:01:13.105217  1628 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.120368  1701 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:01:13.119710  5487 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:13.121029  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2429) took 2394 ms. Trace:
W20260709 13:01:13.082868  2318 connection.cc:256] RPC call timeout handler was delayed by -1.050045477000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
W20260709 13:01:13.083037  1709 rpcz_store.cc:269] 0709 13:01:11.210323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.547500 (+1337177us) service_pool.cc:224] Handling call
0709 13:01:13.027615 (+480115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.071237  1455 catalog_manager.cc:5697] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 3 to 6, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 6 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:13.115340  5472 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:13.093587  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:13.106343  1708 rpcz_store.cc:269] 0709 13:01:11.229718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.053997 (+1824279us) service_pool.cc:224] Handling call
0709 13:01:13.105859 (+ 51862us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39552}
I20260709 13:01:13.116721  1704 rpcz_store.cc:276] 0709 13:01:09.870253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.115283 (+3245030us) service_pool.cc:224] Handling call
0709 13:01:13.115650 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.115858 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.115885 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.115931 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:13.115963 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.116018 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.116147 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:13.121842  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 71769us
I20260709 13:01:13.121455  1701 rpcz_store.cc:276] 0709 13:01:10.727021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.119651 (+2392630us) service_pool.cc:224] Handling call
0709 13:01:13.119988 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.120757 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.120969 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.159705  1701 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:13.122030  5492 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:13.160472  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2441) took 1932 ms. Trace:
I20260709 13:01:13.160822  5492 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.161618  5492 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.062383  1675 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.094162  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:13.094516  1731 debug-util.cc:398] Leaking SignalData structure 0x7b08003f4620 after lost signal to thread 1734
I20260709 13:01:13.122763  1709 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:13.072290  5378 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:13.125308  1707 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:13.164508  5378 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:13.164765  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2432) took 2436 ms. Trace:
I20260709 13:01:13.048106  5423 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.107779  1561 rpcz_store.cc:276] 0709 13:01:10.668941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.066469 (+2397528us) service_pool.cc:224] Handling call
0709 13:01:13.066789 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.067033 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.067058 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.067111 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:13.067143 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.067200 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.067324 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:13.111701  1559 rpcz_store.cc:269] 0709 13:01:10.879420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.413213 (+533793us) service_pool.cc:224] Handling call
0709 13:01:13.111227 (+1698014us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.098870  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.135910  5508 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:13.171555  5508 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.172170  5508 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.127040  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.173939  5423 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:13.171293  1709 rpcz_store.cc:276] 0709 13:01:10.727791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.121998 (+2394207us) service_pool.cc:224] Handling call
0709 13:01:13.122356 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.164394 (+ 42038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.164653 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.125665  1705 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.132980  1557 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:13.150997  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.178146  1634 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:13.178341  1561 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:13.163652  1635 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:13.179489  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:13.112306  1554 rpcz_store.cc:276] 0709 13:01:09.764131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.969853 (+3205722us) service_pool.cc:224] Handling call
0709 13:01:12.970292 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.097664 (+127372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.111520 (+ 13856us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400208354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:13.111763 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256940800}
I20260709 13:01:13.174515  1633 raft_consensus.cc:2410] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:13.180580  5409 raft_consensus.cc:2764] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:01:13.164297  1731 debug-util.cc:398] Leaking SignalData structure 0x7b08003f5980 after lost signal to thread 1735
I20260709 13:01:13.109572  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:13.164894  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1851) took 1930 ms (client timeout 2436 ms). Trace:
W20260709 13:01:13.182209  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 72764.7us
W20260709 13:01:13.182164  1707 rpcz_store.cc:269] 0709 13:01:11.267297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.091631 (+1824334us) service_pool.cc:224] Handling call
0709 13:01:13.164825 (+ 73194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:13.182814  1731 debug-util.cc:398] Leaking SignalData structure 0x7b080000f0c0 after lost signal to thread 2377
W20260709 13:01:13.183151  1731 debug-util.cc:398] Leaking SignalData structure 0x7b080000f5a0 after lost signal to thread 2378
I20260709 13:01:13.183723  1707 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:13.184368  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1843) took 1952 ms. Trace:
I20260709 13:01:13.184688  1707 rpcz_store.cc:276] 0709 13:01:11.232190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.183127 (+1950937us) service_pool.cc:224] Handling call
0709 13:01:13.183508 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.184115 (+   607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.184312 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.103497  1710 rpcz_store.cc:276] 0709 13:01:11.204855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.101242 (+1896387us) service_pool.cc:224] Handling call
0709 13:01:13.102970 (+  1728us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.185729  5355 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.186308  5355 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.186702  5355 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:13.187088  1710 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:01:13.187769  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1845) took 1955 ms. Trace:
I20260709 13:01:13.188081  1710 rpcz_store.cc:276] 0709 13:01:11.232977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.186419 (+1953442us) service_pool.cc:224] Handling call
0709 13:01:13.186758 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.187496 (+   738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.187710 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.165009  5378 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.126261  5478 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:13.121690  2318 connection.cc:256] RPC call timeout handler was delayed by -1.0491454770000246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
I20260709 13:01:13.190832  5478 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:13.191128  2318 connection.cc:256] RPC call timeout handler was delayed by -1.0149454770000261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
I20260709 13:01:13.191267  5478 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.192931  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.193679  5103 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:13.177073  1709 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:13.211650  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1839) took 1981 ms. Trace:
I20260709 13:01:13.212284  1709 rpcz_store.cc:276] 0709 13:01:11.230388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.176155 (+1945767us) service_pool.cc:224] Handling call
0709 13:01:13.176541 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.211294 (+ 34753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.211571 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:13.164013  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 101728us
I20260709 13:01:13.207438  1555 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:13.083242  1560 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:13.215258  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1889) took 2447 ms. Trace:
I20260709 13:01:13.215858  1560 rpcz_store.cc:276] 0709 13:01:10.767525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.082550 (+2315025us) service_pool.cc:224] Handling call
0709 13:01:13.082874 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.214842 (+131968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.215064 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.041697  1556 rpcz_store.cc:276] 0709 13:01:10.668157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.990745 (+2322588us) service_pool.cc:224] Handling call
0709 13:01:13.001410 (+ 10665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.005656 (+  4246us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:13.014416 (+  8760us) log.cc:844] Serialized 14 byte log entry
0709 13:01:13.040646 (+ 26230us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.040698 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:13.040744 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.040821 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.040966 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9336,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":95,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"thread_start_us":25327,"threads_started":1}
I20260709 13:01:13.180671  5423 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:13.217950  2317 connection.cc:256] RPC call timeout handler was delayed by -1.9481580130000111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
W20260709 13:01:13.230392  2317 connection.cc:256] RPC call timeout handler was delayed by -1.8617580129999851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
W20260709 13:01:13.231034  2317 connection.cc:256] RPC call timeout handler was delayed by -1.377558012999998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
W20260709 13:01:13.231426  2317 connection.cc:256] RPC call timeout handler was delayed by -1.3541902590000063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 13:01:13.231814  2317 connection.cc:256] RPC call timeout handler was delayed by -1.0409902590000115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
I20260709 13:01:13.219250  1559 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:13.219039  5103 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:13.160966  1701 rpcz_store.cc:276] 0709 13:01:11.228621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.158938 (+1930317us) service_pool.cc:224] Handling call
0709 13:01:13.159392 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.160179 (+   787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.160407 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.123242  1455 catalog_manager.cc:5697] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:13.220250  1554 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:13.226688  1710 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:13.237530  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1850) took 2003 ms. Trace:
I20260709 13:01:13.238101  1710 rpcz_store.cc:276] 0709 13:01:11.234466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.188920 (+1954454us) service_pool.cc:224] Handling call
0709 13:01:13.210273 (+ 21353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.237156 (+ 26883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.237436 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.245061  1455 catalog_manager.cc:5697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:13.248097  1633 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:13.251204  1558 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
W20260709 13:01:13.253039  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.262553  2318 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:13.269352  1601 connection.cc:256] RPC call timeout handler was delayed by -2.2087380689999918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 13:01:13.244297  5495 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:13.269878  1601 connection.cc:256] RPC call timeout handler was delayed by -2.1223380690000226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 13:01:13.275938  1601 connection.cc:256] RPC call timeout handler was delayed by -2.1223380690000226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
I20260709 13:01:13.268489  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.274610  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:13.272745  1455 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 3 to 5, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 5 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:13.274060  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 94664.4us
I20260709 13:01:13.288192  1526 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.276513  5423 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.295984  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2453) took 2627 ms. Trace:
I20260709 13:01:13.276940  5513 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:13.278990  5103 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:13.300107  1558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1885) took 2538 ms (client timeout 1924 ms). Trace:
W20260709 13:01:13.300976  1558 rpcz_store.cc:269] 0709 13:01:10.762209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.405413 (+643204us) service_pool.cc:224] Handling call
0709 13:01:13.298860 (+1893447us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b5400312254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:13.300015 (+  1155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749229952}
W20260709 13:01:13.259545  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 248381us
I20260709 13:01:13.286527  1557 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:13.285964  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.276459  1628 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:13.308686  1705 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:01:13.311707  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 323572us
I20260709 13:01:13.299412  1561 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:01:13.299527  1657 debug-util.cc:398] Leaking SignalData structure 0x7b080005b8c0 after lost signal to thread 431
W20260709 13:01:13.319700  1657 debug-util.cc:398] Leaking SignalData structure 0x7b08001db9c0 after lost signal to thread 434
I20260709 13:01:13.302271  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:13.275763  5495 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.312353  1630 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:13.316715  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2437) took 2587 ms. Trace:
I20260709 13:01:13.323659  1708 rpcz_store.cc:276] 0709 13:01:10.729068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.126086 (+2397018us) service_pool.cc:224] Handling call
0709 13:01:13.126447 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.126842 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.126874 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:13.127490 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 13:01:13.270189 (+142699us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:13.270227 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.270301 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:13.316381 (+ 46080us) raft_consensus.cc:1685] finished
0709 13:01:13.316405 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.316596 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":149862,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":142321,"threads_started":1,"wal-append.queue_time_us":186282}
I20260709 13:01:13.295737  5423 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.296893  1552 rpcz_store.cc:276] 0709 13:01:10.668438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.997385 (+2328947us) service_pool.cc:224] Handling call
0709 13:01:13.003056 (+  5671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.007525 (+  4469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.007561 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:13.015809 (+  8248us) log.cc:844] Serialized 26 byte log entry
0709 13:01:13.295552 (+279743us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:13.295616 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.295688 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:13.295733 (+    45us) raft_consensus.cc:1685] finished
0709 13:01:13.295744 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.295901 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4243,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":5625,"thread_start_us":247466,"threads_started":1,"wal-append.queue_time_us":266852}
W20260709 13:01:13.309983  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2436) took 2581 ms (client timeout 2842 ms). Trace:
W20260709 13:01:13.321461  1657 debug-util.cc:398] Leaking SignalData structure 0x7b08001c3900 after lost signal to thread 1493
W20260709 13:01:13.289078  1601 connection.cc:256] RPC call timeout handler was delayed by -1.93873806900001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
I20260709 13:01:13.310302  1455 catalog_manager.cc:5697] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:13.318564  5409 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:13.317593  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.318671  5103 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.326560  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:13.307708  5498 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.329668  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:13.301687  5513 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:13.333318  1601 connection.cc:256] RPC call timeout handler was delayed by -1.9369380690000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
W20260709 13:01:13.375782  1601 connection.cc:256] RPC call timeout handler was delayed by -1.9369380690000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
W20260709 13:01:13.333010  1705 rpcz_store.cc:269] 0709 13:01:10.728821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.082826 (+2354005us) service_pool.cc:224] Handling call
0709 13:01:13.309868 (+227042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.321784  1560 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:13.324977  5495 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.351307  1633 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:13.326542  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1844) took 2094 ms. Trace:
W20260709 13:01:13.357935  1676 leader_election.cc:336] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.776s (SENT)
I20260709 13:01:13.355885  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:13.388365  1630 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.369835  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1893) took 2583 ms. Trace:
W20260709 13:01:13.348519  1657 debug-util.cc:398] Leaking SignalData structure 0x7b08000ca4e0 after lost signal to thread 1585
I20260709 13:01:13.377614  1455 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 3 to 6, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 6 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:13.389981  1657 debug-util.cc:398] Leaking SignalData structure 0x7b0800399500 after lost signal to thread 1586
W20260709 13:01:13.391325  1657 debug-util.cc:398] Leaking SignalData structure 0x7b0800386900 after lost signal to thread 1660
W20260709 13:01:13.391633  1657 debug-util.cc:398] Leaking SignalData structure 0x7b0800220e80 after lost signal to thread 1661
I20260709 13:01:13.392169  4416 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.392597  4416 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:13.393056  1657 debug-util.cc:398] Leaking SignalData structure 0x7b0800363be0 after lost signal to thread 1734
I20260709 13:01:13.393155  4416 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:13.393859  1657 debug-util.cc:398] Leaking SignalData structure 0x7b08003834c0 after lost signal to thread 1735
I20260709 13:01:13.332041  5423 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:13.395561  1657 debug-util.cc:398] Leaking SignalData structure 0x7b08003d8de0 after lost signal to thread 2377
W20260709 13:01:13.445228  1657 debug-util.cc:398] Leaking SignalData structure 0x7b080034f060 after lost signal to thread 2378
W20260709 13:01:13.397193  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 111433us
I20260709 13:01:13.318467  5355 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:13.457016  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.414389  1455 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 4 to 8, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 8 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:13.460083  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:13.461264  1705 raft_consensus.cc:2368] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8.
I20260709 13:01:13.446514  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.463003  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.406044  1627 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
W20260709 13:01:13.426615  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 97020.4us
I20260709 13:01:13.464797  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.465253  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.465576  1710 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:13.387560  1707 rpcz_store.cc:276] 0709 13:01:11.232501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.185704 (+1953203us) service_pool.cc:224] Handling call
0709 13:01:13.186193 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.186458 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:13.259333 (+ 72875us) log.cc:844] Serialized 14 byte log entry
0709 13:01:13.326001 (+ 66668us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.326085 (+    84us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:13.326170 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.326278 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.326458 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1634,"prepare.queue_time_us":72145,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":32500,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":138,"thread_start_us":112392,"threads_started":2}
I20260709 13:01:13.434643  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.457024  4416 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:13.375212  5498 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.354651  5103 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:13.471148  5498 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:13.471418  5103 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:13.472800  1630 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
W20260709 13:01:13.448547  1657 debug-util.cc:398] Leaking SignalData structure 0x7b080000fce0 after lost signal to thread 5427
W20260709 13:01:13.441727  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 130894us
I20260709 13:01:13.476292  4416 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:13.395906  1553 rpcz_store.cc:276] 0709 13:01:10.785877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.101179 (+2315302us) service_pool.cc:224] Handling call
0709 13:01:13.128702 (+ 27523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.129194 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:13.136153 (+  6959us) log.cc:844] Serialized 14 byte log entry
0709 13:01:13.367031 (+230878us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.367303 (+   272us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:13.367635 (+   332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.368084 (+   449us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.369368 (+  1284us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":292,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":13553,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":114,"thread_start_us":216026,"threads_started":1}
I20260709 13:01:13.470495  1629 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:13.382325  5513 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.463366  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.437436  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:13.486721  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 49544.4us
I20260709 13:01:13.491145  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.492213  1633 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:01:13.494406  1527 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:13.497113  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 227765us
I20260709 13:01:13.501811  1708 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:13.504542  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:13.505419  2346 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:13.506551  4416 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:13.507103  4416 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:13.506011  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:13.438043  5409 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:13.445693  5507 raft_consensus.cc:2764] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:01:13.515575  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.518833  1708 raft_consensus.cc:2368] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11.
I20260709 13:01:13.503171  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:13.520675  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:13.527518  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.528600  1710 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:13.529452  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.531383  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:13.532864  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:13.533763  2349 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:13.530893  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:13.540578  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.540632  2351 raft_consensus.cc:2410] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:13.505550  5043 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:13.510102  5488 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:13.546154  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.547400  1455 catalog_manager.cc:5697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 6, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 6 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:13.521543  5233 consensus_queue.cc:1059] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:01:13.550799  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.551789  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:13.561887  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:13.563822  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 98632.5us
I20260709 13:01:13.543183  4416 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:13.582643  1629 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:01:13.581701  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.578323  1455 catalog_manager.cc:5697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 3 to 4, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 4 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:13.591516  5409 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:13.594795  5469 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:13.596367  5469 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.535028  5514 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:01:13.597739  1630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3897029c4b5f443296641c39af287e98"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:48418
I20260709 13:01:13.598374  1630 raft_consensus.cc:606] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:13.598732  1630 raft_consensus.cc:614] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:13.601014  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:13.601940  1553 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:13.603554  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:13.604568  1560 raft_consensus.cc:2393] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:13.605911  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:13.606509  1557 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:13.607493  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:13.608062  1552 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:13.608395  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:13.608971  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:13.601850  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:13.609416  2348 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:13.611156  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.611716  1561 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:13.540958  5501 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:13.625958  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:13.626631  1601 connection.cc:256] RPC call timeout handler was delayed by -1.8937380689999941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
W20260709 13:01:13.627049  1601 connection.cc:256] RPC call timeout handler was delayed by -1.8550380689999884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260709 13:01:13.627421  1601 connection.cc:256] RPC call timeout handler was delayed by -1.8460380690000306s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
W20260709 13:01:13.627746  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7587380690000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 13:01:13.628106  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7587380690000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 13:01:13.628409  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7434380689999784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2048s for a response
W20260709 13:01:13.628744  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7029380690000266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
W20260709 13:01:13.629077  1601 connection.cc:256] RPC call timeout handler was delayed by -1.6363380689999758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
W20260709 13:01:13.638888  5505 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.000s
W20260709 13:01:13.629388  1601 connection.cc:256] RPC call timeout handler was delayed by -1.6363380689999758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
I20260709 13:01:13.559412  5043 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.655578  5043 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.657421  1630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:48418
I20260709 13:01:13.643119  1455 catalog_manager.cc:5697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:13.661260  1630 raft_consensus.cc:606] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:13.661849  1630 raft_consensus.cc:614] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:13.650000  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.662642  1629 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:13.573458  5488 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:13.652191  1601 connection.cc:256] RPC call timeout handler was delayed by -1.6039380689999803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
W20260709 13:01:13.664142  1601 connection.cc:256] RPC call timeout handler was delayed by -1.6030380689999788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
W20260709 13:01:13.664544  1601 connection.cc:256] RPC call timeout handler was delayed by -1.6030380689999788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
W20260709 13:01:13.664865  1601 connection.cc:256] RPC call timeout handler was delayed by -1.6030380689999788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
W20260709 13:01:13.665180  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4050380689999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2424s for a response
W20260709 13:01:13.665534  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0306380690000196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.284s for a response
I20260709 13:01:13.642810  5523 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:13.672536  1455 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 4 to 8, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:13.671705  5523 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.684291  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:13.685014  1629 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.686627  5523 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:13.679118  5517 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.126s	user 0.004s	sys 0.000s
W20260709 13:01:13.710886  1702 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.665s	user 0.010s	sys 0.008s
I20260709 13:01:13.727451  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.730077  1702 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:13.730278  1627 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:01:13.668373  5488 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:13.730654  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1842) took 2498 ms (client timeout 1881 ms). Trace:
I20260709 13:01:13.728166  1455 catalog_manager.cc:5697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:13.731030  1702 rpcz_store.cc:269] 0709 13:01:11.231847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.928564 (+1696717us) service_pool.cc:224] Handling call
0709 13:01:13.730570 (+802006us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:13.752331  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 86268.2us
I20260709 13:01:13.753545  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:13.654691  5505 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:13.725927  5435 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.003s
W20260709 13:01:13.759264  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.760985  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:13.672350  5478 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:13.759465  5501 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:13.788245  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.789021  1627 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 5.
I20260709 13:01:13.771886  1703 raft_consensus.cc:2410] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:01:13.772074  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 190671us
I20260709 13:01:13.826157  1702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:01:13.782104  1633 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:13.784087  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:13.827009  1702 raft_consensus.cc:606] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:13.828006  1702 raft_consensus.cc:614] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:13.771229  5492 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.609s	user 0.013s	sys 0.000s
I20260709 13:01:13.771320  5510 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:13.830793  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46836.2us
I20260709 13:01:13.828368  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:13.775805  5525 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:13.767581  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 261644us
W20260709 13:01:13.793001  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 329688us
I20260709 13:01:13.794516  1708 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:13.830138  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1838) took 2600 ms (client timeout 3095 ms). Trace:
W20260709 13:01:13.852205  1703 rpcz_store.cc:269] 0709 13:01:11.230035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.096472 (+1866437us) service_pool.cc:224] Handling call
0709 13:01:13.826838 (+730366us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54002e7cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:13.830075 (+  3237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1549224704}
W20260709 13:01:13.792212  5490 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.005s
I20260709 13:01:13.836910  1629 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:13.837658  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:13.767179  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 286763us
I20260709 13:01:13.839985  5510 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:13.842792  5478 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:13.845628  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.846951  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:13.772840  5355 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:13.851584  1702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
W20260709 13:01:13.833140  5427 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.724s	user 0.016s	sys 0.013s
I20260709 13:01:13.790879  5501 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:13.854171  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.859211  1702 raft_consensus.cc:606] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:13.834496  5476 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:13.838294  5472 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.006s
I20260709 13:01:13.883225  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.848711  5103 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:13.894321  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 333836us
I20260709 13:01:13.850461  5433 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:13.889338  1627 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
W20260709 13:01:13.890385  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52493.8us
I20260709 13:01:13.850844  5525 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:13.894949  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:13.851151  5531 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:13.908269  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61388.2us
I20260709 13:01:13.880025  5501 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.901137  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.851625  5492 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:13.861770  5529 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.921094  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.921284  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:13.914476  1601 leader_election.cc:400] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9.
I20260709 13:01:13.927029  1601 leader_election.cc:403] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:13.929430   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:13.931900   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:13.933696  5523 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:13.936823  5523 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:13.939038  5103 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:13.940408  5505 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:13.941300  5505 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:13.941342  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:13.941954  5505 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:13.941820  5537 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:13.942580  5505 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.945585  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:13.948537  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:13.949702  1557 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:13.949738  5510 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:13.951005  5514 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:13.951633  5514 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:13.887758  5476 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:13.949796  1676 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.904s (SENT)
I20260709 13:01:13.943442  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:13.924974  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.901997  5355 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:13.918844  5508 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.746s	user 0.013s	sys 0.002s
I20260709 13:01:13.944630  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:13.914764  5409 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.952750  5514 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:13.964465  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 110326us
W20260709 13:01:13.959249  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 99059.1us
I20260709 13:01:13.957343  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:13.972888  1558 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:13.885654  5427 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:13.912475  5525 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:13.913170  5531 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.869361  5490 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:13.893236  5472 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.002923  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.003705  1629 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:13.982069  1556 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:01:13.925621  5529 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:14.007859  5529 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:13.986692  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.014326  1633 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:13.902261  5526 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:01:13.925808  5534 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.000746  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:14.053145  1561 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:13.959319  5476 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.903432  5233 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:01:13.991617  1552 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:13.962415  5355 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:14.029219  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 49075us
I20260709 13:01:14.031181  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:14.072505  2351 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:14.072825  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:14.030383  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:14.073271  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:14.073534  2347 raft_consensus.cc:2376] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6. Re-sending same reply.
I20260709 13:01:14.074642  1527 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:14.074915  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:14.030064  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:14.076450  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:14.029958  5433 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.999035  5531 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.046967  1633 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:13.973688  5409 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:14.029306  5523 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 13:01:14.070138  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 155983us
I20260709 13:01:14.060436  1704 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:01:13.920360  5498 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.101442  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:14.068536  5355 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:14.111346  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:14.112213  2344 raft_consensus.cc:2376] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8. Re-sending same reply.
I20260709 13:01:14.100113  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.113999  1629 raft_consensus.cc:2410] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:14.084393  5506 raft_consensus.cc:993] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 13:01:14.103611  5472 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:14.107558  5427 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:14.087741  5433 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.115535  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.116277  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:13.977797  5508 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.084414  5494 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.122478  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:14.069629  5543 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9.
I20260709 13:01:14.100492  5409 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.146044  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.148355  1630 raft_consensus.cc:2410] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:14.156340  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.156661  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.157660  1635 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:14.160355  2350 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:01:14.165498  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.157145  1634 raft_consensus.cc:2376] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9. Re-sending same reply.
I20260709 13:01:14.170603  1706 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:14.172464  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:14.171555  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 250285us
I20260709 13:01:14.127483  5433 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:14.174577  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 217390us
I20260709 13:01:14.134371  5506 raft_consensus.cc:1081] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Signalling peer d62ce354d09a4be59c29094425d09bc1 to start an election
I20260709 13:01:14.108397  5478 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:01:14.226477  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.231204  1629 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
W20260709 13:01:14.231307  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 156774us
I20260709 13:01:14.199796  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:14.202191  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.188694  1628 raft_consensus.cc:2376] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9. Re-sending same reply.
I20260709 13:01:14.188861  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:14.145785  5507 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:14.199071  1708 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:01:14.240459  1708 pending_rounds.cc:85] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 8
I20260709 13:01:14.241592  1708 pending_rounds.cc:107] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 7.9
W20260709 13:01:14.239377  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 137994us
W20260709 13:01:14.272691  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 83913.4us
I20260709 13:01:14.273518  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:14.270756  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:14.270120  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.108299  5512 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:01:14.238847  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:14.275076  1708 raft_consensus.cc:2889] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:14.275743  1701 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:14.188436  5544 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:14.283016  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 108032us
I20260709 13:01:14.230316  5233 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.224246  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:14.289824  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:14.292060  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2440) took 3073 ms. Trace:
I20260709 13:01:14.292490  1704 rpcz_store.cc:276] 0709 13:01:11.218602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.144699 (+1926097us) service_pool.cc:224] Handling call
0709 13:01:13.145211 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:14.100796 (+955585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.291786 (+190990us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b54002e77d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:14.291995 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2103992064}
I20260709 13:01:14.294845  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1833) took 3564 ms. Trace:
I20260709 13:01:14.295300  1706 rpcz_store.cc:276] 0709 13:01:10.730639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.098600 (+2367961us) service_pool.cc:224] Handling call
0709 13:01:14.294633 (+1196033us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b5400077954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:14.294783 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278979456}
I20260709 13:01:14.298753  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:14.284029  5495 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.843s	user 0.013s	sys 0.000s
I20260709 13:01:14.146255  5494 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:14.280962  5378 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.091s	user 0.015s	sys 0.003s
I20260709 13:01:14.312502  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:14.313614  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.323624  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:14.323995  1629 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:14.325871  1561 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:14.282356  5427 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:14.326042  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 87270.4us
I20260709 13:01:14.330834  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.331696  1708 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:14.332677  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:14.334515  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.325402  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.328763  1674 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:14.314499  5043 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.658s	user 0.008s	sys 0.004s
I20260709 13:01:14.330549  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:14.286712  5544 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.349881  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:14.298641  5492 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:14.355786  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.362033  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.314939  5507 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:14.326071  5495 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.373114  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.375842  1634 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:14.378062  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 49401.8us
I20260709 13:01:14.381536  1674 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:14.335083  5550 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:14.301091  5233 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.396443  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.397153  1710 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:14.392580  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:14.391390  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:14.399044  5554 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.126s	user 0.002s	sys 0.000s
I20260709 13:01:14.354772  5043 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.400027  5507 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.401889  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.402949  5043 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:14.403213  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.403729  5557 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:14.404513  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.395627  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.386965  1674 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:14.398850  1561 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:01:14.397224  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.414950  1629 raft_consensus.cc:2376] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8. Re-sending same reply.
W20260709 13:01:14.393991  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.350968  5544 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.419685  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:14.420457  1710 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.420719  2344 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:14.421604  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:14.422490  1709 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:14.431829  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:14.325965  5378 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.440034  5559 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:01:14.440313  5512 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:14.441460  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:14.389998  5534 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:14.440075  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:14.441651  5378 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:14.365566  5487 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:14.346966  5427 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.455420  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:14.456164  1561 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.450004  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.457015  1629 raft_consensus.cc:2376] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11. Re-sending same reply.
W20260709 13:01:14.439465  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 166069us
I20260709 13:01:14.450487  1708 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.397872  5478 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:14.453164  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:14.384176  5556 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:14.412366  1701 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:14.458566  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:14.463261  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.469650  1634 raft_consensus.cc:2410] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:14.437294  5535 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:14.359524  5555 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 13:01:14.467969  1704 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:14.473831  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 249726us
I20260709 13:01:14.475426  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:14.470168  1525 leader_election.cc:336] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.505s (SENT)
I20260709 13:01:14.477805  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:14.481617  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:14.452762  5534 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:14.451918  5546 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:14.485872  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45891.2us
I20260709 13:01:14.463979  5478 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.454747  5550 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:14.494962  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.497854  5566 raft_consensus.cc:2764] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:14.500895  5527 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:14.501241  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.501755  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.502050  1636 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 13:01:14.502597  1635 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:14.500761  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.501381  1631 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:14.503422  1629 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:14.504244  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.504752  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:14.501346  5527 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:14.512866  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:14.526095  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 55975.7us
I20260709 13:01:14.482419  5378 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:14.527148  5521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.007s	sys 0.006s
I20260709 13:01:14.532199  1628 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:14.534466  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44368.8us
I20260709 13:01:14.494614  5546 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.537933  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:14.544658  1525 leader_election.cc:336] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.565s (SENT)
I20260709 13:01:14.547534  5494 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:14.538065  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:14.488691  5534 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.548460  5494 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:14.549732  5553 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:01:14.547354  5571 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:14.506021  5343 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:14.554447  5551 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:01:14.555776  5469 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.959s	user 0.012s	sys 0.003s
W20260709 13:01:14.556950  5488 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.804s	user 0.011s	sys 0.002s
W20260709 13:01:14.557667  5513 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.051s	user 0.012s	sys 0.003s
I20260709 13:01:14.558782  5495 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:14.561607  5546 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:14.561961  5545 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 13:01:14.562018  5487 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:14.530290  5423 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.965s	user 0.004s	sys 0.010s
I20260709 13:01:14.500301  5478 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:14.570324  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 65647.9us
I20260709 13:01:14.571162  5562 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:14.574213  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.560223  5534 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.576108  5549 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.576576  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.576994  5550 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.577538  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.530339  5527 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.578853  5506 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:14.498227  5533 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.290s
W20260709 13:01:14.538296  1455 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.008s
W20260709 13:01:14.706027  1455 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.005s	sys 0.008s
W20260709 13:01:14.534569  5521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.007s	sys 0.007s
W20260709 13:01:14.592223  1525 leader_election.cc:336] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.625s (SENT)
I20260709 13:01:14.707196  1525 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:14.708124  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 115983us
I20260709 13:01:14.708756  5433 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.595574  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.711544  5514 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.711814  1630 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:01:14.712914  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.603462  1628 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:14.713778  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:14.714517  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.715252  5549 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:14.715678  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.716048  5549 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:14.716462  1628 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:14.716500  5549 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:14.717087  5549 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:14.602105  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.636168  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.619693  1561 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:14.542841  5378 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:14.666164  5494 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:14.688585  5529 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.680s	user 0.010s	sys 0.002s
I20260709 13:01:14.666126  5565 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:14.575453  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.678015  1629 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.671160  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:14.733944  1554 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:14.660934  5488 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.688794  1633 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
I20260709 13:01:14.689800  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:14.695102  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:14.690616  1632 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:14.739666  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.740473  1632 raft_consensus.cc:2410] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:01:14.740993  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.741248  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.741745  1629 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:14.741854  1632 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.742653  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:14.743280  1525 leader_election.cc:336] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.777s (SENT)
I20260709 13:01:14.675316  5492 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:14.670127  5423 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.698065  5560 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:01:14.750756  5560 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:14.751140  5560 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:14.752036  5560 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.702409  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1063) took 1201 ms. Trace:
I20260709 13:01:14.753872  2353 rpcz_store.cc:276] 0709 13:01:13.501093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.501204 (+   111us) service_pool.cc:224] Handling call
0709 13:01:13.529454 (+ 28250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.784383 (+254929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.784418 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:13.823986 (+ 39568us) log.cc:844] Serialized 26 byte log entry
0709 13:01:14.001549 (+177563us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:14.001593 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.001678 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:14.702177 (+700499us) raft_consensus.cc:1685] finished
0709 13:01:14.702204 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:14.702348 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":241944,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":179,"thread_start_us":345531,"threads_started":2,"wal-append.queue_time_us":272213}
I20260709 13:01:14.720680  2347 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:01:14.659101  1708 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:01:14.717589  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.718590  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1066) took 1215 ms. Trace:
I20260709 13:01:14.658756  5556 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:14.720091  1630 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:14.757375  2350 rpcz_store.cc:276] 0709 13:01:13.503051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.503164 (+   113us) service_pool.cc:224] Handling call
0709 13:01:14.718351 (+1215187us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b540048ab54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:14.718496 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418395520}
I20260709 13:01:14.753327  5560 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.720681  1706 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:14.758065  5556 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.718394  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:14.760640  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 70909.1us
I20260709 13:01:14.763231  5533 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:01:14.729750  5501 consensus_meta.cc:287] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.774s	user 0.004s	sys 0.009s
I20260709 13:01:14.717666  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:14.729637  5505 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.786s	user 0.014s	sys 0.000s
I20260709 13:01:14.766753  1555 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:14.766635  5505 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.767329  5534 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:14.717422  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.768958  5505 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:14.769946  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:14.771955  5531 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.670s	user 0.005s	sys 0.006s
I20260709 13:01:14.772509  1628 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:14.772374  5531 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.772655  5544 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.595046  5233 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:14.773586  1560 raft_consensus.cc:2368] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11.
I20260709 13:01:14.773586  5505 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.758215  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.775007  1630 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:01:14.778791  5433 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:14.725767  5494 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:14.780210  5494 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.781526  5494 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.782238  1628 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:01:14.782749  1628 pending_rounds.cc:85] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 6
I20260709 13:01:14.783061  1628 pending_rounds.cc:107] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:01:14.783370  1628 raft_consensus.cc:2889] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:14.785405  1630 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:14.663892  5510 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.012s	sys 0.032s
I20260709 13:01:14.727376  1707 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:14.661859  5513 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.680047  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.736744  1636 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:14.701232  5490 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:14.677855  5562 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.743295  1629 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:14.819185  5510 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.013s	sys 0.033s
I20260709 13:01:14.748821  5423 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:14.832635  1552 raft_consensus.cc:2368] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10.
I20260709 13:01:14.718039  1635 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:14.760746  1561 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.765767  5501 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.667726  5487 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:14.723830  1710 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:14.836788  1555 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:14.836953  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:14.838227  1710 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:14.790889  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.820873  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.841547  5487 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.841780  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.842445  1635 raft_consensus.cc:2376] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6. Re-sending same reply.
I20260709 13:01:14.843313  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.844005  1635 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:14.844727  1630 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:14.833674  1556 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:47110
I20260709 13:01:14.846485  1556 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:14.737378  5529 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:14.789939  5506 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:14.736374  5476 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.665s	user 0.000s	sys 0.013s
I20260709 13:01:14.826589  1702 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:14.737649  5565 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:14.827641  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:14.745245  5492 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.719400  5378 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.825366  5543 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
W20260709 13:01:14.726040  5355 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.002s
I20260709 13:01:14.818966  1632 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:14.847689  1556 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:14.831461  5562 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.849579  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.830653  5472 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:01:14.844961  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:14.851338  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.847213  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:14.857462  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.883458  1629 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:14.861866  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:14.885413  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:14.887298  1674 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:14.880904  1636 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:14.864616  1710 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:14.865732  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:14.891265  1525 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.821s (SENT) [suppressed 56 similar messages]
I20260709 13:01:14.832559  5505 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:01:14.893390  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.893636  5559 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:14.894254  1635 raft_consensus.cc:2410] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:14.894192  5559 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:14.895198  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.895241  5559 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.895989  1635 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:14.892066  1525 leader_election.cc:336] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.828s (SENT)
I20260709 13:01:14.871711  5565 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.892908  5569 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.877878  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.898238  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:14.899500  1525 leader_election.cc:336] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.829s (SENT)
I20260709 13:01:14.901728  1633 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:14.905504  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:14.906486  1633 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:14.908238  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.909431  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.875802  5529 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:14.910018  1636 raft_consensus.cc:2376] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11. Re-sending same reply.
I20260709 13:01:14.912936  1561 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:14.873238  5492 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:14.876879  5576 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.930128  1632 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:01:14.915467  1525 leader_election.cc:336] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.901s (SENT)
I20260709 13:01:14.931028  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:14.913461  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:14.897313  5233 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.897493  5469 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.935246  5492 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.935963  5565 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:14.923756  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.916551  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:14.855232  5506 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.860312  5476 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.939522  1635 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:14.911193  5556 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:14.928467  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:14.933235  1636 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:14.910983  5488 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:14.907956  5490 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:14.938020  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:14.880885  5355 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.861428  5527 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.940475  1629 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.945940  1556 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.914422  5513 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:14.963279  1632 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:14.912153  5478 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:14.909273  5521 consensus_queue.cc:1059] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:01:14.973258  1635 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:14.974023  1627 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:14.917872  5472 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:01:14.941821  5505 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:14.986204  5501 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:14.988346  5355 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:14.989413   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:14.964061  5525 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.904s	user 0.012s	sys 0.000s
I20260709 13:01:14.949151  5490 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.991372  1701 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:14.993778  5490 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.991411  1628 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:14.993999  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 56069.8us
I20260709 13:01:14.963487  5556 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:14.994962  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:14.995232  1556 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:15.001053  1635 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:14.995349  5525 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:14.994666  1629 raft_consensus.cc:2410] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:14.995540  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:14.998960  1704 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:14.972055  5378 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:15.006177  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.006771  1710 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:15.009588  5582 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.074s	user 0.001s	sys 0.001s
I20260709 13:01:15.010586  5476 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:15.014837  5575 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:01:15.015070  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2459) took 1076 ms. Trace:
I20260709 13:01:15.015498  1560 rpcz_store.cc:276] 0709 13:01:13.939161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.944258 (+  5097us) service_pool.cc:224] Handling call
0709 13:01:15.014863 (+1070605us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b5400311d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:15.015011 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1845218560}
I20260709 13:01:15.015946  5580 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:14.989255  5505 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:15.019296  5505 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.996058  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:15.004127  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:14.913833  5534 consensus_queue.cc:1175] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba0548b7d2a94e8598475a46c279fe0e" responder_term: 12 status { last_received { term: 7 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: [9.7-11.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:15.032915  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.033725  1633 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:15.024447  5409 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.854s	user 0.006s	sys 0.006s
I20260709 13:01:15.025790  5527 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:15.037145  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.034751  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.026881  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.037797  1635 raft_consensus.cc:2376] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9. Re-sending same reply.
I20260709 13:01:15.038762  1559 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:15.031365  5427 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:15.031617  5472 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.020320  5355 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:15.044340  1710 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:15.001046  5531 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:15.065160  5546 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.037032  5476 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:01:15.040949  5409 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.040961  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.079197  5546 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:15.042761  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:15.080492  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.080704  5476 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:15.019992  5378 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:15.081398  5476 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.081688  5378 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.080368  5409 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:15.082690  5476 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:15.083041  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:15.084100  1631 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:15.084046  5476 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:15.085372  5476 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:15.085721  5476 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.085784  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.086153  5476 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.086719  1631 raft_consensus.cc:2410] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:15.087491  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.088162  5569 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:15.088482  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.040467  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:14.990859  5492 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:01:15.089776  1629 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:15.035782  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:15.090973  1629 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:01:15.091322  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 55677.3us
I20260709 13:01:15.092015  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:15.092710  5409 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:15.093307  5409 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:15.093608  5498 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:15.063810  5525 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:15.101894  1631 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:15.065388  5556 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.069803  5506 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:15.070832  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2319) took 1150 ms. Trace:
I20260709 13:01:15.070935  1634 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.071359  5505 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.072384  5490 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.042891  5501 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:15.073410  5043 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:15.074443  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.075990  5355 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:15.030738  5478 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:15.079362  1633 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:15.093452  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:15.125676  1636 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:15.125699  5355 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:15.127769  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:15.128844  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:15.062258  5544 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:15.121626  5501 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:15.129913  5501 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.130743  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.131279  1633 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.132779  5492 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:15.133174  5492 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:15.133562  5492 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:15.136451  5508 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:15.139328  5583 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.297s	user 0.001s	sys 0.001s
I20260709 13:01:15.144454  1635 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:15.113178  1627 rpcz_store.cc:276] 0709 13:01:13.920392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.920672 (+   280us) service_pool.cc:224] Handling call
0709 13:01:15.070635 (+1149963us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400343d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:15.070780 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2409135872}
I20260709 13:01:15.036058  5543 raft_consensus.cc:2764] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:15.114398  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.038045  5580 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:15.116724  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2460) took 1177 ms. Trace:
I20260709 13:01:15.147145  1631 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:15.096879  5487 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.150254  5580 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:15.151187  5580 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.151907  1632 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:15.113664  5472 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.123661  5505 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:15.154551  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.125176  5490 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:15.042337  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.155419  1630 raft_consensus.cc:2376] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9. Re-sending same reply.
I20260709 13:01:15.155671  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.099694  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.156294  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.156989  1629 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:15.157604  1635 raft_consensus.cc:2410] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:15.157779  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.158495  1629 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:15.158370  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:15.159224  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.065153  5549 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.160213  5571 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:15.161015  5571 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.158603  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.121035  5043 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:15.154748  5378 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.161495  5549 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:15.129858  5544 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:15.162427  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.162173  1635 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 6: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:15.147547  5525 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:15.202615  5476 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.203114  5525 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:15.203545  5525 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:15.204110  5355 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.204419  5501 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.204838  5476 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:15.156358  1628 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:15.205506  1635 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:15.156340  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.206284  1630 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.164887  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.148749  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:15.216287  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 67622.3us
I20260709 13:01:15.217733  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:15.217823  1630 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:15.219266  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:15.166180  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.149672  5550 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:15.174222  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2465) took 1208 ms. Trace:
I20260709 13:01:15.239379  1559 rpcz_store.cc:276] 0709 13:01:13.965572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.965714 (+   142us) service_pool.cc:224] Handling call
0709 13:01:13.983890 (+ 18176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:15.039199 (+1055309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.173984 (+134785us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001ce154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:15.174164 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2425573632}
I20260709 13:01:15.180677  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:15.181042  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.241964  1701 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:15.242409  1701 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:15.180706  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.161430  5571 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.255903  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:15.190394  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.191596  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1074) took 1172 ms. Trace:
I20260709 13:01:15.147334  1553 rpcz_store.cc:276] 0709 13:01:13.939546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.950944 (+ 11398us) service_pool.cc:224] Handling call
0709 13:01:13.951568 (+   624us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:14.110486 (+158918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:14.110535 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:14.113706 (+  3171us) log.cc:844] Serialized 52 byte log entry
0709 13:01:14.284451 (+170745us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:14.451838 (+167387us) log.cc:844] Serialized 14 byte log entry
0709 13:01:14.452187 (+   349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.103088 (+650901us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b540038bdd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:15.103133 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:15.103174 (+    41us) raft_consensus.cc:1685] finished
0709 13:01:15.116444 (+ 13270us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:15.116652 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30396,"prepare.queue_time_us":82574,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":186128,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":37855,"spinlock_wait_cycles":10749824,"thread_start_us":336329,"threads_started":3,"wal-append.queue_time_us":253868}
W20260709 13:01:15.193862  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 110886us
I20260709 13:01:15.263928  1601 leader_election.cc:400] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11.
I20260709 13:01:15.264353  1601 leader_election.cc:403] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:15.265403  1601 leader_election.cc:400] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8.
I20260709 13:01:15.265825  1601 leader_election.cc:403] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:01:15.197535  1675 leader_election.cc:400] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11.
I20260709 13:01:15.249233  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.266402  1675 leader_election.cc:403] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:15.198757  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.266825  1559 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.199167  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.199378  5544 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:15.198758  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.201762  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.268463  1703 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.205724  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:15.232820  4416 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.233923  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:15.235203  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.237002  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:15.175073  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1855) took 1662 ms (client timeout 2113 ms). Trace:
W20260709 13:01:15.270810  1707 rpcz_store.cc:269] 0709 13:01:13.512920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.513271 (+   351us) service_pool.cc:224] Handling call
0709 13:01:15.174878 (+1661607us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400078354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:15.175005 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2588789504}
I20260709 13:01:15.249616  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:15.251240  2317 leader_election.cc:336] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.790s (SENT)
I20260709 13:01:15.252820  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.272665  1709 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.273613  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:15.277606  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.293648  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:15.279714  1527 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:15.283865  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.278666  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 98062.5us
I20260709 13:01:15.296566  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:15.288081  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.284626  2347 raft_consensus.cc:2368] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8.
W20260709 13:01:15.293747  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.288648  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 91328.6us
I20260709 13:01:15.259938  2349 rpcz_store.cc:276] 0709 13:01:14.018804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.018963 (+   159us) service_pool.cc:224] Handling call
0709 13:01:14.069329 (+ 50366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:14.069635 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:14.084811 (+ 15176us) log.cc:844] Serialized 14 byte log entry
0709 13:01:14.351919 (+267108us) log.cc:844] Serialized 14 byte log entry
0709 13:01:15.191124 (+839205us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b18002917c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:01:15.191278 (+   154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:15.191323 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:15.191370 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.191437 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:15.191533 (+    96us) inbound_call.cc:177] Queueing success response
0709 13:01:15.211889 (+ 20356us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b18002917c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda2b5ae58f 00007fda245ed571 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd
Metrics: {"mutex_wait_us":785929,"prepare.queue_time_us":20462,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":10078,"raft.queue_time_us":854424,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":34275,"spinlock_wait_cycles":66224640,"thread_start_us":266186,"threads_started":1,"wal-append.queue_time_us":356946}
I20260709 13:01:15.290799  1701 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.304288  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:15.317917  1633 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:15.239009  5550 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:15.275216  5559 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:15.342454  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:15.312506  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.288017  5476 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11.
I20260709 13:01:15.322072  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.360249  1629 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:15.351158  1630 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.348584  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:15.349083  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 97894.4us
W20260709 13:01:15.347146  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.345959  1709 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:01:15.366642  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 66904.5us
I20260709 13:01:15.304412  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:15.293372  5490 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8.
I20260709 13:01:15.368947  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.369159  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.321033  1701 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:15.370975  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 76158.7us
I20260709 13:01:15.371706  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.372157  4416 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.372529  1628 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:15.294473  5433 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:15.369266  2344 raft_consensus.cc:2368] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11.
W20260709 13:01:15.305015  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 71183.5us
I20260709 13:01:15.376998  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:15.377718  1559 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:15.300961  5505 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:15.374850  1636 raft_consensus.cc:2368] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8.
I20260709 13:01:15.378183  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:15.305184  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:15.385713  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 80599.1us
I20260709 13:01:15.303941  5526 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.315867  5409 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.342878  5566 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:15.387341  1632 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:15.382946  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:15.399286  1635 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:15.384570  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:15.311705  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.402642  1630 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:15.300694  5478 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:01:15.389804  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.348829  5523 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:15.392637  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.393561  4416 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:15.294612  5435 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:01:15.403776  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:15.359371  5487 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:15.405054  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 93459.6us
I20260709 13:01:15.368897  5427 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11.
I20260709 13:01:15.357831  5529 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 13:01:15.409086  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 96331.8us
I20260709 13:01:15.388851  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.394203  5560 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:15.436836  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51706.9us
I20260709 13:01:15.394937  5433 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:15.442194  5472 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:15.397233  5526 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.443166  5355 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:15.409982  5523 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.563004  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:15.381389  5594 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:15.445248  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:15.394667  5559 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.446391  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.565623  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:15.398046  5498 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:15.474695  5529 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:01:15.449635  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:15.391521  5557 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.452399  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:15.572425  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 120092us
I20260709 13:01:15.573076  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:15.573668  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 43755.3us
I20260709 13:01:15.452732  2318 leader_election.cc:400] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10.
I20260709 13:01:15.457113  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:15.443843  5591 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:01:15.350683  5562 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.473435  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:15.576469  2351 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:15.414302  5566 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:15.451272  5501 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:15.370266  5043 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.503051  5547 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:15.395431  5514 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:01:15.457268  1701 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 6.
W20260709 13:01:15.525731  5548 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.025s	sys 0.050s
I20260709 13:01:15.591357  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.591858  5343 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:15.592324  5343 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:15.593586  5552 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.061s	user 0.001s	sys 0.004s
I20260709 13:01:15.593829  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:15.594610  1602 leader_election.cc:400] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8.
I20260709 13:01:15.595237  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:15.595455  1602 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:15.596208  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:15.597131  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:15.451229  5560 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:15.579118  5409 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:15.593225  5343 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.598085  5560 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.598891  2348 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
I20260709 13:01:15.575609  5507 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:15.600023  5507 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:15.593883  2346 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:15.601228  5507 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.407774  5569 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:15.530328  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 65328.9us
I20260709 13:01:15.619349  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:15.620031  1634 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:15.531344  5592 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:01:15.579051  5549 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:15.532989  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:15.535423  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 639) took 1334 ms. Trace:
I20260709 13:01:15.579468  5378 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:15.627645  1705 rpcz_store.cc:276] 0709 13:01:14.200872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.200990 (+   118us) service_pool.cc:224] Handling call
0709 13:01:14.276204 (+ 75214us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:14.350434 (+ 74230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:14.350469 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:14.377792 (+ 27323us) log.cc:844] Serialized 26 byte log entry
0709 13:01:15.192732 (+814940us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b18003c08e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:01:15.404535 (+211803us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:15.404576 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.404646 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:15.535125 (+130479us) raft_consensus.cc:1685] finished
0709 13:01:15.535149 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:15.535359 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56399,"raft.queue_time_us":82660,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":12407,"spinlock_wait_cycles":28007296,"thread_start_us":211642,"threads_started":1,"wal-append.queue_time_us":1134356}
I20260709 13:01:15.383934  5556 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:15.569813  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:15.542726  5488 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.574667  2318 leader_election.cc:403] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:15.574630  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:15.535012  5526 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:15.618688  5508 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:15.447466  5544 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:15.581668  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 124661us
I20260709 13:01:15.566166  5523 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:15.586115  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:15.587627  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:15.589911  5478 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:15.590126  5547 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:15.591076  5514 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:15.591795  5548 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.026s	sys 0.050s
I20260709 13:01:15.574867  5594 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:15.594511  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:15.673830  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.413858  5580 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:01:15.438606  5505 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:15.526435  5558 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:01:15.604305  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:15.605530  1674 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.607884  2351 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:15.597465  1602 leader_election.cc:400] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11.
W20260709 13:01:15.691974  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 86575.3us
I20260709 13:01:15.692682  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:15.620777  4416 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:15.693853  4416 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:15.526590  5593 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:01:15.622000  1703 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
W20260709 13:01:15.625385  1447 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.548s	user 0.003s	sys 0.000s
I20260709 13:01:15.397161  5543 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:15.563910  1709 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:15.633658  5573 consensus_queue.cc:1175] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" responder_term: 8 status { last_received { term: 8 index: 8 } last_committed_idx: 6 last_received_current_leader { term: 8 index: 8 } }
W20260709 13:01:15.634881  5597 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:01:15.673149  5556 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.603457  5355 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:15.602913  5507 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.599242  5560 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.619026  5487 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.579035  1708 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:15.624243  5498 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:15.580974  5521 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:01:15.650954  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.734445  1706 raft_consensus.cc:2376] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11. Re-sending same reply.
I20260709 13:01:15.734158  5498 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.725940  5539 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:15.735352  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:15.735858  5539 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.736219  5539 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.654656  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:15.656013  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 203419us
I20260709 13:01:15.657388  2346 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:01:15.737689  2346 pending_rounds.cc:85] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 6
I20260709 13:01:15.737934  2346 pending_rounds.cc:107] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:01:15.738153  2346 raft_consensus.cc:2889] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:15.659801  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.738251  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.738859  1552 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:15.738986  1559 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.739748  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.655683  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:15.578857  5557 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.672430  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:15.667443  1560 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:15.675851  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.677767  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.679219  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:15.685434  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 110862us
I20260709 13:01:15.742491  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.742648  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:15.692603  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 95232.3us
I20260709 13:01:15.725929  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.743573  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:15.744184  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.465546  5550 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.696197  1629 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:15.701143  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.703258  1633 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:15.747231  1707 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:15.588729  5433 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:15.667400  5526 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.705616  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.708981  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:15.699699  5584 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:01:15.711656  1628 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:15.712406  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.710299  1704 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
W20260709 13:01:15.705859  5525 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.003s
I20260709 13:01:15.706887  5576 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:15.719717  5594 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.725872  5547 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.727309  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.821233  1710 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:15.727648  5492 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.594s	user 0.002s	sys 0.007s
I20260709 13:01:15.825693  5492 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.731595  5615 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:15.828208  5615 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.734158  5521 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:15.830631  5521 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:15.830401  5525 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.831527  5521 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.832808  5521 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:15.734457  5611 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:01:15.738300  2318 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:15.621460  5565 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:01:15.834195  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 95947.4us
I20260709 13:01:15.834895  5565 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:15.835297  5565 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:15.836046  5565 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.836534  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:15.669919  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:15.837776  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:15.680308  5566 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.838348  2346 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:15.747361  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.817078  5550 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.703431  5535 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:15.841534  5512 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.841874  5535 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:15.842031  5512 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.749400  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:15.749619  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 70491.8us
I20260709 13:01:15.750062  1555 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:15.751010  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.741003  5355 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:15.746106  4416 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:15.811131  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 155529us
I20260709 13:01:15.756469  1552 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:15.867868  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.867975  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:15.868307  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.814232  5603 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:15.815085  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:15.791962  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48456.4us
I20260709 13:01:15.876843  1708 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:15.877216  1709 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:15.819921  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.789763  5478 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:15.823243  5606 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:01:15.824404  5562 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:15.838182  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.895505  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.897133  5487 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:15.842983  2348 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:15.899840  5487 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:15.900797  5487 consensus_queue.cc:1243] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:15.745325  1602 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:15.844985  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.837868  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:15.749770  5549 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:15.865514  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.865741  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.868963  2346 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:15.817082  1559 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:15.837210  5565 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.838131  5537 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:15.886521  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.889037  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:15.890926  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:15.891901  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.866343  5506 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.892270  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:15.913002  1707 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:15.892590  5103 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:15.844182  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:15.901999  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:15.868083  5529 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:15.907019  1559 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:15.907341  1703 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:15.906489  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:15.909489  1705 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:01:15.909241  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 163995us
I20260709 13:01:15.866125  5508 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:15.924223  5508 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:15.924712  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:15.926469  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:15.926592  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.926752  5508 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:15.927395  5476 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:15.928122  5562 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:15.928633  5476 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:15.929056  5562 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:15.885839  5514 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:15.911164  5487 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:15.923246  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.915127  1708 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:15.930199  1710 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:15.923210  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:15.917287  1709 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: [8.7-10.8]
I20260709 13:01:15.935950  1629 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:15.933086  1631 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [8.4-9.5]
I20260709 13:01:15.937716  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.940536  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.944790  1559 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.945097  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.911365  5543 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:15.947736  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.842787  5535 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:15.954107  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:15.986272  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:15.960891  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:16.042464  1560 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:15.969763  1627 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:15.930956  5562 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:15.934648  5529 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:16.049329  4416 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:16.045720  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:16.059677  1628 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:15.946619  5378 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:15.902834  5616 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:01:15.921386  5580 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.119s
I20260709 13:01:15.934114  5487 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:16.052911  1707 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:16.053604  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:16.104898  2347 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:15.973971  1559 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.948220  5505 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.949278  5543 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.040704  1631 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:15.948309  5476 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:16.051054  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:16.044770  5617 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:16.061619  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 138461us
I20260709 13:01:16.065769  5501 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.063728  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:16.126386  2346 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:16.074810  1706 raft_consensus.cc:2410] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:16.075057  4416 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:16.075268  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 114492us
I20260709 13:01:16.130601  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:16.130847  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:15.930727  5506 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.105410  1555 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:16.109403  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:16.110118  5576 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:16.106750  5535 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.111986  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:16.045807  5571 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.745s	user 0.010s	sys 0.000s
I20260709 13:01:16.113204  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:16.113960  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:15.946283  5514 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.037750  5355 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.116194  1628 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:16.123462  1627 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:16.136960  4416 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.061616  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:16.062851  1710 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:16.124938  5543 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:16.107000  5490 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:16.074810  5537 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:16.111172  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:16.174172  1559 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:16.170011  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:16.162533  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 239331us
I20260709 13:01:16.184366  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:16.185926  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:16.187425  1704 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.175668  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:16.139928  5505 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.140843  5103 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.137938  5617 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:16.210493  1553 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
W20260709 13:01:16.212268  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 81513.8us
I20260709 13:01:16.155932  5557 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:16.214149  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 52746.1us
I20260709 13:01:16.168407  5616 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:16.164198  5571 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:16.219743  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.220513  1627 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:16.218385  5529 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.226154  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:16.235890  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:16.236611  1627 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:16.218796  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:16.215634  5576 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:16.233873  1557 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:16.137657  5501 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.244055  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:16.214572  5612 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.126199  5549 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.245988  1704 raft_consensus.cc:2410] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:16.246991  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:16.240406  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 109890us
I20260709 13:01:16.230638  5616 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:16.231346  5512 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:16.247674  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:16.251734  2346 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:16.254791  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:16.256487  5409 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.659s	user 0.008s	sys 0.002s
I20260709 13:01:16.176064  5487 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.249146  5505 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:16.261121  5476 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:16.262012  5616 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.263517  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 560) took 1033 ms. Trace:
I20260709 13:01:16.265867  5629 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:16.266350  5612 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:16.266799  5514 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:16.267930  5544 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:16.223996  5618 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 13:01:16.270355  5488 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.001s
W20260709 13:01:16.271864  5539 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.535s	user 0.006s	sys 0.002s
I20260709 13:01:16.272588  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:16.273916  5501 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:16.276952  1553 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:16.242969  5576 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.287201  1631 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:16.187734  5562 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.292693  1556 rpcz_store.cc:276] 0709 13:01:15.230439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.230600 (+   161us) service_pool.cc:224] Handling call
0709 13:01:15.231041 (+   441us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:15.246160 (+ 15119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:15.246213 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:15.295238 (+ 49025us) log.cc:844] Serialized 52 byte log entry
0709 13:01:15.531260 (+236022us) spinlock_profiling.cc:243] Waited 5.65 ms on lock 0x7b4800002958. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:01:15.639809 (+108549us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:16.235920 (+596111us) log.cc:844] Serialized 14 byte log entry
0709 13:01:16.263097 (+ 27177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.263180 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:16.263224 (+    44us) raft_consensus.cc:1685] finished
0709 13:01:16.263317 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:16.263459 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":430909,"raft.queue_time_us":74843,"raft.run_cpu_time_us":2940,"raft.run_wall_time_us":238631,"spinlock_wait_cycles":12993280,"thread_start_us":436914,"threads_started":2,"wal-append.queue_time_us":333124}
W20260709 13:01:16.282652  5560 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.000s
I20260709 13:01:16.249883  5626 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:16.222208  5628 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.065s
W20260709 13:01:16.300482  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 74418.4us
I20260709 13:01:16.363500  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:16.233194  5627 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:01:16.249413  5634 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.282351  5490 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:16.290726  5409 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:16.306358  5487 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:16.314572  5621 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:01:16.314147  5539 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.387228  5539 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:16.330096  5622 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:01:16.389726  5628 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:16.330685  5617 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.390352  5628 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.317011  5614 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:01:16.392598  5628 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:16.350699  5501 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:16.301113  5629 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:16.346796  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.403087  1558 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:01:16.360879  5624 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.126s	user 0.002s	sys 0.000s
I20260709 13:01:16.361378  5544 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:16.366787  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:16.368032  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 95755.7us
I20260709 13:01:16.416239  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:16.349030  5514 consensus_queue.cc:1059] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 13:01:16.369066  5623 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:01:16.309330  5630 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:16.438081  5630 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:16.438874  5630 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:16.439064  5633 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:01:16.439234  5630 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:16.439757  5630 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:16.348325  5576 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:16.381326  5620 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.226s	user 0.001s	sys 0.001s
I20260709 13:01:16.381865  5409 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:16.382992  5543 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.343732  5488 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.384151  5559 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:16.304361  5537 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.358897  5626 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:16.251153  5580 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:01:16.356142  5616 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:16.402333  5602 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.964s	user 0.004s	sys 0.002s
I20260709 13:01:16.350863  5498 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:01:16.371990  5103 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.358599  5378 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:01:16.379706  5612 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.389743  5506 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:16.441375  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:16.472419  1554 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.412873  5629 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.342478  1447 maintenance_manager.cc:643] P 8f622dabf84e4c0595473d2c01f108d0: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 4.076s	user 0.154s	sys 0.061s Metrics: {"bytes_written":29720,"cfile_init":1,"dirs.queue_time_us":124735,"dirs.run_cpu_time_us":598,"dirs.run_wall_time_us":1237,"drs_written":1,"lbm_read_time_us":85291,"lbm_reads_10-100_ms":4,"lbm_write_time_us":5759,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":29,"peak_mem_usage":0,"rows_written":63,"thread_start_us":221310,"threads_started":1}
I20260709 13:01:16.458470  5635 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 13:01:16.458807  5559 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:16.458542  5543 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:16.461617  5409 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.469718  5571 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:16.469329  5544 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.473210  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:16.482100  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 115409us
W20260709 13:01:16.468621  5550 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.002s
W20260709 13:01:16.490288  5523 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.790s	user 0.010s	sys 0.000s
I20260709 13:01:16.498733  1448 maintenance_manager.cc:419] P 8f622dabf84e4c0595473d2c01f108d0: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9474 bytes on disk
I20260709 13:01:16.500109  1554 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
W20260709 13:01:16.500238  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 43611.9us
I20260709 13:01:16.468197  5626 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:16.514132  1702 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:16.475201  5505 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:16.513301  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:16.513445  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:16.518151  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:16.518292  1630 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:16.518673  1627 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:01:16.519232  1627 pending_rounds.cc:85] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 7
I20260709 13:01:16.519513  1627 pending_rounds.cc:107] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:01:16.519773  1627 raft_consensus.cc:2889] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:16.524637  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 108505us
I20260709 13:01:16.525053  2318 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:16.526700  1627 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:16.526535  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:16.527962  1627 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:16.530870  1633 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:01:16.531901  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 277160us
I20260709 13:01:16.532948  1554 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:16.533228  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:16.533437  1705 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:16.528762  1627 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.534785  1627 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:16.529446  1447 maintenance_manager.cc:643] P 8f622dabf84e4c0595473d2c01f108d0: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.017s	user 0.002s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":520,"lbm_reads_lt_1ms":4}
I20260709 13:01:16.539733  1553 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:16.520674  5103 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:16.523653  5571 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:16.525115  5612 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.548133  1707 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:01:16.537276  5603 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.004s
I20260709 13:01:16.549713  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:16.552301  1560 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:16.554934  1627 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:01:16.552978  5647 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:16.555275  1627 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 6
I20260709 13:01:16.553855  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:16.520457  5523 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.559355  1703 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:16.564512  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:16.565176  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.556088  1627 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.7
W20260709 13:01:16.557494  5526 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.763s	user 0.010s	sys 0.002s
I20260709 13:01:16.574019  1629 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:01:16.574555  1629 pending_rounds.cc:85] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 4
I20260709 13:01:16.574894  1629 pending_rounds.cc:107] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:01:16.575163  1629 raft_consensus.cc:2889] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:16.572551  1627 raft_consensus.cc:2917] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Aborting config change with OpId 7.7: Aborted: Op aborted by new leader
I20260709 13:01:16.553488  5543 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:16.577446  2353 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:16.576680  1627 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:16.557418  5571 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:16.561793  5640 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:16.558573  5603 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:16.567310  5609 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:16.578758  5565 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.667s	user 0.011s	sys 0.001s
I20260709 13:01:16.531713  5559 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.577134  5526 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10.
I20260709 13:01:16.563849  5635 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:01:16.553809  5556 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:01:16.586062  5543 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:16.562494  5505 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.581912  5103 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:01:16.581437  5487 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:01:16.596966  5640 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:16.623800  5409 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:16.624058  5650 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:01:16.617028  5501 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:16.627084  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.640000  5435 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:16.603266  5609 consensus_queue.cc:1059] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:01:16.643633  5643 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.128s
W20260709 13:01:16.662070  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.643707  5634 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:16.663407  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:16.667565  1631 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:16.650476  5642 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.135s
W20260709 13:01:16.670480  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 104493us
I20260709 13:01:16.644228  5435 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:16.672712  1704 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:16.677711  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.679669  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:16.666501  5634 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:16.673679  5646 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:01:16.683646  2351 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
W20260709 13:01:16.683867  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 110900us
I20260709 13:01:16.668282  5647 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:16.671425  5648 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:01:16.651185  5571 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.686638  5523 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:16.708513  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:16.711802  1636 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:01:16.670919  5543 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:16.716948  2344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 924) took 1337 ms (client timeout 1547 ms). Trace:
W20260709 13:01:16.722999  2344 rpcz_store.cc:269] 0709 13:01:15.379379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.379586 (+   207us) service_pool.cc:224] Handling call
0709 13:01:16.716716 (+1337130us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b54004ecad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:16.716864 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2055254784}
I20260709 13:01:16.724045  1674 leader_election.cc:400] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8.
I20260709 13:01:16.721253  1631 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:16.724465  1674 leader_election.cc:403] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:01:16.726364  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:16.727645  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:16.728662  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:16.731416  1556 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:16.760092  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:16.765686  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.732355  5556 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:16.770730  5539 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8.
I20260709 13:01:16.755848  5634 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.772219  5634 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.773722  5527 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:16.774147  5514 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:16.774838  5514 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:16.775404  5514 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.778729  5487 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.779618  5487 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.720085  5435 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.783144  5435 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.761689  5565 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.773036  1674 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:16.786279  5565 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.787712  5494 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:16.788583  5494 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:16.789942  5494 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.790449  5548 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:16.791483  5616 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.791847  5548 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:16.792264  5616 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.699400  5629 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.732308  5409 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.732617  5603 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:16.786289  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:16.838587  5527 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.757090  5643 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:01:16.843384  5487 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:16.843734  5630 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.843757  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:16.880671  1633 raft_consensus.cc:2410] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:16.683514  5640 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.844789  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:16.848822  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:16.848203  5612 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:16.849216  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:16.796347  5539 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:16.850438  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:16.851114  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.851704  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:16.722414  5498 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:01:16.853590  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:16.815912  5647 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.697966  5667 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.169s
I20260709 13:01:16.802727  5548 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.865417  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:16.856578  5469 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:16.856408  5495 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.761404  5513 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:16.856642  5580 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:16.761211  5550 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.881837  2350 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:16.883621  2352 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:16.885340  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:16.928421  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:16.788950  5565 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:16.856854  5472 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.890057  1556 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:16.909096  2344 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:16.909569  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:16.882076  5629 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:16.863934  5433 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:01:16.904469  5378 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.948098  1704 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.946382  1633 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:16.949311  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:16.956251  2318 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.736s (SENT)
I20260709 13:01:16.886240  5527 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.898129  5626 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.957823  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:16.961259  1633 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.908736  5469 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:16.949116  5513 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.946553  5667 consensus_queue.cc:1243] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:16.955102  5548 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:16.959257  5505 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:16.971566  1627 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:01:16.971973  5513 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:16.973668  5626 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:16.974498  1556 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:16.975145  5630 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:16.999090  5514 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.952593  5565 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:01:16.949999  5472 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.001725  5487 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:17.003968  5472 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:16.983682  5616 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:16.985491  5576 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.001s
I20260709 13:01:17.004895  1635 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:16.956249  5378 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.005126  5576 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.951551  5580 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.007339  5576 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:17.007903  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.008441  1636 raft_consensus.cc:2410] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:17.008909  1553 raft_consensus.cc:2410] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:16.982901  5527 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:16.948540  5550 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:16.989653  5469 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:16.949182  5495 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.012522  5469 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:17.013697  5495 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:17.007331  5378 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:17.015027  5550 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:16.986850  2318 leader_election.cc:336] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.751s (SENT)
I20260709 13:01:17.015599  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.015849  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:17.016744  5495 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:17.019759  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 168205us
I20260709 13:01:17.004065  5514 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:17.042188  1627 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:16.994506  5435 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.060693  5666 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:17.062604  5595 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.620s	user 0.002s	sys 0.000s
I20260709 13:01:17.061287  5666 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:16.999604  5505 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.089497  5505 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:17.055717  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 925) took 1651 ms. Trace:
I20260709 13:01:17.093801  2353 rpcz_store.cc:276] 0709 13:01:15.404404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.404840 (+   436us) service_pool.cc:224] Handling call
0709 13:01:15.440781 (+ 35941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:16.577928 (+1137147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.055409 (+477481us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540002b2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:17.055642 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2613525248}
I20260709 13:01:17.066995  5580 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:17.072710  5525 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:17.078760  1558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1913) took 1413 ms (client timeout 1733 ms). Trace:
W20260709 13:01:17.097767  1558 rpcz_store.cc:269] 0709 13:01:15.665559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.680505 (+ 14946us) service_pool.cc:224] Handling call
0709 13:01:17.078513 (+1398008us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b5400207e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:17.078677 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1448804096}
I20260709 13:01:17.080566  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:17.098441  5666 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.100200  5666 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:17.100970  5634 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:17.101109  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:17.101619  5659 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.479s	user 0.002s	sys 0.003s
I20260709 13:01:17.101747  5634 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:17.101964  5378 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:17.081040  5435 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.081811  1552 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:17.083432  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2540) took 1940 ms. Trace:
I20260709 13:01:17.085283  5492 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:17.086163  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.086257  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:17.105744  1634 rpcz_store.cc:276] 0709 13:01:15.143382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.198487 (+ 55105us) service_pool.cc:224] Handling call
0709 13:01:17.083223 (+1884736us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b540010dbd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:17.083366 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808930048}
I20260709 13:01:17.088024  5573 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.089565  5526 raft_consensus.cc:3055] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:17.096791  5598 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.685s	user 0.000s	sys 0.007s
I20260709 13:01:17.096848  1704 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:17.123342  1633 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:17.046021  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:17.123808  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:17.124379  2352 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:01:17.047425  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 206867us
I20260709 13:01:17.127754  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:17.131716  1559 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:17.102998  5634 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:17.136085  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 90589.8us
I20260709 13:01:17.141145  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:17.147010  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:17.150336  2348 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:17.155325  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:17.158980  1556 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:17.159809  2347 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:17.161221  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:17.162739  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:17.163400  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:17.109025  5378 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:17.165416  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.163404  1707 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:17.166065  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:17.159878  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:17.169947  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.170171  2353 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:17.123843  5526 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:17.166985  1527 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:17.167810  2350 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEARNER]: Advancing to term 8
I20260709 13:01:17.175695  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:17.176942  1634 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:17.177289  1704 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:17.180966  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:17.182770  1627 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:17.183796  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:17.190313  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:17.191723  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:17.192819  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:17.196779  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1081) took 1087 ms. Trace:
I20260709 13:01:17.197301  2351 rpcz_store.cc:276] 0709 13:01:16.109185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.113729 (+  4544us) service_pool.cc:224] Handling call
0709 13:01:17.196563 (+1082834us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b540081d8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:17.196704 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1021868416}
I20260709 13:01:17.183442  5548 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:17.203745  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 657) took 1527 ms (client timeout 1736 ms). Trace:
W20260709 13:01:17.204274  1706 rpcz_store.cc:269] 0709 13:01:15.676085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.735067 (+ 58982us) service_pool.cc:224] Handling call
0709 13:01:17.203509 (+1468442us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54002a71d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:17.203675 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748143872}
I20260709 13:01:17.207125  5435 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:17.207455  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 648) took 1808 ms (client timeout 1751 ms). Trace:
I20260709 13:01:17.185143  5629 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 13:01:17.207978  1702 rpcz_store.cc:269] 0709 13:01:15.399250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.406350 (+  7100us) service_pool.cc:224] Handling call
0709 13:01:17.207242 (+1800892us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b540036e2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:17.207376 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2183278464}
I20260709 13:01:17.209859  5647 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:17.212131  1702 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:01:17.213153  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.213259  1709 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:17.214299  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.171268  5469 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:17.225631  5469 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.226114  5469 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:17.220271  5629 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:17.228634  5629 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:17.229693  5629 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.231067  5629 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:17.221906  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.242715  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:17.244144  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.244366  5612 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:17.244799  5612 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:17.246063  5648 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:17.246536  5648 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:17.247419  5648 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.252944  1602 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:17.223522  1707 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:17.255707  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2366) took 1318 ms. Trace:
I20260709 13:01:17.295423  1629 rpcz_store.cc:276] 0709 13:01:15.936981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.937104 (+   123us) service_pool.cc:224] Handling call
0709 13:01:15.937425 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:16.575602 (+638177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.255381 (+679779us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b54003b88d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:17.255619 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463232768}
I20260709 13:01:17.256672  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 928) took 1723 ms. Trace:
I20260709 13:01:17.225320  5495 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:17.260713  1710 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:17.263368  5543 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.263545  5676 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.265817  5435 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.267850  5527 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:17.318928  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.214352  5550 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:17.269793  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1911) took 1610 ms. Trace:
I20260709 13:01:17.325456  1557 rpcz_store.cc:276] 0709 13:01:15.659285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.674130 (+ 14845us) service_pool.cc:224] Handling call
0709 13:01:15.703649 (+ 29519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:16.244330 (+540681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.269497 (+1025167us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54003309d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:17.269719 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1218802944}
W20260709 13:01:17.276312  5665 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.437s	user 0.003s	sys 0.000s
I20260709 13:01:17.278764  5635 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:17.122910  5573 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.216467  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:17.336858  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 120495us
I20260709 13:01:17.222580  5562 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:17.303071  5565 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.298084  2349 rpcz_store.cc:276] 0709 13:01:15.535479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.535633 (+   154us) service_pool.cc:224] Handling call
0709 13:01:15.687116 (+151483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:15.687292 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:15.687321 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:15.687359 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:15.687386 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.687435 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:17.256484 (+1569049us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54009902d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:17.256606 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347623808}
W20260709 13:01:17.303555  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50720.7us
I20260709 13:01:17.341746  1602 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:17.308662  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.308671  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.343777  1554 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:17.309540  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:17.309829  1560 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:17.310449  1702 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:01:17.313782  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:17.171669  5640 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.128s
I20260709 13:01:17.269641  1676 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:17.297396  5501 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.334789  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:17.281237  5658 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.658s	user 0.000s	sys 0.002s
I20260709 13:01:17.346848  1558 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:17.352770  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:17.314184  5472 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:17.361616  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 177934us
W20260709 13:01:17.386797  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 117228us
I20260709 13:01:17.335922  5635 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:17.355559  5494 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.357721  5378 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:17.315466  5543 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.401024  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:17.323876  5435 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.360911  5644 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:17.412166  1553 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:17.391422  5576 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:17.440402  1558 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:17.390969  5675 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.330163  5550 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.384925  5501 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.484628  5435 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:17.485321  5644 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:17.533856  5435 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.534358  5644 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.534590  5435 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:17.486220  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 85518us
I20260709 13:01:17.540170  2348 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:17.486773  5685 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:17.487907  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:17.494400  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.545372  1632 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:17.396888  5472 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
W20260709 13:01:17.495806  5678 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:01:17.325938  5612 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.327297  5676 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.361932  5565 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.523150  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:17.392431  5629 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:01:17.407814  5378 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.541268  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:17.541827  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:17.541343  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:17.565048  2347 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:17.548826  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:17.557386  2349 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:17.407939  5498 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.132s
I20260709 13:01:17.521037  5513 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.554908  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:17.538087  5527 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.418779  5494 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.578572  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:17.539958  5487 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:17.580816  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2376) took 1061 ms. Trace:
I20260709 13:01:17.581403  1630 rpcz_store.cc:276] 0709 13:01:16.519170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.519281 (+   111us) service_pool.cc:224] Handling call
0709 13:01:16.519599 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:16.523766 (+  4167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:16.523794 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:16.524233 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 13:01:16.601799 (+ 77566us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:16.601839 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.601903 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:17.280546 (+678643us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:17.564304 (+283758us) raft_consensus.cc:1685] finished
0709 13:01:17.564339 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:17.580720 (+ 16381us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7966,"raft.queue_time_us":3850,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":12966,"spinlock_wait_cycles":10074496,"thread_start_us":47359,"threads_started":1,"wal-append.queue_time_us":82398}
W20260709 13:01:17.583878  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.587694  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:17.588904  2346 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:17.413511  5650 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.180s
I20260709 13:01:17.591351  4416 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:17.549728  5640 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:17.600230  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 246615us
I20260709 13:01:17.574218  5378 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:17.404116  5627 consensus_queue.cc:1059] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.152s
I20260709 13:01:17.595571  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:17.661234  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 65814.6us
I20260709 13:01:17.532059  5675 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:17.589988  5629 consensus_queue.cc:1243] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:17.597309  5435 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:17.633731  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.659207  2348 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:01:17.592279  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 237744us
I20260709 13:01:17.683033  1602 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:17.663357  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:17.684984  2346 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:17.665536  5531 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:17.665804  2349 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:01:17.593225  5527 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.690433  5478 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:17.690979  5478 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:17.691680  5527 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:17.691900  5478 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.692493  5527 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:17.692998  5527 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:17.587591  5513 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.693276  5478 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:17.695258  5513 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:17.605527  5650 consensus_queue.cc:1243] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:01:17.698570  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.704890  5513 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:01:17.709180  5527 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:17.713254  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.602442  5640 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.559439  5635 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:17.704169  5666 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.003s
W20260709 13:01:17.732231  5688 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:01:17.732900  5680 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.544s	user 0.003s	sys 0.000s
I20260709 13:01:17.738443  5550 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:17.743175  5675 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:17.745973  5543 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:17.703882  5505 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.001s
I20260709 13:01:17.748000  5505 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.751300  5648 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:17.752769  5469 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.000s
I20260709 13:01:17.753418  5469 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.757454  5494 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:17.706521  5435 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:17.758802  5435 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.759183  5435 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:17.760061  1708 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:17.762797  5565 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:17.764986  5501 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:17.766738  5501 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.767246  5501 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.768159  5573 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:17.776319  5640 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:17.776685  1702 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:01:17.778798  5548 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.001s
I20260709 13:01:17.784348  5550 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:17.787137  1709 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [5.6-8.7]
I20260709 13:01:17.789614  1708 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:17.789927  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.791337  1702 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:17.785753  5550 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:17.566321  5685 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:17.751137  5505 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:17.832551  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2371) took 1889 ms. Trace:
I20260709 13:01:17.833261  1628 rpcz_store.cc:276] 0709 13:01:15.942908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.163712 (+220804us) service_pool.cc:224] Handling call
0709 13:01:16.164088 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:16.164532 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:16.164571 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:16.911216 (+746645us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b2c00070338. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:01:16.928206 (+ 16990us) log.cc:844] Serialized 26 byte log entry
0709 13:01:17.117398 (+189192us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:17.117461 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.117594 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:17.793831 (+676237us) raft_consensus.cc:1685] finished
0709 13:01:17.793854 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:17.832331 (+ 38477us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1209,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5859,"spinlock_wait_cycles":7311232,"thread_start_us":156608,"threads_started":1,"wal-append.queue_time_us":262895}
I20260709 13:01:17.794729  5548 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:17.839781  5689 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:17.797823  2350 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.616s	user 0.008s	sys 0.006s
W20260709 13:01:17.845721  5571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.076s	user 0.010s	sys 0.047s
W20260709 13:01:17.848979  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.852965  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.855340  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 279420us
I20260709 13:01:17.846552  5689 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:17.860200  2350 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:17.864256  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:17.850066  5539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.019s	sys 0.036s
W20260709 13:01:17.874516  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.864647  5675 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.876686  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.880090  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:17.881067  1628 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:17.859630  5469 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:17.857838  5571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.011s	sys 0.048s
W20260709 13:01:17.877444  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 194512us
W20260709 13:01:17.874532  5539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.019s	sys 0.040s
I20260709 13:01:17.885389  1602 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:17.882447  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:17.864259  5689 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:17.887627  1629 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:17.890127  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:17.890937  1629 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:17.892585  1705 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:17.879359  5494 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:17.871997  5487 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:17.896338  1709 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:17.902292  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:17.894351  5501 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:17.905496  5699 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:17.907122  5478 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:17.907795  5478 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.908756  5378 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:17.897460  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:17.880788  5675 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.875129  5685 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:17.954394  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:17.943686  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.657338   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:17.942440  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:17.952915  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:17.981017  2350 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:17.981705  1628 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:17.960486  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.893828  5634 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.983098  1560 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:17.935760  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:17.957675  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.956022  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:17.982355  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 379759us
I20260709 13:01:17.901130  5537 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:17.985102  5495 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:17.968590  5650 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 13:01:17.984824  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 86706.1us
I20260709 13:01:17.986454  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.986797  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.939927  5615 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:17.984885  2317 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:17.964707  5559 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:17.987644  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:18.027014  2346 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:17.919394  5478 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:17.988382  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:17.989137  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.989348  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.990573  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:17.990688  5501 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:17.896085  5644 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:17.941013  5700 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:17.941258  5699 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:17.996202  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:17.897205  5494 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.008875  5537 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:18.006549  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:18.007599  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:18.073143  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 65617.7us
I20260709 13:01:18.074013  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:17.944296  5562 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.588s	user 0.012s	sys 0.003s
I20260709 13:01:17.941944  5702 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.977018  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.078210  4416 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:18.016388  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:18.016940  2352 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:18.010897  5531 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:17.988291  5043 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.024910  5650 consensus_queue.cc:1243] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:18.054582  1557 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:18.061081  1706 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:18.062942  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:17.985280  5689 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.076604  1708 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:18.063335  5675 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.003108  5634 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.032322  5542 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:01:18.083750  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 77304us
I20260709 13:01:18.078008  5615 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:18.084849  5576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.007s	sys 0.016s
I20260709 13:01:18.085247  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2486) took 2140 ms. Trace:
I20260709 13:01:17.919112  5487 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.070221  5699 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:18.072969  5571 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.071625  5494 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.130700  1552 rpcz_store.cc:276] 0709 13:01:15.944436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.041988 (+ 97552us) service_pool.cc:224] Handling call
0709 13:01:17.081589 (+1039601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.108163 (+ 26574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.084969 (+976806us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400207e50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:18.085158 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2390049536}
I20260709 13:01:18.155910  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:18.050570  5666 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:01:18.082433  5539 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.079039  5469 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.055754  5627 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:01:18.084254  5543 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:18.130496  5576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.007s	sys 0.016s
I20260709 13:01:18.091373  5527 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.187386  5675 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:18.188032  5675 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.188558  5675 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:18.187752  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:18.190743  1674 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:18.072628  5644 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.192561  5695 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:18.194154  5378 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.193259  5543 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.194921  5527 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.133047  5562 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:18.093640  5531 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.191485  5709 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:18.071548  5501 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.081828  5435 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.166491  5495 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:18.216960  5698 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.216686  5600 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
I20260709 13:01:18.219120  5617 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.219972  5701 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.148s
I20260709 13:01:18.210489  5478 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:18.216773  5378 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.197770  5576 raft_consensus.cc:697] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:18.197463  5539 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.081183  5702 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.232075  5634 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.166847  5544 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:01:18.238763  4416 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:18.248361  5709 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:18.239224  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 243086us
W20260709 13:01:18.245167  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.274915  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.167870  5571 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.253198  5617 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.198710  5469 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.256767  1704 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:18.252347  5689 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.254463  5478 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.278491  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:18.285682  5478 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:18.240365  5562 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:18.287109  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 207782us
I20260709 13:01:18.285861  1559 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.289885  1557 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:18.291872  1553 raft_consensus.cc:2410] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:01:18.296855  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.300933  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.307834  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:18.240031  5699 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.309451  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.310633  1560 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:18.310994  5699 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:18.314536  5644 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.279897  5647 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:18.302850  5501 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.302394  5537 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.303521  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.280273  5634 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.305423  5531 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.279042  5709 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:18.250146  5495 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:18.302160  5712 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:01:18.324776  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:18.350832  1710 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.330399  1559 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:18.294699  5488 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.341466  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:18.343050  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 263025us
I20260709 13:01:18.288491  5689 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.289572  5711 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.344591  1558 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:18.262852  5698 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.315403  5576 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.371599  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:18.372308  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.388116  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:18.372594  1630 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:01:18.381702  5717 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.381894  5707 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:18.391677  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:18.393703  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:18.389353  1630 pending_rounds.cc:85] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 5
I20260709 13:01:18.390381  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:18.395866  1630 pending_rounds.cc:107] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:01:18.396292  1630 raft_consensus.cc:2889] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:18.406798  1627 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:18.348886  5700 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:18.352495  5501 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.353139  5537 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.413386  1633 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:18.415138  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:18.418087  1633 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:18.421789  1636 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:01:18.381220  1710 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:18.422298  1636 pending_rounds.cc:85] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 7
I20260709 13:01:18.422780  1636 pending_rounds.cc:107] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:01:18.423013  1636 raft_consensus.cc:2889] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:18.349805  5634 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.403187  5565 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.366003  5488 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.424216  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:18.430650  1710 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:18.435787  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:18.436796  1710 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:18.371215  5689 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.438504  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:18.414757  5644 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.418067  5716 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.439981  1632 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:01:18.345572  5647 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.441622  1710 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:18.448403  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:18.450170  1634 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [8.4-9.5]
W20260709 13:01:18.447019  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 106444us
I20260709 13:01:18.462848  1710 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:18.370173  5711 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.420257  5501 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.439723  5724 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.473802  1631 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:18.484666  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.495085  1559 raft_consensus.cc:2410] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:18.483084  1558 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:18.489868  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.489234  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:18.442909  5720 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.490935  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:18.499073  1634 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:18.492501  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.492652  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:18.467819  5435 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:18.505956  5719 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.506335  5495 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.506928  5505 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:18.526046  5732 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.510535  1632 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:18.496570  5539 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:18.509702  5617 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.412226  5707 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.412043  5717 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.519236  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:18.542517  1634 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:18.521353  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.521974  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:18.519222  5719 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.449409  5644 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.519114  5702 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.511029  1635 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:18.437429  5565 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.509159  5720 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.544086  1710 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:18.544209  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.498795  5700 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.559656  1632 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:18.563606  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:18.564710  1710 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:18.568375  2344 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:01:18.569911  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:18.576511  1636 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:18.536146  5562 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.578495  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:18.579521  1710 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:18.581811  1710 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:01:18.582886  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.584412  1710 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:18.592859  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:18.593829  2350 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:18.593617  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:18.594412  1632 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:18.448894  5716 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:18.499634  5724 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.607127  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:18.608271  1632 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:18.613554  5487 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:18.604092  5378 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:18.618818  5487 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.621266  5725 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.621416  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.632438  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 668) took 1236 ms. Trace:
I20260709 13:01:18.605964  5559 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.642295  1709 rpcz_store.cc:276] 0709 13:01:17.396211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.795336 (+399125us) service_pool.cc:224] Handling call
0709 13:01:17.795759 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:17.901202 (+105443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.631960 (+730758us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b5400250dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:18.632348 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230506368}
I20260709 13:01:18.644994  1709 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:01:18.648531  5727 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.139s	user 0.000s	sys 0.002s
I20260709 13:01:18.649092  1632 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:18.649089  5736 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.548105  5732 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:18.647256  5723 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.115s	user 0.001s	sys 0.001s
I20260709 13:01:18.653043  5736 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:18.663182  5647 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.672904  1634 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:18.664000  5689 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.655117  5559 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:18.691334  5494 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.007s
I20260709 13:01:18.659348  5478 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.702668  5494 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.648551  5725 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.727553  1633 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:18.729584  5740 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.729732  5550 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.732903  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:18.729214  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.735159  1560 raft_consensus.cc:2410] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:18.730542  5691 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:18.736603  1632 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:18.738896  5698 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:18.739459  5717 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.740119  5724 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.741360  5550 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:18.744526  5720 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.745761  5423 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:18.746260  5423 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:18.747005  5711 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.747366  5423 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.735224  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:18.758129  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:18.759397  1707 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:01:18.760495  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:18.761843  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:18.770720  1632 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:18.749032  5423 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:18.762701  5699 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.781698  1560 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:18.782745  1559 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:18.785377  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2482) took 1693 ms. Trace:
I20260709 13:01:18.785836  1703 rpcz_store.cc:276] 0709 13:01:17.092044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.270770 (+178726us) service_pool.cc:224] Handling call
0709 13:01:17.271171 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.549312 (+278141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:17.549360 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:17.549409 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:17.549448 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.785140 (+1235692us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b540016c954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:18.785195 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:18.785303 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639071488}
I20260709 13:01:18.787429  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:18.788686  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:18.788879  5498 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:18.789594  2318 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 2.188s (SENT)
I20260709 13:01:18.789657  5725 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.790894  5609 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.784943  5736 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:18.791450  5609 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.793087  5514 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:18.794543  5469 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:18.794829  5609 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.795277  5730 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:18.782682  5699 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.797751  5103 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.730283  5740 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:18.799144  5612 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.753197  5472 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
W20260709 13:01:18.804044  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.856500  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.804811  1628 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:18.805569  1560 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:18.809370  5732 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.867867  5566 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:18.868561  5566 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.810540  5527 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:18.812106  5510 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:18.812651  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.762461  5711 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:01:18.831396  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.877293  1555 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:18.834095  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:18.835518  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:18.879367  1632 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
W20260709 13:01:18.795433  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.881587  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:18.882382  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:18.882843  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:18.883476  2347 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:18.883801  5647 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.884721  2344 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:18.840021  5559 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.843037  1674 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:18.885862  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:18.762841  5691 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.858603  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:18.855777  5103 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.827437  5576 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:01:18.839093  5736 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.843586  5699 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.891849  5736 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:18.892323  5736 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.893532  5103 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.895182  5732 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:18.896804  5566 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.897660  5494 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:18.899223  5691 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:18.836719  5730 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:18.901351  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:18.902397  2344 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 6: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:18.884320  5647 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.789389  5498 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:18.904775  5576 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:18.904731  5498 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.906193  2352 raft_consensus.cc:2410] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:01:18.857188  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.889755  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 56010.8us
I20260709 13:01:18.885203  5626 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:01:18.856205  5487 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:18.886726  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 43825.1us
I20260709 13:01:18.905438  5576 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:18.909528  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:18.874737  5609 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.921818  5695 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.922468  5695 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:18.922389  5609 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:18.912423  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.914465  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:18.905995  5647 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:18.876762  5510 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:18.859376  5469 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:01:18.917927  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:18.906085  5498 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:18.911396  5675 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.719s	user 0.011s	sys 0.006s
I20260709 13:01:18.916872  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:18.928607  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.910840  5527 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.960294  1555 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:18.934556  1674 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:18.967582  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.968380  1554 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:18.972647  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:18.962244  5514 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:01:18.975883  5534 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:18.975683  5510 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:18.976444  5534 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.976933  5534 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.975234  5427 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.977378  5510 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:18.978142  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:18.996814  5706 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.918s	user 0.004s	sys 0.000s
I20260709 13:01:18.996901  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:19.007019  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:18.961597  5527 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.979982  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:18.984560  2349 raft_consensus.cc:2410] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:19.014153  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:19.014627  1632 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:19.014973  1635 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:18.994863  5427 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.015857  5427 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.987457  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:18.999749  5565 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:19.017529  5488 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.017606  1633 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
W20260709 13:01:19.019330  5682 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.606s	user 0.004s	sys 0.000s
I20260709 13:01:18.985997  5526 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.020723  5526 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:19.021185  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:19.022539  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.984995  5535 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:19.023504  1560 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:01:19.023468  5535 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:19.023590  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:19.024600  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:19.024794  5535 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.025764  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:19.026008  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:19.027369  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:19.028051  1558 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:19.029095  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:19.029798  1559 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:18.933020  5609 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:18.994038  5699 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:19.008770  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.928282  5494 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:19.010727  5634 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.013921  5527 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:19.014072  1631 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:19.031634  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 114934us
W20260709 13:01:19.034682  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.930313  5732 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:19.051311  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.052588  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.040002  1555 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:01:19.053651  1555 pending_rounds.cc:85] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 4
I20260709 13:01:19.053398  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:19.054152  1555 pending_rounds.cc:107] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:01:19.054440  1555 raft_consensus.cc:2889] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:19.055184  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:19.053651  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:19.060282  2317 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:19.061558  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:19.062425  1555 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
W20260709 13:01:19.070914  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.071578  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.073300  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:18.967415  5478 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:19.050196  5494 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:18.987401  5740 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:19.026017  5535 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:19.083745  1708 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:19.081244  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:19.121611  1555 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:19.085603  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:19.091128  1710 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:19.093405  1632 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
W20260709 13:01:19.130586  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45115.4us
I20260709 13:01:19.128322  1631 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
W20260709 13:01:19.132903  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.134752  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.096082  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:19.074293  5732 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:19.095072  5514 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.094792  5513 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:19.114035  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.100845  5523 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:01:19.139863  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.146389  1704 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:19.114874  1707 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:19.115787  1706 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:19.101656  2353 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:18.971213  5675 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.168219  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:18.980633  5736 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.060949  5609 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:19.136785  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.064863  5435 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:19.134196  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.139338  1674 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:19.183521  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48192.9us
I20260709 13:01:19.061368  5731 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:01:19.139322  1631 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:19.151798  5617 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:19.153199  5566 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:19.153522  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.074651  5697 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
W20260709 13:01:19.206431  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.153915  5492 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:19.154232  5716 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.154495  5559 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:19.156054  5549 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:19.159046  5544 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:19.160300  5615 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:19.219605  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:19.164956  5103 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:19.167217  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1882) took 2079 ms. Trace:
I20260709 13:01:19.215215  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:19.220573  1705 rpcz_store.cc:276] 0709 13:01:17.087480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.088426 (+   946us) service_pool.cc:224] Handling call
0709 13:01:17.089088 (+   662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.893217 (+804129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.166941 (+1273724us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54002e81d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:19.167145 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847218560}
I20260709 13:01:19.113831  5506 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:19.226943  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56416.7us
I20260709 13:01:19.136878  1635 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:01:19.103658  5494 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:19.100522  5510 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:19.169714  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.097860  5650 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:19.172799  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:19.168990  1555 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:19.175055  2344 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:19.246584  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:19.182075  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:19.184733  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:19.139586  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 135133us
I20260709 13:01:19.204890  1709 raft_consensus.cc:2368] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11.
I20260709 13:01:19.248615  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:19.207719  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.140511  5626 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:19.187105  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.212129  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.251466  1703 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:19.251901  1632 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:19.214056  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:19.215332  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:19.253455  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 69089.9us
I20260709 13:01:19.173264  5523 consensus_queue.cc:1243] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:19.182844  5435 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.218106  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:19.163290  5548 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:19.228076  5618 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:19.224807  1675 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:19.279628  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 54935.7us
I20260709 13:01:19.206876  5587 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:19.075323  5576 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.255927  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:19.183279  5513 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.257953  5543 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.258625  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.174683  5514 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.250048  1707 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:19.270920  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:19.201372  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.270869  5435 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:19.227809  5478 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:19.277180  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.227356  5506 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.265527  5720 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:01:19.181499  5724 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:01:19.250394  5707 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.281463  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:19.305047  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 127938us
W20260709 13:01:19.333464  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52116.9us
I20260709 13:01:19.306433  5650 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.266497  5697 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:19.145262  5537 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 13:01:19.314318  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.309137  5587 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:19.282982  5748 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:01:19.278754  5618 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:19.288836  5550 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:19.219477  5566 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.349107  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:19.212697  5617 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.271204  5732 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.321624  5492 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.323311  5506 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:19.329736  5514 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.342756  5510 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.333832  5675 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:19.358898  5720 consensus_queue.cc:1243] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:19.358575  5699 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.363291  5615 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:01:19.385666  5736 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:19.386164  5754 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:01:19.385504  5752 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:01:19.387173  5513 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:19.243335  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 62624.8us
W20260709 13:01:19.451550  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 186500us
W20260709 13:01:19.453045  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 225188us
I20260709 13:01:19.456961  1560 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:19.464867  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2385) took 1037 ms. Trace:
I20260709 13:01:19.466439  1628 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:19.475435  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2383) took 1053 ms. Trace:
W20260709 13:01:19.485755  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 171274us
I20260709 13:01:19.492156  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2384) took 1067 ms. Trace:
I20260709 13:01:19.346494  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:19.567849  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.569132  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.569811  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 223260us
I20260709 13:01:19.355032  1703 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:19.328033  1705 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:19.331267  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.385497  5510 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.575081  5566 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.575773  2344 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:19.374785  5550 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:19.578872  5550 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:19.579566  5550 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:19.395881  5618 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:19.582895  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 318853us
I20260709 13:01:19.583340  5618 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:19.359850  5650 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.364322  5697 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:19.404711  5695 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:19.408411  5709 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.040s	user 0.009s	sys 0.005s
I20260709 13:01:19.626350  5709 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:19.628149  1702 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:19.401353  5587 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.494174  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2395) took 1001 ms. Trace:
W20260709 13:01:19.556062  5498 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.594s	user 0.016s	sys 0.004s
I20260709 13:01:19.634970  2345 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:19.633759  1630 rpcz_store.cc:276] 0709 13:01:18.492784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.814128 (+321344us) service_pool.cc:224] Handling call
0709 13:01:18.814452 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:18.814671 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:18.814985 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 13:01:19.493356 (+678371us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.493522 (+   166us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:19.493651 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.493768 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.493971 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":414382,"prepare.queue_time_us":12575,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":44,"raft.queue_time_us":489795,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":154,"spinlock_wait_cycles":5504,"thread_start_us":263191,"threads_started":1,"wal-append.queue_time_us":391468}
W20260709 13:01:19.558689  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 213007us
I20260709 13:01:19.559218  5523 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:19.636605  5523 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.637084  5523 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:19.637639  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:19.639047  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.639467  5494 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.377269  5492 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.566764  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:19.560911  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:19.643519  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 77097.3us
W20260709 13:01:19.643694  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 83123.4us
I20260709 13:01:19.643978  5492 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.644917  5492 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:19.645418  5492 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:19.599835  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.551280  1636 rpcz_store.cc:276] 0709 13:01:18.430179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.431153 (+   974us) service_pool.cc:224] Handling call
0709 13:01:19.464085 (+1032932us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400197654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:19.464440 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288652032}
I20260709 13:01:19.573998  5617 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.604704  1707 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:19.575114  5534 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.551919  1629 rpcz_store.cc:276] 0709 13:01:18.422117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.425214 (+  3097us) service_pool.cc:224] Handling call
0709 13:01:18.440687 (+ 15473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.639312 (+198625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.639350 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:18.641694 (+  2344us) log.cc:844] Serialized 26 byte log entry
0709 13:01:19.007814 (+366120us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b1800401c80. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:01:19.205158 (+197344us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:19.207599 (+  2441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.474690 (+267091us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400349ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:19.474775 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:19.474847 (+    72us) raft_consensus.cc:1685] finished
0709 13:01:19.474862 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.475206 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":759,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":118,"spinlock_wait_cycles":411039360,"thread_start_us":197101,"threads_started":1,"wal-append.queue_time_us":657925}
I20260709 13:01:19.611863  5650 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.587968  5697 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.614125  5501 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.614894  5510 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.612356  5559 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:19.581369  5550 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.569850  5759 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:01:19.645967  5548 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:19.553274  1627 rpcz_store.cc:276] 0709 13:01:18.425173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.425518 (+   345us) service_pool.cc:224] Handling call
0709 13:01:18.425940 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:18.575495 (+149555us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:18.575873 (+   378us) log.cc:844] Serialized 14 byte log entry
0709 13:01:18.670993 (+ 95120us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:18.671069 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:18.671121 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.491474 (+820353us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54003b92d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:19.491578 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.491941 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12562,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":117,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":7013,"spinlock_wait_cycles":343104384,"thread_start_us":94721,"threads_started":1,"wal-append.queue_time_us":89105}
I20260709 13:01:19.646078  5492 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:19.621045  5526 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.600s	user 0.015s	sys 0.004s
W20260709 13:01:19.661247  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.655733  5635 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:19.657572  5644 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.706713  5510 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:19.707049  5662 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:19.707613  5662 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.708148  5662 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:19.709723  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.711119  5571 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:19.711776  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.711561  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.712281  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:19.712752  5719 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.697245  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.712975  5673 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.713497  5719 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.714607  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.715278  1707 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:19.716279  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.716967  1707 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:19.657757  5724 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:01:19.737983  1709 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:19.678511  5709 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:19.691737  5498 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.717140  5618 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:01:19.746845  5498 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.752959  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:19.754086  1627 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:01:19.755090  5501 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:19.755587  5501 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.655150  5711 consensus_queue.cc:1243] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer 95fc6a4cf5874b7eb91280dffcd3b3fb log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:19.755936  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:19.757855  5673 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.657089  5505 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.703548  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:19.711052  5650 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.714268  5719 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.718216  5103 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:19.718511  5648 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:19.766086  5740 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:19.766884  5740 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:19.766222  5648 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.768347  5740 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.702176  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:19.720768  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:19.722808  5488 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:19.696844  5666 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.666517  5676 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:19.777696  5724 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:19.778111  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:19.778616  5523 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.779541  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:19.780930  5378 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:19.726994  5534 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:19.749385  5498 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.781839  5711 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:19.782403  5711 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.782110  5505 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.686941  5548 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.729292  1635 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:19.783454  5548 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:19.784317  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:19.784600  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:19.785094  1708 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:19.727625  5427 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.711s	user 0.018s	sys 0.003s
I20260709 13:01:19.731528  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:19.733723  5760 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:19.735482  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:19.736107  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 163135us
I20260709 13:01:19.661412  5513 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:01:19.673906  5559 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:19.697669  5487 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.848800  5487 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:19.842922  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 139541us
I20260709 13:01:19.849431  5559 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.851687  5487 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.716753  5707 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:19.852692  5559 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:19.720788  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.781869  5666 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.702545  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2397) took 1159 ms. Trace:
I20260709 13:01:19.707526  5510 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:19.753235  5612 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.878s	user 0.014s	sys 0.005s
I20260709 13:01:19.746301  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.720705  5695 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:19.769773  5650 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.894518  5609 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.895313  5650 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.895215  5609 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.895658  5760 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:19.896843  5492 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.771733  5762 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:19.908998  5762 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:19.909747  5740 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:19.782835  5711 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:19.755964  5501 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:19.730348  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 48564.9us
I20260709 13:01:19.781414  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:19.785456  1635 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:19.834627  2346 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:19.776820  5506 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.789302  5427 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.835804  2352 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:19.836318  5527 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.772s	user 0.016s	sys 0.005s
I20260709 13:01:19.840905  2351 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:01:19.841295  5378 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:19.841645  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.842043  1707 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [5.6-8.7]
I20260709 13:01:19.842213  1630 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:19.842978  5523 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.844533  5719 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.770905  5103 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.846199  1526 leader_election.cc:400] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11.
I20260709 13:01:19.947791  1526 leader_election.cc:403] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:19.774039  5488 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:19.746210  1710 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:19.792896  5618 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.142s
I20260709 13:01:19.790987  1636 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:19.888973  2344 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:19.856036  1633 rpcz_store.cc:276] 0709 13:01:18.542897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.018781 (+475884us) service_pool.cc:224] Handling call
0709 13:01:19.019176 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.019435 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:19.019784 (+   349us) log.cc:844] Serialized 14 byte log entry
0709 13:01:19.701797 (+682013us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.701935 (+   138us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:19.702073 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.702194 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.702358 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":299171,"prepare.queue_time_us":3344,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":338879,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":164,"spinlock_wait_cycles":6416768,"thread_start_us":360109,"threads_started":2,"wal-append.queue_time_us":294154}
I20260709 13:01:19.890868  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:19.892201  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:19.892697  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.770845  5648 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:19.913085  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.785148  5505 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.914948  1708 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:19.926043  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:19.860478  1629 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:01:19.965966  5423 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.966325  1631 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
W20260709 13:01:19.966696  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 56193.4us
I20260709 13:01:19.967254  5549 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:19.967756  1627 raft_consensus.cc:1217] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Deduplicated request from leader. Original: 5.7->[6.8-8.9]   Dedup: 6.8->[8.9-8.9]
I20260709 13:01:19.968447  1627 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 13:01:19.968554  5609 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:19.969906  5492 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.967780  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:19.971894  5103 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.973798  1704 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:19.975306  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:19.975697  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 129835us
I20260709 13:01:19.976614  5724 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:19.976787  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:19.945302  5494 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:19.910444  5762 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:19.977675  2350 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
W20260709 13:01:19.978240  5435 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.630s	user 0.007s	sys 0.010s
I20260709 13:01:19.978507  5762 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:19.978838  5435 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:19.981684  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:19.960794  5760 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:01:19.970024  1705 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:19.945226  5510 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:19.990185  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:19.993134  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:19.985199  1526 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 2.097s (SENT) [suppressed 7 similar messages]
I20260709 13:01:19.964493  5707 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:20.001883  2348 raft_consensus.cc:2410] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:20.002532  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:19.947422  5527 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:20.010545  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 229225us
I20260709 13:01:20.002676  5492 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:20.011873  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:20.007661  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.014178  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:20.016000  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:20.000178  5724 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:01:20.029721  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 178060us
I20260709 13:01:20.008466  5494 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:20.020848  5492 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:19.966730  5423 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.018112  5435 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:20.037619  1635 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
W20260709 13:01:20.039316  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 71733.5us
I20260709 13:01:20.040560  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:20.047037  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.050422  1708 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:20.054817  1703 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:20.049947  1560 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:20.057278  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.057331  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:20.053754  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:20.054271  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:20.063412  5378 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:20.064658  5647 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.065445  5647 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.066226  5427 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:20.066684  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2392) took 1605 ms. Trace:
I20260709 13:01:20.068770  5506 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:20.067159  1634 rpcz_store.cc:276] 0709 13:01:18.461741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.674392 (+212651us) service_pool.cc:224] Handling call
0709 13:01:18.674667 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:18.674846 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:18.690749 (+ 15903us) log.cc:844] Serialized 14 byte log entry
0709 13:01:18.785295 (+ 94546us) log.cc:844] Serialized 14 byte log entry
0709 13:01:18.980072 (+194777us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b18003ab940. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:19.484737 (+504665us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b18003ab940. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:01:19.914874 (+430137us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.053634 (+138760us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:20.060284 (+  6650us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:20.066225 (+  5941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.066318 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.066479 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":794859,"prepare.queue_time_us":460448,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":110,"raft.queue_time_us":782664,"raft.run_cpu_time_us":483,"raft.run_wall_time_us":483,"spinlock_wait_cycles":68546688,"thread_start_us":505312,"threads_started":3,"wal-append.queue_time_us":162553,"wal-append.run_cpu_time_us":2071,"wal-append.run_wall_time_us":1040445}
I20260709 13:01:20.071895  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.073093  1553 raft_consensus.cc:2410] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:01:20.073746  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:20.074023  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.074668  2353 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:20.075106  1556 raft_consensus.cc:2410] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:20.075860  1706 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:20.040895  5673 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:20.077153  5772 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:01:20.077860  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.078208  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:20.078969  1557 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:20.079186  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.080739  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:20.081645  2353 raft_consensus.cc:2410] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:01:20.086045  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.079123  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:20.089974  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.102043  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:20.103282  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 75224.9us
W20260709 13:01:20.109189  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.112118  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:20.117918  5760 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:20.118659  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.119653  1556 raft_consensus.cc:2410] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:20.120471  5770 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:20.122658  1705 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: [8.7-11.8]
I20260709 13:01:20.125499  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:20.126931  5775 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:20.127774  5775 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:20.128149  5775 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:20.128531  5775 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:20.129945  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.094009  5494 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:20.151860  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.155766  1630 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:20.162624  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:20.163681  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:20.114639  5697 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:20.172787  1635 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:20.151437  5673 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:20.179497  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.179279  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:20.184155  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 58798.6us
I20260709 13:01:20.186334  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.161835  5648 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.191577  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:20.192958  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 153835us
I20260709 13:01:20.171515  5697 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.172590  5494 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:20.164489  5760 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.212014  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.207698  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:20.204977  2353 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:20.182796  5673 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.209656  5427 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:20.220526  1553 raft_consensus.cc:2410] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:20.220667  5427 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.221436  5427 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:20.224788  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.226227  1630 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:20.226475  1560 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:01:20.226925  1560 pending_rounds.cc:85] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 6
I20260709 13:01:20.227161  1560 pending_rounds.cc:107] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:01:20.227512  1560 raft_consensus.cc:2889] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:20.229806  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 94860.1us
I20260709 13:01:20.232486  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 953) took 1473 ms. Trace:
I20260709 13:01:20.232966  2350 rpcz_store.cc:276] 0709 13:01:18.759280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.760171 (+   891us) service_pool.cc:224] Handling call
0709 13:01:20.232202 (+1472031us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400796b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:20.232400 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2656954240}
I20260709 13:01:20.233752  5648 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.234563  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:20.235239  2350 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:20.236589  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.236613  5648 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:20.245819  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2494) took 1115 ms. Trace:
I20260709 13:01:20.245385  5427 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:20.246656  1710 rpcz_store.cc:276] 0709 13:01:19.130392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.136430 (+  6038us) service_pool.cc:224] Handling call
0709 13:01:20.245509 (+1109079us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400223654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:20.245734 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147448064}
I20260709 13:01:20.247256  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1102) took 1059 ms. Trace:
I20260709 13:01:20.248745  5427 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:20.247718  2347 rpcz_store.cc:276] 0709 13:01:19.187445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.187665 (+   220us) service_pool.cc:224] Handling call
0709 13:01:19.188090 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.272839 (+ 84749us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:19.574173 (+301334us) log.cc:844] Serialized 14 byte log entry
0709 13:01:19.935001 (+360828us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.945970 (+ 10969us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:19.987207 (+ 41237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.246755 (+259548us) spinlock_profiling.cc:243] Waited 84.4 ms on lock 0x7b540005bc54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:20.246807 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.247027 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10293,"prepare.queue_time_us":133125,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":16519,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":10388,"spinlock_wait_cycles":194057984,"thread_start_us":610495,"threads_started":2,"wal-append.queue_time_us":460422}
I20260709 13:01:20.248746  1710 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:01:20.206755  5647 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:20.251351  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 273905us
I20260709 13:01:20.204013  5697 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:20.259862  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:20.263736  2352 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:20.264364  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:20.265779  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 50608.2us
W20260709 13:01:20.259877  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52173.5us
I20260709 13:01:20.272294  5494 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.274562  1710 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:20.268011  1602 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:20.275339  1635 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:20.277776  5505 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:20.278987  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:20.251078  5673 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:20.315466  2353 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:20.319185  2353 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:20.321130  2353 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:20.281704  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.282065  5423 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:20.282446  5755 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.922s	user 0.004s	sys 0.012s
I20260709 13:01:20.283179  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:20.283902  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.287847  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 208914us
W20260709 13:01:20.289664  5548 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.007s
I20260709 13:01:20.265362  5648 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.337745  5648 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.338493  5648 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.339373  1675 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.292248  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:20.341207  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 263570us
I20260709 13:01:20.342715  5492 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.343417  5492 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:20.344305  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2496) took 1213 ms. Trace:
I20260709 13:01:20.344475  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.344732  1702 rpcz_store.cc:276] 0709 13:01:19.131040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.511449 (+380409us) service_pool.cc:224] Handling call
0709 13:01:19.512389 (+   940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.628629 (+116240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.344029 (+715400us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54000cb2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:20.344241 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265275648}
W20260709 13:01:20.297019  2351 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 659) took 2756 ms (client timeout 2023 ms). Trace:
I20260709 13:01:20.297652  5760 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:20.302953  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:20.322940  5501 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:20.311030  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.312583  5773 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.157s	user 0.002s	sys 0.000s
W20260709 13:01:20.322330  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 62151.1us
I20260709 13:01:20.323073  5549 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.324424  5617 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:20.325387  5782 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:20.313172  5527 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:20.332192  5505 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:20.328265  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 221963us
I20260709 13:01:20.323451  5762 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:20.287876  5662 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.577s	user 0.015s	sys 0.000s
I20260709 13:01:20.345211  1558 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
W20260709 13:01:20.375233  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.375864  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.331197  5488 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:20.295193  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 951) took 1538 ms. Trace:
I20260709 13:01:20.348543  2347 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:20.307188  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.352054  1557 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
W20260709 13:01:20.348202  2351 rpcz_store.cc:269] 0709 13:01:17.540870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.540994 (+   124us) service_pool.cc:224] Handling call
0709 13:01:20.296693 (+2755699us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b5400478954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:20.296896 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5235786112}
I20260709 13:01:20.280756  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.357003  5549 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.359696  5709 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:20.378858  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.385810  2350 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:20.383280  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2499) took 1251 ms. Trace:
I20260709 13:01:20.357224  5617 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:20.386734  1704 rpcz_store.cc:276] 0709 13:01:19.131826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.573276 (+441450us) service_pool.cc:224] Handling call
0709 13:01:19.573886 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.974314 (+400428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.382999 (+408685us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b540036d8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:20.383214 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":884857728}
I20260709 13:01:20.334901  5759 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:01:20.378979  2353 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:01:20.392210  2353 pending_rounds.cc:85] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 8
I20260709 13:01:20.381876  2348 rpcz_store.cc:276] 0709 13:01:18.757268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.757820 (+   552us) service_pool.cc:224] Handling call
0709 13:01:20.294851 (+1537031us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400690854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:20.295113 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2619377536}
I20260709 13:01:20.399333  1525 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:20.400612  2353 pending_rounds.cc:107] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:01:20.401187  2353 raft_consensus.cc:2889] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:20.360734  5505 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.403805  5505 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:20.404378  5427 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:20.404852  5617 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.404049  5343 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:20.406163  5617 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:20.403995  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:20.362157  5762 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:20.389056  5559 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.536s	user 0.015s	sys 0.000s
I20260709 13:01:20.384940  5549 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.353035  5548 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.419039  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:20.421707  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 82504.6us
I20260709 13:01:20.429590  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:20.430529  1635 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.431270  5738 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.432011  5738 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:20.436146  2352 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:20.430548  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:20.437683  1628 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:20.439399  5548 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:20.443306  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.444015  5544 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:20.444360  5427 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:20.444536  5544 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:20.445858  1675 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.446755  5427 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:20.446873  5760 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:20.449893  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2504) took 1316 ms. Trace:
I20260709 13:01:20.450527  1701 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
W20260709 13:01:20.451005  5778 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:01:20.450603  1706 rpcz_store.cc:276] 0709 13:01:19.133096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.148713 (+ 15617us) service_pool.cc:224] Handling call
0709 13:01:20.449458 (+1300745us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b540016ba54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:20.449814 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1774001536}
W20260709 13:01:20.452363  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 654) took 4848 ms (client timeout 2188 ms). Trace:
I20260709 13:01:20.452674  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1098) took 1286 ms. Trace:
W20260709 13:01:20.452858  1701 rpcz_store.cc:269] 0709 13:01:15.603889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.604007 (+   118us) service_pool.cc:224] Handling call
0709 13:01:20.451902 (+4847895us) spinlock_profiling.cc:243] Waited 4.76 s on lock 0x7b5400246654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:20.452236 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10943785344}
I20260709 13:01:20.453744  5569 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.454149  5730 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.455436  5730 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:20.454999  5569 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.425863  1552 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:20.415794  5762 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:20.462016  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 82447.8us
W20260709 13:01:20.458737  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 151655us
I20260709 13:01:20.453171  2345 rpcz_store.cc:276] 0709 13:01:19.166068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.171028 (+  4960us) service_pool.cc:224] Handling call
0709 13:01:19.171383 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.635390 (+464007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.452395 (+817005us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b5400642dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:20.452616 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1064595712}
I20260709 13:01:20.389204  5662 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.470958  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.452881  5770 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:20.492501  5724 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11.
I20260709 13:01:20.504725  2347 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:20.505119  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:20.511157  2352 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:01:20.511696  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.519651  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:20.514348  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:20.527846  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:20.535724  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.538166  1631 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:20.532863  1629 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:20.550248  2353 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 13:01:20.573621  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 127860us
I20260709 13:01:20.546195  5724 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:20.593287  5724 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.594077  5724 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:20.594616  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.597544  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:20.598781  1631 raft_consensus.cc:2410] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:01:20.605206  5630 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.606289  5770 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:20.606103  5630 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.608763  5738 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:20.542339  5612 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.605036  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:20.624603  1634 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:01:20.625448  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52542.1us
W20260709 13:01:20.626500  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 71031.4us
W20260709 13:01:20.633741  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.630694  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.637743  5770 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:20.639552  5711 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.709s	user 0.011s	sys 0.003s
I20260709 13:01:20.640067  5711 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.641703  1627 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: [6.7-9.8]
I20260709 13:01:20.644428  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2598) took 1609 ms. Trace:
I20260709 13:01:20.644923  1632 rpcz_store.cc:276] 0709 13:01:19.034945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.252841 (+217896us) service_pool.cc:224] Handling call
0709 13:01:19.263488 (+ 10647us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.945123 (+681635us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:20.058322 (+113199us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.480652 (+422330us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.480886 (+   234us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:20.489445 (+  8559us) log.cc:844] Serialized 26 byte log entry
0709 13:01:20.489643 (+   198us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:20.489676 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.489786 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.643949 (+154163us) raft_consensus.cc:1685] finished
0709 13:01:20.643971 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.644190 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":265668,"prepare.queue_time_us":157063,"prepare.run_cpu_time_us":186,"prepare.run_wall_time_us":12615,"raft.queue_time_us":305928,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":222,"spinlock_wait_cycles":22611840,"thread_start_us":669918,"threads_started":3,"wal-append.queue_time_us":522779}
I20260709 13:01:20.647574  5609 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.644949  5501 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:20.648355  5609 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.644510  5548 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:01:20.652233  5548 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:20.652720  5548 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:20.654284  5789 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:01:20.653967  5548 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:20.546998  5544 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:20.661371  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:20.667503  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:20.629755  5780 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:20.636428  5740 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.726s	user 0.009s	sys 0.006s
I20260709 13:01:20.623827  5630 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:20.622083  5612 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.670235  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:20.695369  5711 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:20.663614  5775 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.005s
I20260709 13:01:20.668586  5548 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:20.696635  5775 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.700122  5775 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:20.702867  5662 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:20.664016  1630 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:20.671586  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:20.695546  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:20.677853  1675 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.686079  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:20.690407  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 61513.2us
I20260709 13:01:20.720180  5770 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:20.693634  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 113475us
W20260709 13:01:20.731081  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.738701  1634 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:20.739388  1628 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:01:20.690606  5501 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:20.708380  5762 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.746062  1602 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.740507  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:20.751768  1630 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:20.701198  5775 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:01:20.721086  5770 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:01:20.670989  5780 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.773192  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:20.779752  1631 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
W20260709 13:01:20.780690  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 151231us
I20260709 13:01:20.738497  5760 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.777354  1635 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:20.785995  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:20.787303  1629 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:20.793514  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.794914  5612 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:20.795145  2317 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:20.795746  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.796487  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.798451  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:20.797284  5662 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:20.801864  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:20.804102  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:20.806816  1631 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:20.808702  1556 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:20.809324  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 68938.7us
I20260709 13:01:20.742138  5435 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:20.814317  5492 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:20.814914  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:20.815686  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 146079us
I20260709 13:01:20.818015  5709 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:20.818436  5569 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:20.819710  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2498) took 1688 ms. Trace:
I20260709 13:01:20.860136  1630 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 7: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:20.801157  5559 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.860265  1703 rpcz_store.cc:276] 0709 13:01:19.131571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.572497 (+440926us) service_pool.cc:224] Handling call
0709 13:01:19.573241 (+   744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.062278 (+489037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.819323 (+757045us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b540028da54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:20.819602 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":874017152}
I20260709 13:01:20.863758  1635 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:20.866755  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:20.827262  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:20.827637  5780 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:20.856576  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:20.839913  5762 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:20.784657  5711 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:20.843590  5666 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:20.845853  5609 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:20.848940  5673 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.535s	user 0.013s	sys 0.000s
I20260709 13:01:20.850832  5724 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:20.854128  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 52357us
I20260709 13:01:20.780972  5617 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.821780  5760 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:20.862495  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:20.812709  5630 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:20.810793  5612 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:20.785995  5378 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
W20260709 13:01:20.881012  5697 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.607s	user 0.014s	sys 0.000s
I20260709 13:01:20.859880  5492 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.858408  5662 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:20.888926  1631 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:20.891664  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:20.900038  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 46369.6us
I20260709 13:01:20.901296  1633 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:01:20.889735  5569 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:20.906376  5662 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.908854  5662 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.910166  1632 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:20.907682  1633 pending_rounds.cc:85] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 6
I20260709 13:01:20.912504  1633 pending_rounds.cc:107] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:01:20.912796  1633 raft_consensus.cc:2889] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:20.912963  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2508) took 1248 ms. Trace:
I20260709 13:01:20.913602  5707 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.914165  5707 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.910084  5492 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.897876  5630 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.917649  5492 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:20.917929  5630 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:20.920017  1632 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [13.9-14.10]
I20260709 13:01:20.887833  5709 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:20.923092  1630 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:20.913403  1705 rpcz_store.cc:276] 0709 13:01:19.664785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.123813 (+459028us) service_pool.cc:224] Handling call
0709 13:01:20.124310 (+   497us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.169204 (+ 44894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.169243 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:20.214370 (+ 45127us) log.cc:844] Serialized 26 byte log entry
0709 13:01:20.519615 (+305245us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:20.554481 (+ 34866us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.613143 (+ 58662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.613246 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.912711 (+299465us) raft_consensus.cc:1685] finished
0709 13:01:20.912735 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.912897 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93724,"prepare.queue_time_us":34957,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":3162,"raft.queue_time_us":102990,"raft.run_cpu_time_us":476,"raft.run_wall_time_us":41687,"spinlock_wait_cycles":11977984,"thread_start_us":279141,"threads_started":1,"wal-append.queue_time_us":628547}
I20260709 13:01:20.922766  5709 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:20.924445  1630 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:20.921640  1632 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:20.906729  5617 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.926949  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:20.928241  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2506) took 1264 ms. Trace:
I20260709 13:01:20.928325  5617 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:20.898363  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 488714us
I20260709 13:01:20.928753  1708 rpcz_store.cc:276] 0709 13:01:19.664119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.093380 (+429261us) service_pool.cc:224] Handling call
0709 13:01:20.112917 (+ 19537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.160614 (+ 47697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.160663 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:20.189545 (+ 28882us) log.cc:844] Serialized 26 byte log entry
0709 13:01:20.545645 (+356100us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:20.545689 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.545772 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.928026 (+382254us) raft_consensus.cc:1685] finished
0709 13:01:20.928048 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.928179 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18324,"raft.queue_time_us":51097,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":23309,"thread_start_us":263810,"threads_started":1,"wal-append.queue_time_us":666555}
W20260709 13:01:20.947191  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.961206  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:20.947220  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 526970us
I20260709 13:01:20.962518  2347 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:20.964082  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:20.961697  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.965297  2352 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:20.923106  5576 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:20.907368  5549 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:20.969269  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 93269.5us
I20260709 13:01:20.955750  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:20.959019  5786 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:01:20.971050  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:20.890388  1635 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:20.976418  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.884851  5673 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:20.888373  5559 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.000093  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:20.996721  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 674) took 1291 ms. Trace:
I20260709 13:01:21.007058  2344 rpcz_store.cc:276] 0709 13:01:19.705138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.046975 (+341837us) service_pool.cc:224] Handling call
0709 13:01:20.047445 (+   470us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:20.048007 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.048049 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:20.048806 (+   757us) log.cc:844] Serialized 52 byte log entry
0709 13:01:20.211764 (+162958us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:20.527016 (+315252us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.545196 (+ 18180us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.996327 (+451131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.996436 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.996480 (+    44us) raft_consensus.cc:1685] finished
0709 13:01:20.996496 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.996653 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31913,"prepare.queue_time_us":1020349,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":72237,"raft.run_cpu_time_us":449,"raft.run_wall_time_us":18383,"thread_start_us":852839,"threads_started":3,"wal-append.queue_time_us":302830}
W20260709 13:01:21.018249  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 106211us
I20260709 13:01:20.905345  5697 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.030704  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:21.031181  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:21.031491  2352 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:21.031766  2347 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.031009  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:21.034520  2348 raft_consensus.cc:2368] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8.
I20260709 13:01:21.037268  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:21.038277  1705 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:21.040565  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:21.036922  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:21.037590  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:21.046303  1706 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:21.046573  1675 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:21.024708  5648 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.686s	user 0.012s	sys 0.003s
I20260709 13:01:21.011755  5378 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:21.051854  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:21.042695  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:21.043344  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:21.072194  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.076917  1701 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:21.012310  5780 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.035013  5612 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.082230  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:21.082233  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:21.098615  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.091511  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:21.090358  2346 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:21.086366  1710 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.107138  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:21.107734  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.109249  1560 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:21.116693  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 260227us
I20260709 13:01:21.118131  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:21.072909  5427 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:21.119308  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 119245us
I20260709 13:01:21.042080  5548 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:01:21.083573  5501 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:21.067864  5648 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.142199  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:21.147058  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:21.108631  5618 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:21.155910  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:21.159271  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.099606  5780 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:21.169552  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.170641  1708 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:21.170989  2347 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:21.163727  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 735) took 1439 ms. Trace:
I20260709 13:01:21.173856  1633 rpcz_store.cc:276] 0709 13:01:19.723907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.972477 (+248570us) service_pool.cc:224] Handling call
0709 13:01:19.972911 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.913331 (+940420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.155437 (+242106us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b54002dbed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:21.163650 (+  8213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2130441600}
I20260709 13:01:21.138722  5802 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:21.176839  5609 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.113806  5612 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.199723  2351 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:21.200415  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:21.191665  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:21.201264  1633 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.163304  5569 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:21.204272  1557 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:21.214097  1630 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:21.176851  5492 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.178293  5730 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.209535  5378 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:21.237300  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45829.4us
I20260709 13:01:21.238250  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:21.226174  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:21.198276  5759 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.220557  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:21.163591  5494 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:01:21.210745  5569 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:21.261864  1704 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:21.266822  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:21.228238  5760 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.240567  5806 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:21.250607  5782 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:21.265280  5569 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:21.306252  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.271759  5673 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:21.276539  5343 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:21.215408  5609 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:21.323130  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 149242us
I20260709 13:01:21.212268  5802 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:21.250633  5730 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:21.332139  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 224960us
I20260709 13:01:21.239185  5492 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:21.363169  2318 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:21.364063  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:21.357693  5343 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:21.347675  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 307786us
W20260709 13:01:21.326063  5794 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.299s	user 0.002s	sys 0.000s
I20260709 13:01:21.405853  1701 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
W20260709 13:01:21.407852  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44777.1us
W20260709 13:01:21.350646  5808 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:01:21.341929  5505 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.937s	user 0.013s	sys 0.003s
W20260709 13:01:21.349757  5805 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:01:21.328117  5740 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:21.359755  5612 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.343114  5648 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.343158  5817 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:01:21.418200  1629 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.001s
I20260709 13:01:21.419179  5343 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:21.419763  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1110) took 1074 ms. Trace:
I20260709 13:01:21.343708  5494 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 13:01:21.324905  5815 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.459285  5830 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:21.422672  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 58699.5us
I20260709 13:01:21.330549  5697 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:21.269101  5759 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.348094  5807 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:21.461447  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:21.324378  5501 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:21.354483  5811 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:01:21.429598  5548 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.439071  1629 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:21.456632  5569 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:21.416812  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:21.445971  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:21.458120  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:21.458841  5815 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:21.428681  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 155805us
I20260709 13:01:21.416764  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:21.443868  2350 rpcz_store.cc:276] 0709 13:01:20.345443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.448179 (+102736us) service_pool.cc:224] Handling call
0709 13:01:20.883244 (+435065us) spinlock_profiling.cc:243] Waited 8.99 ms on lock 0x7b5800003c98. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:01:20.883474 (+   230us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.901945 (+ 18471us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:20.901995 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:20.911919 (+  9924us) log.cc:844] Serialized 26 byte log entry
0709 13:01:21.148980 (+237061us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:21.149045 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.150205 (+  1160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.419407 (+269202us) raft_consensus.cc:1685] finished
0709 13:01:21.419431 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.419689 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7158,"raft.queue_time_us":18482,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":4856,"spinlock_wait_cycles":20668288,"thread_start_us":224869,"threads_started":1,"wal-append.queue_time_us":385079}
I20260709 13:01:21.437242  5103 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.485735  5103 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.473361  1559 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.470709  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:21.479362  1555 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:21.489357  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2513) took 1048 ms. Trace:
I20260709 13:01:21.495682  5801 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:21.498185  5801 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:21.491143  5569 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:21.494949  5615 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:21.502991  5615 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.503430  5615 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:21.501783  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.495771  1709 rpcz_store.cc:276] 0709 13:01:20.440751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.462063 (+ 21312us) service_pool.cc:224] Handling call
0709 13:01:20.482061 (+ 19998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.521917 (+ 39856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.521968 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:20.823105 (+301137us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b2c000732a8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:01:20.823763 (+   658us) log.cc:844] Serialized 26 byte log entry
0709 13:01:21.073251 (+249488us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:21.073301 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.073385 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.474181 (+400796us) raft_consensus.cc:1685] finished
0709 13:01:21.474211 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.489260 (+ 15049us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1853,"raft.queue_time_us":7160,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":25016,"spinlock_wait_cycles":4180224,"thread_start_us":248885,"threads_started":1,"wal-append.queue_time_us":413013}
I20260709 13:01:21.439345  5762 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:21.504155  5815 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.511340  5566 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:21.511761  5759 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:21.465128  5740 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:21.429816  5505 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:21.525502  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.516132  5566 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:21.504599  5817 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:21.513439  5800 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:21.495136  5617 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:01:21.555634  1634 raft_consensus.cc:2368] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8.
I20260709 13:01:21.518422  5806 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:21.542510  5759 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:21.483636  5548 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:21.560708  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 278343us
I20260709 13:01:21.548529  5780 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:21.559623  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1112) took 1168 ms. Trace:
W20260709 13:01:21.569634  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 74460.3us
I20260709 13:01:21.557821  5523 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.505999  5378 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:21.561209  5759 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:21.568907  1674 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:21.582619  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.565518  2345 rpcz_store.cc:276] 0709 13:01:20.391272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.463990 (+ 72718us) service_pool.cc:224] Handling call
0709 13:01:20.481601 (+ 17611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.489753 (+  8152us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:20.970475 (+480722us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.556980 (+586505us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.557080 (+   100us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:21.557148 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.557269 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.559543 (+  2274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43163,"prepare.queue_time_us":392687,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":214223,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":7484,"spinlock_wait_cycles":19858688,"thread_start_us":1010154,"threads_started":3,"wal-append.queue_time_us":274730}
W20260709 13:01:21.599253  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 82594.2us
I20260709 13:01:21.614971  1710 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:21.616946  1633 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
W20260709 13:01:21.621789  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 73677.6us
W20260709 13:01:21.599388  5822 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:01:21.624891  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:21.573809  5343 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:01:21.602499  5697 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:21.602099  5505 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:21.630179  5617 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.638038  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 739) took 1912 ms. Trace:
I20260709 13:01:21.638410  5617 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.638475  1627 rpcz_store.cc:276] 0709 13:01:19.725335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.969687 (+244352us) service_pool.cc:224] Handling call
0709 13:01:19.970182 (+   495us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:20.642229 (+672047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.635916 (+993687us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b54002dcdd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:21.637961 (+  2045us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1542978432}
I20260709 13:01:21.610210  5720 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.640074  5720 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:21.642565  5615 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:21.644690  5724 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.645843  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:21.562418  2350 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0adeaf85877a4103bbd070927413d267"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:60938
I20260709 13:01:21.648077  2350 raft_consensus.cc:606] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:21.648413  2350 raft_consensus.cc:614] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:21.650156  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:21.650484  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.626672  5802 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.655701  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:21.648363  5630 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.730s	user 0.011s	sys 0.001s
I20260709 13:01:21.645296  5615 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.630877  5492 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.599761  5780 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:01:21.633970  5730 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.635094  5840 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.667151  5523 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:21.667692  5720 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:21.668448  5843 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.085s	user 0.003s	sys 0.000s
W20260709 13:01:21.670269  5576 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.671s	user 0.012s	sys 0.001s
I20260709 13:01:21.637637  5740 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:21.673981  2353 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:21.678938  5782 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:21.680825  5738 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:21.686359  1559 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:21.681602  5617 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:21.691819  5103 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.746093  5527 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.754168  5527 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.755553  1707 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 6.
W20260709 13:01:21.766134  5829 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.103s	user 0.001s	sys 0.001s
I20260709 13:01:21.706032  5630 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.698161  5759 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:21.772578  5841 log.cc:927] Time spent T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:01:21.773499  5759 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.771890  2353 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:60938
I20260709 13:01:21.775677  2353 raft_consensus.cc:606] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:21.777704  2353 raft_consensus.cc:614] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:21.748909  5662 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.838s	user 0.011s	sys 0.004s
I20260709 13:01:21.779742  5548 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.779649  5662 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.651746  5806 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.783309  5662 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:21.783896  5544 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.784734  2349 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:21.634150  5817 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:21.787367  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1111) took 1402 ms. Trace:
W20260709 13:01:21.788296  5821 log.cc:927] Time spent T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.430s	user 0.000s	sys 0.001s
I20260709 13:01:21.789964  5630 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:21.790774  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:21.787837  2349 rpcz_store.cc:276] 0709 13:01:20.385054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.454792 (+ 69738us) service_pool.cc:224] Handling call
0709 13:01:20.919507 (+464715us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b5800003c98. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:01:20.919683 (+   176us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.785171 (+865488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.786974 (+  1803us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b5400686d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:21.787292 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993393664}
I20260709 13:01:21.793776  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:21.794392  1558 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:21.810628  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 393933us
W20260709 13:01:21.774539  5812 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:01:21.819186  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:21.820999  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:21.803262  2348 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1108) took 1458 ms (client timeout 1538 ms). Trace:
W20260709 13:01:21.793421  5809 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.434s	user 0.000s	sys 0.001s
I20260709 13:01:21.840996  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:21.836503  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:21.840983  2348 rpcz_store.cc:269] 0709 13:01:20.344865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.407182 (+ 62317us) service_pool.cc:224] Handling call
0709 13:01:21.803028 (+1395846us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b540069e6d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:21.803192 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1397502848}
I20260709 13:01:21.710332  5802 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:21.847522  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:21.852130  2347 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:21.847958  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 211035us
I20260709 13:01:21.845101  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:21.801823  5770 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:21.813017  5103 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:21.797840  5559 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:21.842810  5612 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:21.856990  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:21.797277  5549 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:21.858170  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:21.859453  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:21.845968  5697 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:21.862236  5780 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:21.863511  5720 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:21.863132  5697 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:21.847261  5618 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:21.860101  2349 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:21.863988  5720 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:21.701103  5840 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.855831  5548 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:21.870721  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:21.850793  5802 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:21.873734  2349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:60938
I20260709 13:01:21.874639  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:21.876322  2349 raft_consensus.cc:606] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:21.877198  2349 raft_consensus.cc:614] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:21.879653  5559 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:21.882264  5828 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:21.882851  5828 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:21.883272  5828 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:21.867601  5612 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:21.886715  5612 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.867843  5618 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.889806  5618 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.893787  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:21.902144  1558 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:21.872779  5548 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.859663  5103 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.877597  5378 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:21.916312  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:21.915371  1456 catalog_manager.cc:5697] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 6 to 7, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 7 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:01:21.917119  5719 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:21.917660  5815 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:21.872220  5740 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:01:21.909807  1527 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:21.910064  5548 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.932472  5549 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:21.933205  5549 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:21.930576  5782 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:21.935436  5782 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.935587  5707 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:21.936000  5782 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.938014  5501 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:21.938616  5501 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.939114  5501 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.935593  5549 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.911520  5505 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:01:21.946617  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2409) took 1149 ms. Trace:
I20260709 13:01:21.947117  1634 rpcz_store.cc:276] 0709 13:01:20.797345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.812955 (+ 15610us) service_pool.cc:224] Handling call
0709 13:01:21.946351 (+1133396us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b540010d1d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:21.946519 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579937536}
W20260709 13:01:21.948022  1455 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 187) took 20589 ms (client timeout 15000 ms). Trace:
I20260709 13:01:21.931376  2318 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:21.950599  5673 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:21.951126  5673 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:21.952122  5673 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:21.952792  5813 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.594s	user 0.002s	sys 0.002s
I20260709 13:01:21.954084  1456 catalog_manager.cc:5697] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 3 to 8, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:21.957532  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:21.958014  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 82179.8us
W20260709 13:01:21.958523  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.920771  5378 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:21.961917  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45672.1us
I20260709 13:01:21.962877  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:21.969441  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:21.976861  5719 raft_consensus.cc:2764] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:21.977802  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:21.978866  5817 raft_consensus.cc:2764] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:01:21.979869  5819 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.618s	user 0.000s	sys 0.003s
W20260709 13:01:21.981354  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1958) took 1721 ms (client timeout 2117 ms). Trace:
W20260709 13:01:21.981940  1556 rpcz_store.cc:269] 0709 13:01:20.260382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.260900 (+   518us) service_pool.cc:224] Handling call
0709 13:01:21.980864 (+1719964us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400389ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:21.981235 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1059332992}
I20260709 13:01:21.982295  1456 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 4 to 6, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:21.984048  1456 catalog_manager.cc:5697] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:21.928699  5852 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:21.964741  5378 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.910567  5720 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:21.985327  5609 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:21.993216  5815 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:21.994750  5852 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.039113  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:22.039800  2347 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:22.041417  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 72119us
I20260709 13:01:21.960115  5802 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.045833  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:22.047885  5851 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.070417  1456 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 9 to 13, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:22.077939  5615 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.084821  5615 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.078748  5549 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.088590  5549 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.092070  5549 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:22.042706  5720 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.031692  5840 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:22.078835  5103 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:22.094085  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 48362.9us
I20260709 13:01:22.041581  5852 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:22.109651  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44482.4us
I20260709 13:01:22.104974  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:22.007403  5559 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.108386  5817 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:22.102089  5615 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:22.122604  5576 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:22.130075  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:22.092854  5618 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.139593  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:22.140345  2350 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:22.139804  1456 catalog_manager.cc:5697] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 2 to 5, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:21.948619  1455 rpcz_store.cc:269] 0709 13:01:01.358225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.375231 (+ 17006us) service_pool.cc:224] Handling call
0709 13:01:21.947661 (+20572430us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:14.671592 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:14.672260 (+   668us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:14.672307 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:01:14.672338 (+    31us) tablet.cc:662] Decoding operations
0709 13:01:14.824080 (+151742us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:14.824172 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:14.824198 (+    26us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:01:14.976044 (+151846us) tablet.cc:701] Row locks acquired
0709 13:01:14.979127 (+  3083us) write_op.cc:260] PREPARE: finished
0709 13:01:15.000517 (+ 21390us) write_op.cc:270] Start()
0709 13:01:15.004650 (+  4133us) write_op.cc:276] Timestamp: P: 1783602074997923 usec, L: 0
0709 13:01:15.018784 (+ 14134us) op_driver.cc:348] REPLICATION: starting
0709 13:01:15.032396 (+ 13612us) log.cc:844] Serialized 5251 byte log entry
0709 13:01:15.625725 (+593329us) op_driver.cc:464] REPLICATION: finished
0709 13:01:16.030797 (+405072us) write_op.cc:301] APPLY: starting
0709 13:01:16.030909 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:16.032279 (+  1370us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:16.032459 (+   180us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:16.777019 (+744560us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:16.777362 (+   343us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=13,key_file_lookups=13,delta_file_lookups=0,mrs_lookups=26
0709 13:01:16.777434 (+    72us) write_op.cc:312] APPLY: finished
0709 13:01:16.777941 (+   507us) log.cc:844] Serialized 176 byte log entry
0709 13:01:16.778194 (+   253us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:16.778570 (+   376us) write_op.cc:454] Released schema lock
0709 13:01:16.778636 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:16.779220 (+   584us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:16.779256 (+    36us) write_op.cc:454] Released schema lock
0709 13:01:17.273562 (+494306us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b180037d260. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
Metrics: {"mutex_wait_us":40789,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":401440,"apply.run_cpu_time_us":26388,"apply.run_wall_time_us":748402,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25080,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6217,"cfile_init":1,"lbm_read_time_us":1062,"lbm_reads_lt_1ms":8,"num_ops":13,"prepare.queue_time_us":637263,"prepare.run_cpu_time_us":54159,"prepare.run_wall_time_us":474128,"raft.queue_time_us":32150,"raft.run_cpu_time_us":50952,"raft.run_wall_time_us":323876,"replication_time_us":605658,"spinlock_wait_cycles":18126080,"thread_start_us":1080028,"threads_started":4,"wal-append.queue_time_us":699891,"wal-append.run_cpu_time_us":15122,"wal-append.run_wall_time_us":2394439}]]}
I20260709 13:01:22.153127  2347 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:22.159102  1456 catalog_manager.cc:5697] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.157219  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:22.141239  5817 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:22.118657  5738 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 13:01:22.180948  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.103935  5851 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:22.181722  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.194228  5488 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.202000  1456 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:01:22.204103  5707 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:22.203596  1456 catalog_manager.cc:5697] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 7 to 8, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:22.204293  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 74498.3us
I20260709 13:01:22.205289  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:22.207082  5730 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:22.207665  5730 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:22.208180  5730 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.209679  5505 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:22.210752  5505 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:22.211192  5505 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:22.213932  5840 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.214656  5840 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:22.216665  5856 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:01:22.216914  5840 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:22.218688  5858 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:01:22.193097  5806 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:01:22.225767  5759 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.257046  5760 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.257220  5759 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.257640  5760 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.226271  5801 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:01:22.262224  5759 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:22.263213  5103 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:01:22.225410  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.232829  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:22.175068  5783 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:22.267640  5783 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.268071  5783 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.238327  1456 catalog_manager.cc:5697] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 2 to 7, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:22.268673  5815 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.238916  5840 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.232407  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 602) took 1150 ms. Trace:
I20260709 13:01:22.269933  1557 rpcz_store.cc:276] 0709 13:01:21.081670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.081847 (+   177us) service_pool.cc:224] Handling call
0709 13:01:22.232214 (+1150367us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b5400207454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:22.232339 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251251968}
I20260709 13:01:22.269222  5815 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.273648  5815 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:22.274390  5840 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.276602  5851 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:22.277252  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.280025  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.280916  1632 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:22.282868  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.283674  1632 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:22.283345  5840 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:22.280701  1456 catalog_manager.cc:5697] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 2 to 4, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.286854  5828 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.287335  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:22.288164  1635 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:22.175104  5523 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 13:01:22.289883  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.290356  5806 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:22.291507  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.292194  1631 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 5.
I20260709 13:01:22.293416  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:22.294034  1631 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:22.293201  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1906) took 2548 ms (client timeout 1983 ms). Trace:
I20260709 13:01:22.239194  5618 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:22.239283  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.298120  2318 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:22.243944  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.239967  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.241356  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.249019  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.249133  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.248621  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.298151  1705 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:22.238021  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:22.265049  5103 consensus_queue.cc:1243] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:22.300566  1710 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:22.239550  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:22.304280  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 183078us
I20260709 13:01:22.307619  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:22.308598  2347 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:22.310242  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:22.311450  1558 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:22.312049  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:22.240680  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:22.314921  1706 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.248183  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:22.295115  1707 rpcz_store.cc:269] 0709 13:01:19.744611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.956730 (+212119us) service_pool.cc:224] Handling call
0709 13:01:22.292994 (+2336264us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54001865d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:22.293103 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4059077888}
I20260709 13:01:22.330521  1557 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:22.331566  1556 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:22.266083  5817 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.243075  5549 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.334425  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:22.348806  2347 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:22.289402  5828 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:22.335556  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:22.316399  5807 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.855s	user 0.011s	sys 0.004s
I20260709 13:01:22.286234  1456 catalog_manager.cc:5697] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:22.341815  1676 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:22.353783  1558 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:22.345292  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.346082  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:22.347445  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:22.322484  5343 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.014s	sys 0.014s
I20260709 13:01:22.298353  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.349858  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:22.349874  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:22.320333  5738 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:22.352766  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:22.253798  5427 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.354765  1555 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:22.315547  5839 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:22.354803  2345 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:22.355230  1552 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:22.356993  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.344424  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:22.357781  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.358140  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.357173  2352 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:22.366590  2347 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:22.367543  2353 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:22.372890  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:22.373525  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.373696  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:22.392556  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.396420  1704 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:22.334595  5559 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.389894  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:22.404006  1556 raft_consensus.cc:2410] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:22.285459  5782 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.391906  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.423237  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.412391  1629 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.415695  1706 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:22.237604  5378 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:22.365733  5343 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.014s	sys 0.015s
I20260709 13:01:22.426743  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:22.374044  5840 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:22.444496  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 223447us
I20260709 13:01:22.368326  5839 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.424036  1703 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 6.
I20260709 13:01:22.374359  5807 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:22.369657  5423 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:22.432247  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.396487  5576 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:22.407519  5851 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:22.443732  1456 catalog_manager.cc:5697] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 6 to 8, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:22.354102  5815 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:22.395145  5738 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:22.443285  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.455549  1456 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.460412  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:22.462136  1556 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:22.462584  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.463670  1706 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:22.469961  1634 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:22.358685  5549 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.367565  5427 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.358891  5719 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:01:22.472715  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:22.473006  1456 catalog_manager.cc:5697] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 3 to 6, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 6 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:22.481936  1635 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:22.446027  5840 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.491688  1556 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:22.493420  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.494267  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:22.472229  5501 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.533s	user 0.013s	sys 0.000s
I20260709 13:01:22.454720  5839 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.458556  5492 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.410126  5559 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.465018  5711 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:22.528647  5711 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:22.529102  5711 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:22.494102  1703 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:22.410601  5828 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:01:22.457998  5851 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.499759  5720 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:22.428674  5548 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.500321  5488 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:22.507262  5492 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.507917  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.511336  5427 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:22.512319  5576 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.515621  5559 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:22.516829  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:22.526613  5782 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:22.530357  1629 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:22.539717  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.512746  5738 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.538491  1456 catalog_manager.cc:5697] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 1 to 6, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:22.492924  5840 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:22.540311  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 178915us
I20260709 13:01:22.544981  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:22.559314  1561 raft_consensus.cc:2368] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10.
W20260709 13:01:22.564013  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 63952.2us
I20260709 13:01:22.517740  5549 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:22.488090  5815 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.545966  5851 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.579187  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.579649  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:22.579900  1556 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.580396  1632 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:22.581384  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.555913  5720 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:22.587742  1456 catalog_manager.cc:5697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:22.568209  5618 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:22.504899  5501 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:22.576797  5770 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.714s	user 0.013s	sys 0.000s
I20260709 13:01:22.597920  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.597904  1630 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:22.601815  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.602948  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.555285  5783 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.632678  5783 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:22.636531  5852 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.640542  5527 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:22.642253  1632 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:22.644107  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.648258  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:22.598729  1703 raft_consensus.cc:2376] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9. Re-sending same reply.
I20260709 13:01:22.652575  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.654225  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.655194  1560 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:22.656363  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:22.657157  1560 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:22.661547  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.662307  1633 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
W20260709 13:01:22.662874  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.663440  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.664644  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.615200  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:22.604733  1456 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 8 to 9, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.668483  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.668288  1456 catalog_manager.cc:5697] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 5 to 8, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:22.669157  1702 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:22.670114  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:22.670333  1456 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 6 to 8, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.671586  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.618707  1628 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: [6.7-7.8]
I20260709 13:01:22.674412  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 685) took 1262 ms. Trace:
I20260709 13:01:22.674821  2344 rpcz_store.cc:276] 0709 13:01:21.412408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.412935 (+   527us) service_pool.cc:224] Handling call
0709 13:01:22.674208 (+1261273us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54004bf0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:22.674339 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1633841024}
I20260709 13:01:22.676868  5802 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.618924  5548 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:22.618541  5720 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.680757  5505 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.682168  5505 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.620694  5435 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:22.683609  5435 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.621407  5618 raft_consensus.cc:697] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:22.685860  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:22.686486  5618 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.687469  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.691527  5435 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:22.694805  5827 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.335s	user 0.007s	sys 0.040s
I20260709 13:01:22.649396  1555 raft_consensus.cc:2410] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:22.701875  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.702934  1555 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:22.703043  5850 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.000s	user 0.000s	sys 0.004s
I20260709 13:01:22.704233  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.709342  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.710291  1710 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:22.713951  1631 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:22.715334  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.621950  5815 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:22.625831  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.627436  5492 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:22.630033  5730 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:22.733379  1627 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:22.721102  1702 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:22.730690  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.736316  1560 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.735672  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.744797  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.602066  5770 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:22.747320  1556 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:22.749917  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:22.750733  1633 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.753304  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:22.755875  1555 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:22.748960  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:22.762429  5865 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.763234  5865 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.770143  5566 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.771438  5566 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:22.773962  2317 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:22.779043  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:22.826120  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47183.8us
I20260709 13:01:22.848501  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:22.849718  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:22.851253  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 77395.5us
I20260709 13:01:22.851493  1632 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:22.854198  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:22.859905  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:22.862779  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.868799  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:22.873651  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44430us
W20260709 13:01:22.873961  5870 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 13:01:22.874754  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.878115  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:22.881464  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:22.883265  1675 leader_election.cc:400] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10.
I20260709 13:01:22.886055  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:22.887456  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.887830  5760 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:22.895058  1561 raft_consensus.cc:2410] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:01:22.898617  5569 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:22.902199  1558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:01:22.902921  1558 raft_consensus.cc:606] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:22.903250  1558 raft_consensus.cc:614] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:22.907238  2351 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1109) took 2562 ms (client timeout 2568 ms). Trace:
W20260709 13:01:22.907830  2351 rpcz_store.cc:269] 0709 13:01:20.345183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.403463 (+ 58280us) service_pool.cc:224] Handling call
0709 13:01:22.906981 (+2503518us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b54004ed754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:22.907169 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1781176576}
W20260709 13:01:22.910231  5866 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:01:22.858777  5630 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:22.911767  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 969) took 1236 ms. Trace:
I20260709 13:01:22.912284  2346 rpcz_store.cc:276] 0709 13:01:21.675788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.744778 (+ 68990us) service_pool.cc:224] Handling call
0709 13:01:21.748561 (+  3783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.749039 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.749086 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:21.760831 (+ 11745us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.213965 (+453134us) spinlock_profiling.cc:243] Waited 8.89 ms on lock 0x7b180028ab00. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:01:22.452645 (+238680us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:22.452707 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.452827 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.911289 (+458462us) raft_consensus.cc:1685] finished
0709 13:01:22.911328 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.911685 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11572,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":3559,"spinlock_wait_cycles":20452224,"thread_start_us":238439,"threads_started":1,"wal-append.queue_time_us":848077}
I20260709 13:01:22.921592  5609 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:22.922816  5501 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:22.877851  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.923940  1629 raft_consensus.cc:2368] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8.
I20260709 13:01:22.924857  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:22.877908  5782 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:01:22.928778  1557 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:22.929271  5865 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:22.930968  5730 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:22.933190  5852 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:22.933604  5711 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:22.907356  2349 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:22.897207  5863 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.372s	user 0.002s	sys 0.000s
I20260709 13:01:22.917122  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:22.877810  5840 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:22.885131  1675 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:22.952659  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:22.986627  1632 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.986804  2353 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.988101  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2620) took 1315 ms. Trace:
I20260709 13:01:22.988830  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:22.870023  2345 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:22.925418  1629 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:22.989966  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:22.941850  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 597498us
I20260709 13:01:22.945873  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:22.991654  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45908us
I20260709 13:01:22.992530  1525 leader_election.cc:400] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8.
I20260709 13:01:22.992724  1525 leader_election.cc:403] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:23.006676  1633 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:23.009034  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:23.009632  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:23.010396  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:22.988734  1636 rpcz_store.cc:276] 0709 13:01:21.672448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.674450 (+  2002us) service_pool.cc:224] Handling call
0709 13:01:21.674844 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.813062 (+138218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.813212 (+   150us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:21.847188 (+ 33976us) log.cc:844] Serialized 26 byte log entry
0709 13:01:21.962868 (+115680us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:21.962946 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.963052 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.920420 (+957368us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b1800396a00. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:22.934833 (+ 14413us) raft_consensus.cc:1685] finished
0709 13:01:22.934856 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.987999 (+ 53143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":132731,"raft.queue_time_us":138093,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":22734,"spinlock_wait_cycles":27848960,"thread_start_us":61732,"threads_started":1,"wal-append.queue_time_us":83724}
I20260709 13:01:23.010691  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:23.011235  2351 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:01:23.011994  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:23.012568  1636 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.012689  5802 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:23.013249  5802 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.013635  5802 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.014048  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.015261  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.015815  1627 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:23.017102  1627 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [13.9-14.10]
I20260709 13:01:23.024164  5609 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:23.025512  5609 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.031220  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:23.032274  1557 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:23.034576  5523 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:23.035216  5523 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.033668  1636 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:23.038872  5711 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:23.040340  1556 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:01:23.040717  1556 pending_rounds.cc:85] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 3
I20260709 13:01:23.040931  1556 pending_rounds.cc:107] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:01:23.041170  1556 raft_consensus.cc:2889] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:23.041666  1631 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:23.041671  5780 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:23.045755  5780 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:23.046602  5780 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:23.047962  5780 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:23.048161  1636 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:23.010437  1555 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:23.047775  5851 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:23.055014  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.056566  1556 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:23.011287  1558 raft_consensus.cc:2410] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:23.059568  1557 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:01:23.060006  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 68638.3us
I20260709 13:01:22.934535  5423 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:23.066552  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 181502us
I20260709 13:01:22.861547  5378 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:23.089179  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:23.071774  1560 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:23.075690  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.076567  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:23.083570  1703 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:23.083807  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:22.938726  5817 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.087793  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.091897  1706 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:22.873232  5343 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:23.097687  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 105257us
I20260709 13:01:23.093873  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.116492  1628 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 8 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:01:23.116609  2351 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:23.091873  1602 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:22.987085  5559 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:23.096690  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 158063us
I20260709 13:01:23.101549  1675 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:23.122113  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.124235  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:23.125432  1702 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:23.128746  5494 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:23.129662  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.064267  5775 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.130877  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.134207  1560 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:01:23.136682  1560 pending_rounds.cc:85] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 5
I20260709 13:01:23.137044  1560 pending_rounds.cc:107] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:01:23.137394  1560 raft_consensus.cc:2889] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:23.088224  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:23.105538  5427 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.105341  5711 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:23.143483  5817 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:23.150020  5709 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:23.143605  5378 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:23.152850  5378 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:23.153270  5378 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:23.154452  1710 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:23.106093  5782 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:23.098899  5548 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8.
I20260709 13:01:23.093156  5423 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.167097  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.107614  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:23.109212  1561 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:23.110220  1555 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:23.098834  5869 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:01:22.994685  5806 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.115665  1701 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:23.051714  5873 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:23.118557  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.119246  1558 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:01:23.182166  1558 pending_rounds.cc:85] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 4
I20260709 13:01:23.182463  1558 pending_rounds.cc:107] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:01:23.143520  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.156579  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:23.183169  1705 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:23.186028  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.093689  5783 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:23.132187  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:23.160977  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.161607  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.199962  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.176870  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.119541  2317 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:23.201253  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 81797.7us
I20260709 13:01:23.157632  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:23.182046  1709 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:23.203323  1628 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 5.
I20260709 13:01:23.204613  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.205847  1628 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:23.207839  1676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.030s	sys 0.072s
W20260709 13:01:23.209220  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 51682us
W20260709 13:01:23.209611  1676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.031s	sys 0.072s
W20260709 13:01:23.211788  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.187371  1558 raft_consensus.cc:2889] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:23.179694  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:23.159833  5847 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:23.177875  5423 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:23.231438  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:23.232203  1635 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:23.179224  5505 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.224713  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:23.186995  1708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.018s	sys 0.006s
W20260709 13:01:23.237986  1708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.018s	sys 0.006s
W20260709 13:01:23.191165  5738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.054s	sys 0.132s
I20260709 13:01:23.229115  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.180111  5711 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:23.211825  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:01:23.175694  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 87526.5us
I20260709 13:01:23.115337  5343 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.251531  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:23.246842  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.253104  1628 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:23.214145  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:23.243108  1705 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:23.154640  5775 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:23.218994  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:23.255807  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 44083.9us
I20260709 13:01:23.256489  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:23.259042  1709 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:01:23.220345  5549 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.032s	sys 0.072s
W20260709 13:01:23.211251  5720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.007s	sys 0.016s
I20260709 13:01:23.262432  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:23.267647  1553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 604) took 2170 ms (client timeout 1928 ms). Trace:
I20260709 13:01:23.270268  5759 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:23.275637  5876 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.249579  5711 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.267680  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.293347  5711 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:23.293901  1631 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:23.260830  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:23.297192  5630 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:23.297857  5630 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.299166  5851 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:23.191430  5806 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.301647  5806 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:23.278214  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:23.306056  5759 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:23.306643  5759 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:23.280131  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:23.309227  2349 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:23.281498  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.309504  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 694) took 1053 ms. Trace:
I20260709 13:01:23.221053  5847 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.230875  5559 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:23.292659  5707 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:23.276001  5488 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:23.249909  5103 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:23.248528  5738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.055s	sys 0.132s
W20260709 13:01:23.291113  1553 rpcz_store.cc:269] 0709 13:01:21.097462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.097675 (+   213us) service_pool.cc:224] Handling call
0709 13:01:22.614995 (+1517320us) spinlock_profiling.cc:243] Waited 8.23 ms on lock 0x7b5800002798. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:01:23.267369 (+652374us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b5400389ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:23.267541 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":992506112}
I20260709 13:01:23.309921  1703 rpcz_store.cc:276] 0709 13:01:22.258648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.652212 (+393564us) service_pool.cc:224] Handling call
0709 13:01:23.309254 (+657042us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b54000cbcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:23.309409 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":989106432}
I20260709 13:01:23.311103  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 701) took 1050 ms. Trace:
W20260709 13:01:23.313553  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51225.4us
W20260709 13:01:23.288273  5720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.008s	sys 0.016s
I20260709 13:01:23.271832  1525 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:23.322520  1707 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:01:23.322754  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 44630.8us
I20260709 13:01:23.229808  5760 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:23.321144  1632 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:23.198805  5709 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 13:01:23.383899  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.384537  1702 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:23.386976  1561 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:01:23.250795  5510 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:23.389572  1632 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: [6.7-9.8]
I20260709 13:01:23.358752  1556 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:01:23.463464  1556 pending_rounds.cc:85] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 9
I20260709 13:01:23.463893  1556 pending_rounds.cc:107] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:01:23.464231  1556 raft_consensus.cc:2889] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:23.354406  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:23.361771  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.366677  2351 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:01:23.368274  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 149396us
I20260709 13:01:23.466075  1633 raft_consensus.cc:2410] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:23.281540  5873 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:23.495772  1552 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:23.254082  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.370599  1701 rpcz_store.cc:276] 0709 13:01:22.260544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.671342 (+410798us) service_pool.cc:224] Handling call
0709 13:01:23.310922 (+639580us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b540036ddd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:23.311048 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1019371392}
I20260709 13:01:23.371225  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:23.372149  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:23.373001  1630 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:01:23.497834  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 126712us
I20260709 13:01:23.377882  1629 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:23.379462  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:23.498721  1558 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:23.250543  5830 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 13:01:23.499440  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:23.280695  5612 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:23.382287  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:23.249183  5840 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:23.348469  5847 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.346961  5852 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:23.259678  5505 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.397506  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:23.363854  5527 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:23.387367  1561 pending_rounds.cc:85] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 7
I20260709 13:01:23.531507  1561 pending_rounds.cc:107] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:01:23.531823  1561 raft_consensus.cc:2889] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:23.279784  5875 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.130s
I20260709 13:01:23.307644  5759 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.364519  5566 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.363622  5103 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:23.372810  5630 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:23.368925  5488 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:23.381026  5851 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:23.418833  1635 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:23.385637  5760 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.421154  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.550967  1627 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:23.412477  5872 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.885s	user 0.006s	sys 0.000s
W20260709 13:01:23.541581  5576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.007s	sys 0.015s
I20260709 13:01:23.424754  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.415256  5492 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:23.592519  1456 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.010s	sys 0.016s
W20260709 13:01:23.593004  1456 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.010s	sys 0.016s
W20260709 13:01:23.423426  5839 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.882s	user 0.003s	sys 0.008s
I20260709 13:01:23.399029  5802 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:23.452395  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:23.599017  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:23.599429  1703 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:23.461282  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:23.476176  5885 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:01:23.482115  5877 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:01:23.483424  5343 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:23.490355  5775 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:23.287882  5549 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.032s	sys 0.072s
I20260709 13:01:23.501072  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2527) took 1055 ms. Trace:
I20260709 13:01:23.603111  5759 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:23.521286  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:23.521596  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 139429us
I20260709 13:01:23.524567  1705 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:23.491737  5873 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.518739  5615 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:23.517431  5612 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:23.524266  5852 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.519306  5882 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:23.552500  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:23.531167  5527 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.532557  5709 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:23.565623  2318 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.025s	sys 0.054s
I20260709 13:01:23.395144  5510 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.569144  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:23.575563  1558 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
W20260709 13:01:23.663561  2318 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.025s	sys 0.054s
W20260709 13:01:23.663976  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 941911us
I20260709 13:01:23.586957  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:23.589784  2353 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.602s	user 0.007s	sys 0.006s
I20260709 13:01:23.548213  5103 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:23.540043  5738 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:23.563719  5730 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.006s	sys 0.020s
W20260709 13:01:23.711550  5730 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.006s	sys 0.021s
I20260709 13:01:23.603955  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:23.638203  5775 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:23.583362  5576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.008s	sys 0.016s
I20260709 13:01:23.600212  1707 raft_consensus.cc:2376] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8. Re-sending same reply.
I20260709 13:01:23.546998  5566 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:23.600772  5609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.003s	sys 0.010s
I20260709 13:01:23.617667  1552 rpcz_store.cc:276] 0709 13:01:22.446126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.098624 (+652498us) service_pool.cc:224] Handling call
0709 13:01:23.500989 (+402365us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.662422  5720 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.640743  1710 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:23.512497  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.671856  5709 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:23.585774  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:23.663980  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 95066.9us
I20260709 13:01:23.665371  2353 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:23.672788  5630 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.677343  5510 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:23.586331  1636 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.034s	sys 0.069s
I20260709 13:01:23.671063  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.671463  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:23.602977  5492 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.636065  5873 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.631224  5879 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:23.664856  1709 raft_consensus.cc:2376] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10. Re-sending same reply.
I20260709 13:01:23.679003  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:23.679730  2351 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.685616  5738 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:23.686575  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1965) took 1368 ms. Trace:
I20260709 13:01:23.759321  1632 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:23.758104  1559 rpcz_store.cc:276] 0709 13:01:22.318544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.318706 (+   162us) service_pool.cc:224] Handling call
0709 13:01:22.319132 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.335122 (+ 15990us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:22.392906 (+ 57784us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.470944 (+ 78038us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:22.503528 (+ 32584us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.503709 (+   181us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:22.503746 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.503823 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.909743 (+405920us) spinlock_profiling.cc:243] Waited 7.29 ms on lock 0x7b4800002958. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:01:23.155725 (+245982us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:23.686192 (+530467us) raft_consensus.cc:1685] finished
0709 13:01:23.686223 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.686456 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89677,"prepare.queue_time_us":44659,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":11200,"raft.run_cpu_time_us":2924,"raft.run_wall_time_us":442536,"spinlock_wait_cycles":25111680,"thread_start_us":66467,"threads_started":1,"wal-append.queue_time_us":106588}
I20260709 13:01:23.760157  5612 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.761005  1559 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:23.761361  5612 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:23.686784  5884 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:01:23.689246  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:23.690239  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:23.694221  5817 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:23.696111  5523 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:23.698989  5806 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:23.765058  5806 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:23.766055  5806 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:23.766636  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.767107  5806 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.768738  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:23.706858  5615 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.769694  5615 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:23.709203  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 695) took 1450 ms (client timeout 1797 ms). Trace:
I20260709 13:01:23.734632  5720 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:23.773672  1704 rpcz_store.cc:269] 0709 13:01:22.259014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.431838 (+172824us) service_pool.cc:224] Handling call
0709 13:01:23.708922 (+1277084us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54002e95d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:23.709124 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2161921408}
I20260709 13:01:23.774143  5802 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:23.774931  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:23.775523  5720 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:23.776312  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.776329  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.710850  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:23.778003  1556 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:23.777912  5505 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:23.552107  5760 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:23.780272  5830 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:01:23.780877  5881 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.347s	user 0.001s	sys 0.001s
I20260709 13:01:23.782363  5830 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:23.783010  5830 consensus_queue.cc:1243] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:23.778012  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.755039  1630 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:23.718231  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:23.657923  5852 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:23.623556  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.796814  1555 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:01:23.795415  1703 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:23.812399  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.815053  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:23.723539  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 339899us
I20260709 13:01:23.674161  5527 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.606994  5839 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.728495  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.666114  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:23.730980  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:23.733321  1636 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.034s	sys 0.069s
I20260709 13:01:23.735813  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1969) took 1398 ms. Trace:
W20260709 13:01:23.720759  5866 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:01:23.719887  5609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.004s	sys 0.010s
I20260709 13:01:23.747618  5782 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:23.720239  5882 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:23.728600  5840 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:23.754978  5566 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:23.731593  5709 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.741714  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.756045  1629 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:23.724560  5775 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.742139  5630 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.661871  5488 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.757622  1561 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:23.829931  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.830773  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:23.966535  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:23.831058  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:23.967170  1558 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:23.873848  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.879069  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:23.816735  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:23.883461  2351 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
W20260709 13:01:23.894042  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 622242us
I20260709 13:01:23.852538  5548 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:23.928720  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:23.831939  5378 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.004s
W20260709 13:01:23.860168  5423 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.584s	user 0.003s	sys 0.008s
I20260709 13:01:23.769212  5711 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:23.879753  5883 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.443s	user 0.004s	sys 0.001s
I20260709 13:01:23.874753  5492 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:23.959615  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:23.960875  1630 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:01:23.857692  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:23.967262  1707 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:23.969092  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:23.992116  1555 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:23.931201  1554 rpcz_store.cc:276] 0709 13:01:22.333404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.418921 (+ 85517us) service_pool.cc:224] Handling call
0709 13:01:22.487192 (+ 68271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.498471 (+ 11279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.498523 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:22.509792 (+ 11269us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.832537 (+322745us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:22.832583 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.832662 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.731913 (+899251us) raft_consensus.cc:1685] finished
0709 13:01:23.731948 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.732134 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1730,"raft.queue_time_us":11334,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":9638,"thread_start_us":322232,"threads_started":1,"wal-append.queue_time_us":649831}
I20260709 13:01:23.970757  1705 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:23.898545  5559 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:23.972368  1704 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:23.972249  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:23.994148  1558 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:23.899356  5876 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:23.968492  2351 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 7
I20260709 13:01:23.980048  2346 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:23.980918  1553 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:01:23.831210  5576 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.138s
I20260709 13:01:23.861001  5673 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.108s
W20260709 13:01:23.986858  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 362400us
I20260709 13:01:23.995287  1703 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:23.994292  2352 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:01:24.006338  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:23.942436  5523 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:01:24.013695  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:24.013712  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:23.940191  5501 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:01:24.018517  1702 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:23.931147  5847 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:24.040367  1703 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:24.040942  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:24.042173  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 182955us
I20260709 13:01:24.043200  5817 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:24.044406  5864 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:24.022961  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 972) took 1235 ms. Trace:
I20260709 13:01:23.974370  5840 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:24.047523  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:24.048029  5840 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.048605  5612 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:24.046749  5494 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:24.049171  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:24.073000  2318 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:24.089598  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:24.094286  1628 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:24.100425  2318 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:24.079478  2344 rpcz_store.cc:276] 0709 13:01:22.787150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.856067 (+ 68917us) service_pool.cc:224] Handling call
0709 13:01:22.880178 (+ 24111us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:22.988018 (+107840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.988053 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:23.008869 (+ 20816us) log.cc:844] Serialized 52 byte log entry
0709 13:01:23.518468 (+509599us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:24.011509 (+493041us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.022399 (+ 10890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.022530 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.022600 (+    70us) raft_consensus.cc:1685] finished
0709 13:01:24.022620 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.022858 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69106,"raft.queue_time_us":137276,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":116,"thread_start_us":963207,"threads_started":2,"wal-append.queue_time_us":684708}
I20260709 13:01:24.008440  2351 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.8
I20260709 13:01:24.104614  2351 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Aborting config change with OpId 7.8: Aborted: Op aborted by new leader
I20260709 13:01:24.105288  2351 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:24.012367  5759 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:24.121560  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.122105  2351 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 10.9
W20260709 13:01:24.126060  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 197442us
I20260709 13:01:24.128306  2351 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:24.136130  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 122678us
I20260709 13:01:24.137420  1706 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:24.144076  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.033267  5492 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:23.961309  5882 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:23.981205  5378 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.143857  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:24.150156  1628 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:23.979312  5548 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.025771  5851 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:24.105168  5609 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:24.153391  5888 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:01:24.094574  5879 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:01:24.017757  5876 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:24.149446  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 253997us
W20260709 13:01:24.123093  5855 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:01:24.120594  5505 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:01:24.162474  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.165827  2344 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac931d9355284a168be64d7debd009ae"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
mode: GRACEFUL
new_leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
 from {username='slave'} at 127.0.0.1:60938
I20260709 13:01:24.167758  1628 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:24.171001  2351 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:24.148985  5903 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:24.138091  5897 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.174753  1632 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:24.152079  1627 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:23.988730  5423 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.199981  1635 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:24.373019  5926 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:24.375075  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2625) took 1275 ms. Trace:
I20260709 13:01:24.375514  1635 rpcz_store.cc:276] 0709 13:01:23.100285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.732401 (+632116us) service_pool.cc:224] Handling call
0709 13:01:23.732736 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.373747 (+641011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.374787 (+  1040us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b540017c854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:24.374998 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074009472}
I20260709 13:01:24.201725  2344 raft_consensus.cc:606] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Received request to transfer leadership to TS a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:24.376714  5806 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.609s	user 0.009s	sys 0.000s
I20260709 13:01:24.377059  2344 raft_consensus.cc:614] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:24.094076  5612 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:24.153381  5759 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:24.119828  5494 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:24.211544  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 106859us
I20260709 13:01:24.232717  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 706) took 1969 ms. Trace:
I20260709 13:01:24.199944  5879 consensus_queue.cc:1243] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:24.215900  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2533) took 1343 ms. Trace:
W20260709 13:01:24.282706  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.413511  1702 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:24.413699  1706 rpcz_store.cc:276] 0709 13:01:22.871940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.097937 (+225997us) service_pool.cc:224] Handling call
0709 13:01:24.215550 (+1117613us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b540014c8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:24.215793 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162258816}
I20260709 13:01:24.415863  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:24.216599  1704 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:24.233421  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 90366.6us
I20260709 13:01:24.192020  5909 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.210224  5903 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.200819  5912 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.116s
I20260709 13:01:24.321095  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2449) took 1374 ms. Trace:
I20260709 13:01:24.289765  5919 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:24.344949  5902 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
W20260709 13:01:24.345711  5895 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:01:24.312767  5559 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.350463  5738 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:24.355095  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:24.339501  5505 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:24.348230  5510 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.598s	user 0.005s	sys 0.007s
I20260709 13:01:24.233283  1553 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:24.378737  2347 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:24.169271  5566 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:24.412436  1708 rpcz_store.cc:276] 0709 13:01:22.288532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.678504 (+389972us) service_pool.cc:224] Handling call
0709 13:01:22.678914 (+   410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:22.679361 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.679405 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:22.680047 (+   642us) log.cc:844] Serialized 52 byte log entry
0709 13:01:23.239258 (+559211us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:23.601183 (+361925us) log.cc:844] Serialized 14 byte log entry
0709 13:01:23.753066 (+151883us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:23.978956 (+225890us) log.cc:844] Serialized 14 byte log entry
0709 13:01:23.979164 (+   208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.979247 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.232382 (+253135us) raft_consensus.cc:1685] finished
0709 13:01:24.232421 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.232636 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32894,"prepare.queue_time_us":509340,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":4330,"raft.queue_time_us":84956,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":55286,"spinlock_wait_cycles":3052416,"thread_start_us":1268493,"threads_started":3,"wal-append.queue_time_us":756384}
W20260709 13:01:24.433213  1557 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.008s	sys 0.014s
I20260709 13:01:24.360560  5928 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:24.360436  5630 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:24.429437  1631 rpcz_store.cc:276] 0709 13:01:22.946451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.294903 (+348452us) service_pool.cc:224] Handling call
0709 13:01:23.295558 (+   655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.295845 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:23.625972 (+330127us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.320695 (+694723us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.320766 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:24.320819 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.320891 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.321028 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":440069,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":150154,"raft.run_wall_time_us":23191,"spinlock_wait_cycles":7859584,"thread_start_us":1023954,"threads_started":3,"wal-append.queue_time_us":546345}
W20260709 13:01:24.451210  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.438308  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:24.462316  1557 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.009s	sys 0.015s
I20260709 13:01:24.448514  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2448) took 1679 ms. Trace:
I20260709 13:01:24.448961  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:24.421499  5864 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.193507  5103 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:24.465939  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 365584us
I20260709 13:01:24.461580  5898 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.478330  5864 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:24.478981  5898 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:24.481563  5103 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:24.482115  1702 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:24.433939  5919 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:24.484901  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 129913us
I20260709 13:01:24.503113  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:24.430179  5883 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:01:24.347556  5851 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:24.429391  5903 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:24.430548  5916 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.073s	user 0.002s	sys 0.000s
I20260709 13:01:24.481756  5423 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:24.510969  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:24.524159  1553 raft_consensus.cc:2393] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:24.511260  1707 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.005s
I20260709 13:01:24.525384  1707 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:24.526589  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:24.487740  5760 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.705s	user 0.012s	sys 0.000s
I20260709 13:01:24.502624  5919 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.518451  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:24.481388  5898 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:24.564651  2353 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:24.430392  5343 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:24.539979  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.514796  1702 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:24.508940  5378 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:24.511036  5930 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:01:24.557628  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 765) took 1605 ms. Trace:
I20260709 13:01:24.473765  5510 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.582286  1708 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:24.571610  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:24.528878  5909 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:24.574697  1702 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:24.549834  5932 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.560365  5882 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:24.592981  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2555) took 1119 ms. Trace:
I20260709 13:01:24.602044  1559 rpcz_store.cc:276] 0709 13:01:23.473352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.761869 (+288517us) service_pool.cc:224] Handling call
0709 13:01:23.762207 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.762972 (+   765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.763007 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.763546 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:01:23.971771 (+208225us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:24.151737 (+179966us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.153708 (+  1971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.153813 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.355142 (+201329us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b1800457ae0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:24.582399 (+227257us) raft_consensus.cc:1685] finished
0709 13:01:24.592755 (+ 10356us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.592911 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17400,"prepare.queue_time_us":415462,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":1830,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":36796,"spinlock_wait_cycles":11550464,"thread_start_us":370900,"threads_started":2,"wal-append.queue_time_us":432991}
I20260709 13:01:24.546425  5548 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:24.501052  5928 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.607574  5928 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:24.552397  5711 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:24.570240  5343 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:24.577751  1634 rpcz_store.cc:276] 0709 13:01:22.952577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.312021 (+359444us) service_pool.cc:224] Handling call
0709 13:01:23.368601 (+ 56580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.369417 (+   816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.369448 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.370137 (+   689us) log.cc:844] Serialized 26 byte log entry
0709 13:01:23.654144 (+284007us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:23.654208 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.654311 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.557296 (+902985us) raft_consensus.cc:1685] finished
0709 13:01:24.557328 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.557530 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2890,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":6619,"thread_start_us":252316,"threads_started":1,"wal-append.queue_time_us":559103}
I20260709 13:01:24.549434  5760 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:24.609170  5548 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.613597  2350 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 8: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:24.614382  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2546) took 1974 ms. Trace:
I20260709 13:01:24.583572  5423 raft_consensus.cc:2764] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:24.611369  5775 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:24.614998  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:24.616561  1557 rpcz_store.cc:276] 0709 13:01:22.639646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.061357 (+421711us) service_pool.cc:224] Handling call
0709 13:01:23.104870 (+ 43513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.128364 (+ 23494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.134848 (+  6484us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.156879 (+ 22031us) log.cc:844] Serialized 26 byte log entry
0709 13:01:23.654592 (+497713us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:24.482629 (+828037us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.486913 (+  4284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.487002 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.614154 (+127152us) raft_consensus.cc:1685] finished
0709 13:01:24.614186 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.614318 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3996,"prepare.queue_time_us":686699,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":18550,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":9102,"thread_start_us":1170029,"threads_started":2,"wal-append.queue_time_us":730817}
I20260709 13:01:24.623680  1560 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:24.621296  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 49746.8us
I20260709 13:01:24.625399  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:24.630795  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:24.632359  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 114000us
W20260709 13:01:24.631183  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.609839  5378 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.635237  5378 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.623477  5922 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:24.639128  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.649252  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:24.653030  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:24.623163  5940 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.578002  5759 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:24.657739  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:24.659832  1557 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:24.662325  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:24.663081  1632 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:24.587488  5505 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:24.675976  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:24.677882  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:24.678561  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:24.678845  1631 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:01:24.610519  5932 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:24.679168  1632 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:24.621634  5548 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.639272  5566 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:24.689899  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:24.691408  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:24.691826  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:24.692256  1632 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:24.692525  1631 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:24.695809  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:24.698197  5864 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:24.698700  5103 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:24.705549  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:24.700709  5839 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:24.650843  5935 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:24.658473  5423 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:24.702347  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:24.735077  1632 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:01:24.710088  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 79388.7us
I20260709 13:01:24.735569  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:24.736452  1631 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:01:24.737737  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:24.738406  1630 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.668444  5939 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:24.726613  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:24.743741  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:24.744928  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:24.739327  5423 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:24.750543  5423 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.754650  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2551) took 1112 ms. Trace:
I20260709 13:01:24.755147  1710 rpcz_store.cc:276] 0709 13:01:23.642128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.957236 (+315108us) service_pool.cc:224] Handling call
0709 13:01:23.975044 (+ 17808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.192492 (+217448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.192534 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.192599 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:24.192659 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.754323 (+561664us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b540016c954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:24.754379 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.754545 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499385600}
I20260709 13:01:24.756814  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:24.719624  5922 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.719147  5630 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:01:24.758231  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 770) took 1443 ms. Trace:
I20260709 13:01:24.758725  1629 rpcz_store.cc:276] 0709 13:01:23.314823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.957882 (+643059us) service_pool.cc:224] Handling call
0709 13:01:23.974626 (+ 16744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.303148 (+328522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.303200 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.303264 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:24.303321 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.757836 (+454515us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54001b6f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:24.757895 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.758055 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754829184}
I20260709 13:01:24.759732  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:24.760636  1629 pending_rounds.cc:85] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 7
W20260709 13:01:24.760960  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55769.6us
I20260709 13:01:24.761934  5612 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:24.762686  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:24.763154  5494 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:24.764442  2346 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1141) took 1813 ms (client timeout 1582 ms). Trace:
W20260709 13:01:24.764858  2346 rpcz_store.cc:269] 0709 13:01:22.951266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.951390 (+   124us) service_pool.cc:224] Handling call
0709 13:01:24.764174 (+1812784us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b54006185d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:24.764331 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2258847616}
I20260709 13:01:24.765321  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1975) took 1413 ms. Trace:
I20260709 13:01:24.765749  1560 rpcz_store.cc:276] 0709 13:01:23.351657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.354011 (+  2354us) service_pool.cc:224] Handling call
0709 13:01:24.765111 (+1411100us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400320ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:24.765263 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2663807360}
I20260709 13:01:24.766937  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:24.767016  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:24.767560  1561 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:24.768281  5612 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:24.769170  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:24.770355  5494 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.770637  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1917) took 1124 ms. Trace:
I20260709 13:01:24.482365  1636 rpcz_store.cc:276] 0709 13:01:22.769319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.049383 (+280064us) service_pool.cc:224] Handling call
0709 13:01:23.049804 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.050087 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:23.930801 (+880714us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.445559 (+514758us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.445648 (+    89us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:24.445725 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.445832 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.448412 (+  2580us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":685669,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":244178,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":11359,"thread_start_us":1394525,"threads_started":3}
I20260709 13:01:24.771075  1704 rpcz_store.cc:276] 0709 13:01:23.645823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.007655 (+361832us) service_pool.cc:224] Handling call
0709 13:01:24.216300 (+208645us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.418250 (+201950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.770425 (+352175us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b540014c8d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:24.770562 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478758656}
I20260709 13:01:24.771843  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:24.772461  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:24.773672  5492 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:24.774828  5852 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.921s	user 0.010s	sys 0.005s
I20260709 13:01:24.777097  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:24.778880  5494 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:24.781942  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:24.782029  5343 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.783257  5711 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:24.783957  5711 consensus_queue.cc:1243] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:24.784426  5869 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:24.785696  5923 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:24.785930  5927 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:24.786171  5923 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:24.786446  5882 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:24.787473  5927 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:24.787878  5927 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:24.787249  5923 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.788303  5882 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:24.788810  5923 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:24.785537  5730 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:24.785259  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:24.792582  1553 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:24.780766  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:24.795470  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:24.792660  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2558) took 1295 ms. Trace:
I20260709 13:01:24.796139  1632 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:24.761075  1629 pending_rounds.cc:107] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:01:24.798297  1629 raft_consensus.cc:2889] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:24.767045  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:24.800002   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:24.800514  5929 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:01:24.800928  5566 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.802054  5730 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:24.802380   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:24.802573  5730 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:24.803259  5730 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:24.788741  5927 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.805678  5939 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:24.805819  5927 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:24.806221  5939 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:24.806368  5738 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:24.807245  5738 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:24.807631  5738 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:24.777984  5852 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.808558  5738 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:24.810596  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:24.807257  5939 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:24.811690  5939 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.811937  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:24.805698  5864 consensus_queue.cc:1059] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:01:24.814015  5864 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:24.814410  5864 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:24.815467  5864 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:24.818154  5869 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.818537  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:24.819213  1708 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:24.820197  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:24.820315  2317 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:24.821599  5839 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:24.823087  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.792634  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:24.824867  1704 raft_consensus.cc:2368] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 6. Current term is 7.
W20260709 13:01:24.825582  5921 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:01:24.783281  5343 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.796029  1561 rpcz_store.cc:276] 0709 13:01:23.474111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.975694 (+501583us) service_pool.cc:224] Handling call
0709 13:01:24.767280 (+791586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.768022 (+   742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.769256 (+  1234us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b5400320ad0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:24.769413 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1735412992}
W20260709 13:01:24.830260  5911 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 13:01:24.832273  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2623) took 1747 ms. Trace:
I20260709 13:01:24.685105  5937 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.832783  1633 rpcz_store.cc:276] 0709 13:01:23.084912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.522581 (+437669us) service_pool.cc:224] Handling call
0709 13:01:23.538282 (+ 15701us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.538803 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.538850 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.583123 (+ 44273us) log.cc:844] Serialized 26 byte log entry
0709 13:01:23.988452 (+405329us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:23.988493 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.988570 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.831970 (+843400us) raft_consensus.cc:1685] finished
0709 13:01:24.832002 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.832198 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":400,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7497,"thread_start_us":372020,"threads_started":1,"wal-append.queue_time_us":877147}
I20260709 13:01:24.819036  5869 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:24.813184  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:24.853653  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61373.8us
I20260709 13:01:24.854784  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:24.857963  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:24.859329  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:24.861883  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:24.862120  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 76936.3us
I20260709 13:01:24.862908  2345 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:24.863430  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:24.862088  1630 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:24.674410  5940 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:24.878278  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 97546us
I20260709 13:01:24.878613  2346 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:24.878796  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:24.772472  5609 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.001s
I20260709 13:01:24.890188  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:24.890728  5494 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:24.893551  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:24.893966  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:24.740301  5378 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:24.902258  2349 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:24.905570  2345 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:24.784493  5711 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:24.909421  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:24.914947  2351 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 6.
I20260709 13:01:24.911938  2352 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:24.915753  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:24.916906  1708 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:24.924460  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:24.857600  5864 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.927318  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:24.941375  1630 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:24.944485  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47591.5us
I20260709 13:01:24.865119  5775 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:24.913515  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:24.893988  5909 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.959823  1635 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:24.949661  1631 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 7.
I20260709 13:01:24.892573  5707 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:01:24.947683  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:24.944417  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 63303.8us
I20260709 13:01:24.909816  5928 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:24.923200  5922 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:24.979100  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:24.948465  5898 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:24.907384  5609 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:24.928732  5615 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.158s	user 0.004s	sys 0.010s
I20260709 13:01:24.968657  1602 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:24.875982  5907 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.949209  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:24.875814  5630 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:01:24.998556  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:25.002251  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.956749  5711 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:24.950585  5738 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:25.017498  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43576.5us
W20260709 13:01:25.017942  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 49402.8us
I20260709 13:01:25.022501  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:24.991935  5898 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.027309  5900 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:25.028800  5922 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:25.029129  5951 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:01:25.021914  5928 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:25.029969  5922 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.030483  5928 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.032989  5945 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:25.027540  5898 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:25.033852  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 86255.4us
I20260709 13:01:25.035089  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:25.036091  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:25.037703  5935 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:25.038259  5935 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:25.038695  5935 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:25.039692  5423 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.041826  5501 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:25.042543  5940 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:25.042675  5501 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:25.043934  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:25.044690  2349 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
W20260709 13:01:25.045539  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.046885  5423 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:25.047467  5936 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
W20260709 13:01:25.050278  5941 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:01:25.051817  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 773) took 1250 ms. Trace:
I20260709 13:01:25.052234  1628 rpcz_store.cc:276] 0709 13:01:23.801838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.206706 (+404868us) service_pool.cc:224] Handling call
0709 13:01:24.207106 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.207522 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.207555 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:24.301939 (+ 94384us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.427691 (+125752us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:24.427763 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.427853 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.051533 (+623680us) raft_consensus.cc:1685] finished
0709 13:01:25.051560 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.051745 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":224389,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":6179,"thread_start_us":84516,"threads_started":1,"wal-append.queue_time_us":372346}
I20260709 13:01:24.983069  5760 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:25.058548  5760 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:25.059161  5760 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.059435  5898 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.061059  1553 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:01:25.058331  1708 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:25.065382  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 708) took 1598 ms. Trace:
W20260709 13:01:25.068105  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.996706  5847 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:25.078347  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.065855  1701 rpcz_store.cc:276] 0709 13:01:23.467064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.518648 (+ 51584us) service_pool.cc:224] Handling call
0709 13:01:23.531374 (+ 12726us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:23.987579 (+456205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.987611 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:23.991211 (+  3600us) log.cc:844] Serialized 52 byte log entry
0709 13:01:24.582805 (+591594us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:24.614609 (+ 31804us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.622056 (+  7447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.622142 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.065105 (+442963us) raft_consensus.cc:1685] finished
0709 13:01:25.065131 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.065307 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17460,"prepare.queue_time_us":40488,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":86181,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":19553,"spinlock_wait_cycles":6016,"thread_start_us":862128,"threads_started":2,"wal-append.queue_time_us":544271}
I20260709 13:01:24.994514  5510 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:25.063465  5840 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.032s	sys 0.049s
I20260709 13:01:25.024451  5947 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:25.113276  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 254027us
I20260709 13:01:25.108633  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:25.116231  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:25.118985  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:25.120769  5492 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:25.122462  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.066403  5932 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:25.097549  5840 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.033s	sys 0.049s
I20260709 13:01:25.016024  5615 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.143968  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:25.087683  5937 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.088251  5940 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.094259  5423 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
W20260709 13:01:25.170488  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.014866  5907 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:25.139494  5873 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.364s	user 0.010s	sys 0.008s
I20260709 13:01:25.133069  5934 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:25.106801  5707 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:25.134245  5760 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:25.191223  1556 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:25.070312  5908 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:01:25.138159  5505 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.196625  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:25.199714  5709 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.217s	user 0.011s	sys 0.007s
W20260709 13:01:25.202005  5919 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.004s
W20260709 13:01:25.208542  5903 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.669s	user 0.013s	sys 0.002s
W20260709 13:01:25.195751  5527 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.255s	user 0.020s	sys 0.002s
I20260709 13:01:25.135393  5947 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.190681  5548 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:25.228150  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 213316us
I20260709 13:01:25.228870  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:25.199220  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 344516us
I20260709 13:01:25.229728  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:25.226239  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.230346  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:25.210501  5510 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:25.187728  5940 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.272686  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:25.234189  2318 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:25.282059  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 47971.4us
I20260709 13:01:25.211611  5946 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.245s
I20260709 13:01:25.208132  5707 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.231750  5548 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:25.330909  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 209568us
I20260709 13:01:25.354488  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:25.210994  5103 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:25.226595  5527 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:25.347891  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 229087us
I20260709 13:01:25.350937  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:25.199442  5873 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.182752  5937 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.225258  5964 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:25.277189  5875 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.207492  5711 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:25.277056  5929 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:25.202428  5934 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:25.285699  5950 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:01:25.291261  5898 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:25.386880  5952 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.363s	user 0.001s	sys 0.001s
I20260709 13:01:25.474551  5898 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:25.476161  5898 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:25.476377  5937 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:25.476742  5898 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.273018  5908 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.374738  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:25.478086  5615 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:25.478385  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 103754us
I20260709 13:01:25.477919  5908 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.479432  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:25.480007  5953 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.295s	user 0.001s	sys 0.001s
I20260709 13:01:25.376538  2352 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:25.481808  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:25.482175  5972 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:25.482749  1559 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:01:25.482321  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:25.483063  5972 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:25.222153  5709 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.483587  1558 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:25.484333  1559 pending_rounds.cc:85] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 4
I20260709 13:01:25.484503  1710 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:25.484686  1559 pending_rounds.cc:107] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:01:25.485004  1559 raft_consensus.cc:2889] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:25.485154  1558 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:25.354100  5548 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.486636  5907 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:25.382547  5103 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEARNER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.384169  1706 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:25.226702  5919 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.365456  5612 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.488153  5612 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:25.489792  1601 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 2.758s (SENT) [suppressed 30 similar messages]
I20260709 13:01:25.378925  5959 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.404524  1709 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:25.406409  5957 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:01:25.496837  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 224320us
W20260709 13:01:25.407711  5958 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 13:01:25.411780  5956 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
W20260709 13:01:25.418232  5955 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 13:01:25.418864  5970 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:01:25.507037  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:25.427237  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:25.463950  5947 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:25.466544  5527 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:25.470104  5940 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:25.471654  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:25.485544  5709 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:25.293061  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:25.485926  5898 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.486975  5548 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:25.489593  5919 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:25.398603  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 47808us
I20260709 13:01:25.284770  5738 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:25.502313  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 217204us
I20260709 13:01:25.549599  2318 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:25.511708  1558 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:01:25.550809  1558 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 7
I20260709 13:01:25.551091  1558 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.8
I20260709 13:01:25.521885  2349 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:25.383519  5882 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.125s
W20260709 13:01:25.508519  5864 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.003s
I20260709 13:01:25.376444  5943 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.359s
I20260709 13:01:25.541268  5966 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.486676  5908 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:25.226637  5903 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.546712  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:25.529901  5875 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:25.561468  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:25.560886  1558 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Aborting config change with OpId 7.8: Aborted: Op aborted by new leader
I20260709 13:01:25.576234  1558 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:25.561158  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 206771us
I20260709 13:01:25.578402  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:25.585189  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:25.549233  5947 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:25.574131  1628 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:25.587343  1558 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:01:25.559960  5971 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:25.598953  1558 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:25.557922  5873 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:25.600082  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 49621.6us
I20260709 13:01:25.600149  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:25.566681  5935 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.527s	user 0.015s	sys 0.000s
I20260709 13:01:25.586717  5510 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:25.560407  5943 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:25.608281  1554 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:25.502313  5959 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.506920  5929 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.596706  5612 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:25.617690  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:25.573796  5875 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:25.560056  5948 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:01:25.625371  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2560) took 2150 ms. Trace:
I20260709 13:01:25.608922  5759 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:25.597119  5947 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:25.610423  5939 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.798s	user 0.009s	sys 0.009s
I20260709 13:01:25.600972  5919 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:25.606274  5966 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:25.491526  5964 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.562445  5738 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.624965  5971 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:25.643795  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.671609  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1925) took 2023 ms. Trace:
I20260709 13:01:25.621912  5782 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:25.620843  5937 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.636500  1555 rpcz_store.cc:276] 0709 13:01:23.474733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.032583 (+557850us) service_pool.cc:224] Handling call
0709 13:01:24.112927 (+ 80344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:24.199621 (+ 86694us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:24.199663 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:24.788418 (+588755us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b2c00000658. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:01:24.789199 (+   781us) log.cc:844] Serialized 52 byte log entry
0709 13:01:24.960702 (+171503us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:24.960854 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.624951 (+664097us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b5400304dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:25.625014 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.625085 (+    71us) raft_consensus.cc:1685] finished
0709 13:01:25.625098 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.625291 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125125,"raft.queue_time_us":11271,"raft.run_cpu_time_us":479,"raft.run_wall_time_us":107524,"spinlock_wait_cycles":242567808,"thread_start_us":170868,"threads_started":1,"wal-append.queue_time_us":343853}
I20260709 13:01:25.615105  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:25.624702  5943 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:25.638160  5730 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.834s	user 0.016s	sys 0.000s
W20260709 13:01:25.668555  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 156354us
W20260709 13:01:25.674005  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 82723.1us
I20260709 13:01:25.672143  1707 rpcz_store.cc:276] 0709 13:01:23.648437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.526347 (+877910us) service_pool.cc:224] Handling call
0709 13:01:24.526721 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.680146 (+153425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.680186 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:24.719457 (+ 39271us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.885319 (+165862us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:24.888225 (+  2906us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.892082 (+  3857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.892193 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.671331 (+779138us) raft_consensus.cc:1685] finished
0709 13:01:25.671358 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.671515 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8915,"prepare.queue_time_us":1956,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":45677,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":10007,"thread_start_us":264762,"threads_started":2,"wal-append.queue_time_us":434714}
I20260709 13:01:25.693740  1628 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:25.675366  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:25.636015  5875 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:25.675678  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:25.698218  2351 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:25.696089  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:25.702127  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:25.702879  2351 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:25.624547  5935 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.664376  5965 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:25.664695  5707 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:25.664222  5977 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:25.714680  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.712906  5923 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.923s	user 0.018s	sys 0.005s
I20260709 13:01:25.720808  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:25.701606  5966 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:25.676662  5980 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.665896  5759 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.725610  1634 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:25.693423  5559 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:01:25.725919  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:25.751821  1629 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:25.703657  5873 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.753067  1628 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:25.701045  5730 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.739012  5971 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:25.740725  5343 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.914s	user 0.011s	sys 0.006s
W20260709 13:01:25.741055  5839 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.919s	user 0.015s	sys 0.005s
I20260709 13:01:25.741389  5967 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:25.741706  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:25.776046  1707 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:25.742568  5103 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:25.705461  5937 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:25.721763  5927 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.915s	user 0.014s	sys 0.005s
I20260709 13:01:25.745687  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:25.751302  5488 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.199s	user 0.032s	sys 0.013s
I20260709 13:01:25.786875  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:25.787783  1632 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:25.703902  5908 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:25.754431  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:25.719707  5977 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.723116  5965 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.789839  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:25.797047  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 307239us
I20260709 13:01:25.803233  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:25.809224  1629 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:25.784142  5501 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.741s	user 0.012s	sys 0.002s
I20260709 13:01:25.766296  5987 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.816705  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:25.799431  5488 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.259s	user 0.033s	sys 0.014s
I20260709 13:01:25.817239  1628 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:01:25.820858  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:25.821655  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:25.784178  5948 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.782083  5839 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:25.813966  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 118064us
I20260709 13:01:25.820230  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:25.825696  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:25.718931  5947 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.798642  5934 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:25.751721  5510 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.812880  5980 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:25.832916  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:25.833710  2346 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:25.775085  5873 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:25.823391  5974 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.223s	user 0.002s	sys 0.000s
I20260709 13:01:25.809187  5977 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:25.836974  1632 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:25.798880  5103 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.850219   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:25.829926  5939 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:25.850590  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:25.825832  5707 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:25.825034  5501 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:25.808266  5908 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.875656  5873 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:25.817224  5965 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.865664  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:25.874637  5990 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
W20260709 13:01:25.899351  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 72264.5us
I20260709 13:01:25.903329  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:25.861271  5707 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.881604  5947 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:25.833772  5948 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.922116  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:25.898901  5982 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.898995  5992 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:25.940951  1629 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:01:25.906136  5738 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:25.911845  5707 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:25.917987  5959 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:25.952415  5982 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.960325  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 717) took 1014 ms. Trace:
I20260709 13:01:25.967501  1707 rpcz_store.cc:276] 0709 13:01:24.946291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.826744 (+880453us) service_pool.cc:224] Handling call
0709 13:01:25.843065 (+ 16321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:25.843455 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.843489 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:25.844037 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:01:25.959757 (+115720us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:25.959830 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.959956 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.960030 (+    74us) raft_consensus.cc:1685] finished
0709 13:01:25.960052 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.960234 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15849,"raft.queue_time_us":2733,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10293,"thread_start_us":55043,"threads_started":1,"wal-append.queue_time_us":79951}
W20260709 13:01:25.974098  5991 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:01:25.843752  5966 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:25.946568  5559 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:25.848990  5980 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:25.982337  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:25.985610  5948 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:25.987042  1525 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:25.990442  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:25.991438  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:25.996953  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:25.997325  1632 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:25.953877  5929 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:25.995378  2317 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:25.998564  1629 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:01:25.999395  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:25.963219  5505 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.000669  1629 pending_rounds.cc:85] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 6
I20260709 13:01:26.006165  1629 pending_rounds.cc:107] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:01:26.006385  1629 raft_consensus.cc:2889] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.011261  5964 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:26.011813  5103 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:25.988086  5559 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:26.017980  5998 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:01:26.018532  5982 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:26.021205  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1922) took 2373 ms. Trace:
I20260709 13:01:26.021737  1703 rpcz_store.cc:276] 0709 13:01:23.647609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.041303 (+393694us) service_pool.cc:224] Handling call
0709 13:01:24.124320 (+ 83017us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:24.302230 (+177910us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:24.454633 (+152403us) log.cc:844] Serialized 14 byte log entry
0709 13:01:25.722163 (+1267530us) spinlock_profiling.cc:243] Waited 9.84 ms on lock 0x7b1800419f20. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:01:25.821826 (+ 99663us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:25.834930 (+ 13104us) log.cc:844] Serialized 52 byte log entry
0709 13:01:25.835153 (+   223us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:25.835197 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.835255 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.020784 (+185529us) raft_consensus.cc:1685] finished
0709 13:01:26.020941 (+   157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.021129 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120411,"prepare.queue_time_us":179415,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":7827,"raft.queue_time_us":232512,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":7738,"spinlock_wait_cycles":22630144,"thread_start_us":247943,"threads_started":2,"wal-append.queue_time_us":1427273}
I20260709 13:01:25.987759  5488 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:01:26.007723  5973 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:26.026400  5875 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.027127  5908 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:26.006994  5760 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:26.000547  5968 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:26.032202  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:26.007764  5919 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:26.034197  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:26.008100  5730 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:26.012566  5759 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:26.015082  5559 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.029356  5875 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:25.958047  5992 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.037139  5968 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.048293  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:26.054278  5968 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:26.055763  5980 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:26.041826  5760 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:26.010823  5505 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.030418  5898 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.063629  1555 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:26.064960  5898 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:26.065681  5876 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:26.067088  5900 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:26.067582  5900 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:26.068411  2347 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 695) took 2397 ms (client timeout 1842 ms). Trace:
W20260709 13:01:26.068831  2347 rpcz_store.cc:269] 0709 13:01:23.671085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.671213 (+   128us) service_pool.cc:224] Handling call
0709 13:01:26.068176 (+2396963us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b5400541ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:26.068332 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1482837888}
I20260709 13:01:26.070343  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:26.071695  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:26.072875  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:26.073927  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2624) took 2979 ms. Trace:
I20260709 13:01:26.074334  1627 rpcz_store.cc:276] 0709 13:01:23.094915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.609805 (+514890us) service_pool.cc:224] Handling call
0709 13:01:24.150528 (+540723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.368663 (+218135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.073710 (+1705047us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b540010dbd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:26.073837 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1208460544}
I20260709 13:01:26.044734  5959 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:01:26.052350  5965 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:26.038200  5973 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:26.078440  5973 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.061940  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:26.080339  1628 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:26.044472  5919 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:26.089778  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:26.037189  5982 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.025s
W20260709 13:01:26.085176  5981 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.270s	user 0.000s	sys 0.003s
I20260709 13:01:26.112491  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:26.113242  1627 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:26.083199  5992 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:26.117544  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:26.057281  5760 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.123555  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:26.126343  1629 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:26.140105  1558 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
W20260709 13:01:26.147022  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.153060  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2456) took 1492 ms. Trace:
I20260709 13:01:26.153604  1635 rpcz_store.cc:276] 0709 13:01:24.660148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.660262 (+   114us) service_pool.cc:224] Handling call
0709 13:01:24.660581 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.960381 (+299800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.152766 (+1192385us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400196754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:26.152987 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675472256}
I20260709 13:01:26.156636  5900 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.160416  5505 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:26.161092  6005 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:01:26.129940  5548 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.001s
I20260709 13:01:26.160365  5900 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:26.162410  5917 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.849s	user 0.006s	sys 0.000s
I20260709 13:01:26.162007  5548 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.146227  5919 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.165221  5548 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:26.165580  5900 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:26.166781  5947 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.167364  5947 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.175727  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1942) took 1083 ms. Trace:
I20260709 13:01:26.176335  1707 rpcz_store.cc:276] 0709 13:01:25.092126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.970267 (+878141us) service_pool.cc:224] Handling call
0709 13:01:26.002854 (+ 32587us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:26.012939 (+ 10085us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.012979 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:26.029309 (+ 16330us) log.cc:844] Serialized 52 byte log entry
0709 13:01:26.095151 (+ 65842us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:26.103979 (+  8828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.104114 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.175294 (+ 71180us) raft_consensus.cc:1685] finished
0709 13:01:26.175324 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.175642 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5228,"raft.queue_time_us":5394,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":21369,"thread_start_us":65332,"threads_started":1,"wal-append.queue_time_us":108880}
W20260709 13:01:26.179390  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 84690.3us
I20260709 13:01:26.182330  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1981) took 2785 ms. Trace:
I20260709 13:01:26.183288  1552 rpcz_store.cc:276] 0709 13:01:23.396136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.720360 (+324224us) service_pool.cc:224] Handling call
0709 13:01:23.720841 (+   481us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:23.721345 (+   504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.721386 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:23.726125 (+  4739us) log.cc:844] Serialized 52 byte log entry
0709 13:01:24.033915 (+307790us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:24.033968 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.034289 (+   321us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.161720 (+2127431us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:26.169417 (+  7697us) raft_consensus.cc:1685] finished
0709 13:01:26.169445 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.180948 (+ 11503us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6305,"raft.queue_time_us":2599,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":10911,"spinlock_wait_cycles":15157120,"thread_start_us":207990,"threads_started":1,"wal-append.queue_time_us":427739}
W20260709 13:01:26.188902  2350 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 697) took 2516 ms (client timeout 1674 ms). Trace:
W20260709 13:01:26.189845  2350 rpcz_store.cc:269] 0709 13:01:23.672482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.672607 (+   125us) service_pool.cc:224] Handling call
0709 13:01:26.188624 (+2516017us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54004b7154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:26.188818 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952973184}
I20260709 13:01:26.193542  5510 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.194278  5510 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:26.195008  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1915) took 2549 ms (client timeout 2758 ms). Trace:
I20260709 13:01:26.195827  5987 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:26.196832  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1923) took 2549 ms. Trace:
I20260709 13:01:26.197161  5510 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:26.189028  5378 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:26.197299  1706 rpcz_store.cc:276] 0709 13:01:23.647889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.501649 (+853760us) service_pool.cc:224] Handling call
0709 13:01:24.520965 (+ 19316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.487708 (+966743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.196463 (+708755us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b540036e2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:26.196749 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1984626944}
W20260709 13:01:26.195616  1709 rpcz_store.cc:269] 0709 13:01:23.645127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.777726 (+132599us) service_pool.cc:224] Handling call
0709 13:01:26.194786 (+2417060us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540016ce54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:26.194945 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3720605440}
W20260709 13:01:26.205484  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.209811  1705 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:26.197749  5378 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:26.213411  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:26.216451  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:26.217530  5947 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:26.218420  2350 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:26.219962  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:26.220212  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:26.222216  1627 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:26.228785  2318 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:26.236056  5840 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:26.236723  5840 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:26.237447  5576 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 13:01:26.238022  5576 raft_consensus.cc:1068] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:26.239652  5510 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:26.241294  5782 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:26.241999  5782 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:26.241432  5378 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.242800  5378 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:26.250870  5986 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.481s	user 0.003s	sys 0.001s
I20260709 13:01:26.253244  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 709) took 1301 ms. Trace:
I20260709 13:01:26.254328  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2568) took 1463 ms. Trace:
I20260709 13:01:26.253784  2345 rpcz_store.cc:276] 0709 13:01:24.952077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.952775 (+   698us) service_pool.cc:224] Handling call
0709 13:01:26.188916 (+1236141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.189348 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.189383 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:26.189448 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:26.189628 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.189714 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.252996 (+ 63282us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54004b7150. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:26.253170 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1231,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":102,"spinlock_wait_cycles":2842003200}
I20260709 13:01:26.254762  1559 rpcz_store.cc:276] 0709 13:01:24.791189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.483642 (+692453us) service_pool.cc:224] Handling call
0709 13:01:25.484027 (+   385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:25.542263 (+ 58236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.542307 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:25.543021 (+   714us) log.cc:844] Serialized 52 byte log entry
0709 13:01:25.620211 (+ 77190us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:25.656526 (+ 36315us) log.cc:844] Serialized 14 byte log entry
0709 13:01:25.660770 (+  4244us) log.cc:844] Serialized 14 byte log entry
0709 13:01:25.671015 (+ 10245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.671101 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.974629 (+303528us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b18001f0420. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:26.249038 (+274409us) spinlock_profiling.cc:243] Waited 9.33 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:26.254096 (+  5058us) raft_consensus.cc:1685] finished
0709 13:01:26.254121 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.254267 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56729,"prepare.queue_time_us":88059,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":4757,"raft.queue_time_us":137497,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":25663,"spinlock_wait_cycles":42160640,"thread_start_us":65325,"threads_started":1,"wal-append.queue_time_us":151942}
I20260709 13:01:26.269682  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:26.270267  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:26.259992  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:26.331350  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.270787  5840 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.332085  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:26.333295  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51697.6us
I20260709 13:01:26.634346  5939 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:26.760154  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1935) took 2100 ms (client timeout 1983 ms). Trace:
W20260709 13:01:26.760699  1704 rpcz_store.cc:269] 0709 13:01:24.660100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.772149 (+112049us) service_pool.cc:224] Handling call
0709 13:01:26.759905 (+1987756us) spinlock_profiling.cc:243] Waited 51.5 ms on lock 0x7b54001865d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:26.760073 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118418688}
W20260709 13:01:26.673184  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.417527  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:26.624614  5942 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:26.560351  5505 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:26.663319  5973 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.012s	sys 0.070s
W20260709 13:01:26.497382  6012 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.172s	user 0.004s	sys 0.007s
I20260709 13:01:26.498935  5968 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.560781  5876 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.332737  1557 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:26.409092  5707 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.523541  5903 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:26.413980  5864 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.759598  1559 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:26.285588  5559 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.763906  2345 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:26.770066  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:26.767712  5977 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.883s	user 0.013s	sys 0.005s
W20260709 13:01:26.742652  5839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.009s	sys 0.023s
W20260709 13:01:26.770781  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:26.771929  5973 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.016s	sys 0.071s
I20260709 13:01:26.801330  2350 raft_consensus.cc:2368] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11.
W20260709 13:01:26.808729  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.834038  5523 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:26.834918  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2557) took 2181 ms (client timeout 1867 ms). Trace:
W20260709 13:01:26.835321  1710 rpcz_store.cc:269] 0709 13:01:24.653356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.756439 (+103083us) service_pool.cc:224] Handling call
0709 13:01:26.834663 (+2078224us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b5400247054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:26.834833 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1671568640}
I20260709 13:01:26.841213  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:26.834718  5523 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.784443  5840 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:26.791021  5839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.010s	sys 0.024s
I20260709 13:01:26.827489  5934 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:26.777812  5505 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.773692  5939 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.809167  5730 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:26.797299  5864 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.783212  5876 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.869104  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:26.878867  1632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.004s	sys 0.020s
W20260709 13:01:26.886911  1632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.004s	sys 0.024s
I20260709 13:01:26.812207  5977 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.866984  5839 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:26.899679  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:26.878366  5730 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:26.886744  6019 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.267s	user 0.001s	sys 0.001s
I20260709 13:01:26.908390  2347 raft_consensus.cc:2368] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13.
W20260709 13:01:26.904522  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.877813  5501 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.914047  5730 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:26.914191  5501 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.916460  5501 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:26.914076  5977 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:26.918356  5977 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:26.918866  5977 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.920909  5977 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:26.926476  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:26.927363  1557 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:26.870990  5934 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.956306  5501 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:26.967110  5501 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:26.971933  5943 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:26.972712  5501 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:26.974572  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2567) took 2183 ms. Trace:
I20260709 13:01:26.975152  1553 rpcz_store.cc:276] 0709 13:01:24.790872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.855709 (+ 64837us) service_pool.cc:224] Handling call
0709 13:01:24.920328 (+ 64619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:25.061451 (+141123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.974197 (+1912746us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540038a4d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:26.974492 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322989952}
I20260709 13:01:26.996412  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:26.997521  1627 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 7: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:26.997895  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:26.997911  1629 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 13:01:27.001447  1633 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:01:27.009239  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.976962  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:27.014147  1635 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:01:27.014619  1635 pending_rounds.cc:85] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 7
I20260709 13:01:27.014886  1635 pending_rounds.cc:107] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:01:27.015146  1635 raft_consensus.cc:2889] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:27.017616  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:27.018031  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:27.018484  1635 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:27.019364  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:27.020169  1635 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:27.020066  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1154) took 2455 ms. Trace:
W20260709 13:01:27.020943  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 366332us
I20260709 13:01:27.019443  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:27.024115  1601 leader_election.cc:336] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.983s (SENT)
I20260709 13:01:27.025338  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:27.010613  1553 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
W20260709 13:01:27.018158  6022 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.148s	user 0.005s	sys 0.000s
I20260709 13:01:27.022082  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:27.027880  5847 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:27.032254  5501 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:26.910097  5839 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:27.021109  2344 rpcz_store.cc:276] 0709 13:01:24.548669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.549345 (+   676us) service_pool.cc:224] Handling call
0709 13:01:24.555764 (+  6419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:24.563223 (+  7459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.563257 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:24.563933 (+   676us) log.cc:844] Serialized 52 byte log entry
0709 13:01:26.067526 (+1503593us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b18003feb00. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:01:26.107476 (+ 39950us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:26.119313 (+ 11837us) log.cc:844] Serialized 14 byte log entry
0709 13:01:26.119892 (+   579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.119984 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.158565 (+ 38581us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b18003feb00. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:27.003685 (+845120us) raft_consensus.cc:1685] finished
0709 13:01:27.003709 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.003887 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":553,"prepare.queue_time_us":12090,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":9954,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":5083,"spinlock_wait_cycles":17818112,"thread_start_us":39445,"threads_started":1,"wal-append.queue_time_us":1623832}
I20260709 13:01:27.032296  1635 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:27.037834  5847 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:27.053354  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:27.058921  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:27.060091  1635 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:27.061461  1627 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:27.092511  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:27.046262  5968 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:27.094666  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:27.064368  5919 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.023s	sys 0.047s
I20260709 13:01:27.054582  5876 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:27.038841  5576 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:01:27.116209  5576 raft_consensus.cc:3055] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:27.116706  5576 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:27.123076  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2649) took 2125 ms. Trace:
I20260709 13:01:27.125078  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 1999) took 1277 ms. Trace:
I20260709 13:01:27.125607  1558 rpcz_store.cc:276] 0709 13:01:25.847303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.902714 (+ 55411us) service_pool.cc:224] Handling call
0709 13:01:27.124835 (+1222121us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b5400311d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:27.124996 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495426816}
I20260709 13:01:27.127331  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:27.128211  1558 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:26.870986  5942 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:27.101750  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:27.058727  5707 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:27.098671  1553 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
W20260709 13:01:27.064117  5782 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.821s	user 0.013s	sys 0.003s
W20260709 13:01:27.099895  5919 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.023s	sys 0.047s
I20260709 13:01:27.115163  5523 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:27.057432  5851 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:01:27.117933  5576 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:27.132057  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 984) took 2208 ms. Trace:
I20260709 13:01:27.129407  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:27.219923  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:27.220989  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:27.218062  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 96616.1us
I20260709 13:01:27.221818  1635 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:27.123701  1636 rpcz_store.cc:276] 0709 13:01:24.997515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.998190 (+   675us) service_pool.cc:224] Handling call
0709 13:01:27.122757 (+2124567us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b5400349ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:27.122956 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1921433856}
I20260709 13:01:27.219875  2352 rpcz_store.cc:276] 0709 13:01:24.923483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.951455 (+ 27972us) service_pool.cc:224] Handling call
0709 13:01:24.952003 (+   548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.481689 (+529686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.131681 (+1649992us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b5400756554. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:27.131946 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":969347968}
I20260709 13:01:27.221009  2344 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:27.233587  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 717) took 1356 ms. Trace:
I20260709 13:01:27.234097  2344 rpcz_store.cc:276] 0709 13:01:25.876864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.070983 (+1194119us) service_pool.cc:224] Handling call
0709 13:01:27.233497 (+162514us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.070454  5559 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:27.236991  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:27.238345  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 626) took 1133 ms. Trace:
I20260709 13:01:27.238953  1558 rpcz_store.cc:276] 0709 13:01:26.105231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.236669 (+1131438us) service_pool.cc:224] Handling call
0709 13:01:27.238271 (+  1602us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.256367  1636 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:27.219385  5830 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.261370  5730 consensus_queue.cc:1059] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:27.261793  5830 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:27.262135  5851 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:27.262629  5851 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:27.263708  5851 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:27.238674  6021 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.348s	user 0.002s	sys 0.000s
I20260709 13:01:27.265064  5851 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:27.265854  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2480) took 1176 ms. Trace:
I20260709 13:01:27.266294  1632 rpcz_store.cc:276] 0709 13:01:26.089773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.089900 (+   127us) service_pool.cc:224] Handling call
0709 13:01:26.090231 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.209657 (+119426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.209685 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.210109 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.887626 (+677517us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:27.220148 (+332522us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.220428 (+   280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.220522 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.265582 (+ 45060us) raft_consensus.cc:1685] finished
0709 13:01:27.265608 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.265782 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117519,"prepare.queue_time_us":232643,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":124783,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":20125,"thread_start_us":924629,"threads_started":2,"wal-append.queue_time_us":973540}
I20260709 13:01:27.270078  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:27.271402  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1167) took 1103 ms. Trace:
I20260709 13:01:27.271835  2352 rpcz_store.cc:276] 0709 13:01:26.167627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.245855 (+1078228us) service_pool.cc:224] Handling call
0709 13:01:27.271312 (+ 25457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.276170  1527 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:27.264611  5830 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:27.277549  5959 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:27.273234  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:27.279242  5851 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:27.279670  2352 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEARNER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:01:27.237670  6027 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.286352  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1159) took 1165 ms. Trace:
I20260709 13:01:27.283483  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:27.287592  2352 rpcz_store.cc:276] 0709 13:01:26.121301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.272831 (+1151530us) service_pool.cc:224] Handling call
0709 13:01:27.286265 (+ 13434us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.287221  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1995) took 1474 ms. Trace:
I20260709 13:01:27.288448  1676 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:27.288681  1553 rpcz_store.cc:276] 0709 13:01:25.812730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.286308 (+1473578us) service_pool.cc:224] Handling call
0709 13:01:27.286671 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.286879 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.286907 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:27.286962 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:27.286997 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.287049 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.287165 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.287681  1557 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:27.282218  1558 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:27.290223  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:27.290834  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1994) took 1484 ms. Trace:
I20260709 13:01:27.290951  1553 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:27.291213  1558 rpcz_store.cc:276] 0709 13:01:25.806874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.260114 (+1453240us) service_pool.cc:224] Handling call
0709 13:01:27.260574 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.290534 (+ 29960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.290776 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:27.277168  5378 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.033s	user 0.011s	sys 0.003s
I20260709 13:01:27.093251  5494 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.179s
I20260709 13:01:27.292639  1558 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:27.299870  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 1996) took 1484 ms. Trace:
I20260709 13:01:27.300534  1558 rpcz_store.cc:276] 0709 13:01:25.815960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.292010 (+1476050us) service_pool.cc:224] Handling call
0709 13:01:27.292324 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.299588 (+  7264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.299807 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.301137  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:27.302423  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:27.303443  2344 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:27.304678  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1164) took 1138 ms. Trace:
I20260709 13:01:27.303057  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:27.305543  2344 rpcz_store.cc:276] 0709 13:01:26.166601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.261168 (+1094567us) service_pool.cc:224] Handling call
0709 13:01:27.304597 (+ 43429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.307772  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1913) took 3654 ms. Trace:
I20260709 13:01:27.308377  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:27.308174  1705 rpcz_store.cc:276] 0709 13:01:23.644105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.993419 (+349314us) service_pool.cc:224] Handling call
0709 13:01:24.032906 (+ 39487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.211739 (+2178833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.297760 (+1086021us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b540014d7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:27.297960 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5005712256}
I20260709 13:01:27.309068  1558 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:27.096719  5427 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.198s
I20260709 13:01:27.321553  1558 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:27.322818  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 627) took 1212 ms. Trace:
I20260709 13:01:27.323434  1558 rpcz_store.cc:276] 0709 13:01:26.110641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.320895 (+1210254us) service_pool.cc:224] Handling call
0709 13:01:27.321310 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.322024 (+   714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.322738 (+   714us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.325305  1558 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:27.321995  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 255147us
I20260709 13:01:27.312611  5576 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:27.330446  1557 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:01:27.331758  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 624) took 1227 ms. Trace:
I20260709 13:01:27.332290  1557 rpcz_store.cc:276] 0709 13:01:26.104323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.328561 (+1224238us) service_pool.cc:224] Handling call
0709 13:01:27.329858 (+  1297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.331269 (+  1411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.331679 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:27.347072  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.347929  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:27.348867  2352 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:27.306298  5505 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:27.281258  5851 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:27.351879  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 628) took 1237 ms. Trace:
I20260709 13:01:27.361469  1558 rpcz_store.cc:276] 0709 13:01:26.114037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.324595 (+1210558us) service_pool.cc:224] Handling call
0709 13:01:27.325034 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.334202 (+  9168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.351791 (+ 17589us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.353619  5492 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:27.236670  5782 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:27.359764  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 71758.9us
W20260709 13:01:27.365115  2318 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.928s (SENT)
I20260709 13:01:27.347364  5494 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:27.367264  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:27.367717  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:27.342044  5999 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:27.350003  5864 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:27.304483  6027 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:01:27.370209  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.356803  5934 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:27.364835  5505 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.366173  5492 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:27.366751  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:27.394268  1557 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:27.397311  6024 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:01:27.399035  5840 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.005s
I20260709 13:01:27.399749  5840 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:27.400399  5840 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:27.407747  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2559) took 3933 ms. Trace:
I20260709 13:01:27.408303  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 714) took 1641 ms. Trace:
I20260709 13:01:27.408191  1554 rpcz_store.cc:276] 0709 13:01:23.474427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.992853 (+518426us) service_pool.cc:224] Handling call
0709 13:01:24.356144 (+363291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.630385 (+1274241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.407447 (+1777062us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002ff654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:27.407670 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2870350592}
I20260709 13:01:27.409595  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:27.408807  2351 rpcz_store.cc:276] 0709 13:01:25.766969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.767488 (+   519us) service_pool.cc:224] Handling call
0709 13:01:25.767992 (+   504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:25.817437 (+ 49445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.817503 (+    66us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:25.818310 (+   807us) log.cc:844] Serialized 52 byte log entry
0709 13:01:25.912223 (+ 93913us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:26.017256 (+105033us) log.cc:844] Serialized 14 byte log entry
0709 13:01:26.019231 (+  1975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.407896 (+1388665us) spinlock_profiling.cc:243] Waited 49 ms on lock 0x7b540048ab54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:27.407963 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.408008 (+    45us) raft_consensus.cc:1685] finished
0709 13:01:27.408096 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.408245 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26946,"prepare.queue_time_us":38146,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":1354,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":6910,"spinlock_wait_cycles":112720512,"thread_start_us":134052,"threads_started":2,"wal-append.queue_time_us":186443,"wal-append.run_cpu_time_us":2456,"wal-append.run_wall_time_us":1017493}
I20260709 13:01:27.370939  5999 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.380803  5847 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:27.370848  5494 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.382048  5942 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:27.423832  5942 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:27.424530  5942 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:27.426420  6033 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:01:27.427588  5942 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:27.384070  5427 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.429179  5427 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:27.387547  5851 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:27.369323  5839 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:27.375056  5559 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:27.432245  5559 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:27.379773  2352 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:27.379875  5830 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:27.433741  5559 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:27.434727  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 722) took 1397 ms. Trace:
I20260709 13:01:27.435166  2352 rpcz_store.cc:276] 0709 13:01:26.037616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.317122 (+1279506us) service_pool.cc:224] Handling call
0709 13:01:27.434653 (+117531us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.423017  5494 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:27.436503  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:27.404310  5934 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.437500  5847 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:27.438038  5847 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:27.437851  5934 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:27.438982  5847 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:27.440079  5934 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:27.394179  6035 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:27.411926  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:27.444761  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1169) took 1252 ms. Trace:
I20260709 13:01:27.445242  2351 rpcz_store.cc:276] 0709 13:01:26.192413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.411670 (+1219257us) service_pool.cc:224] Handling call
0709 13:01:27.444694 (+ 33024us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.446951  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:27.449710  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1163) took 1283 ms. Trace:
I20260709 13:01:27.450244  2351 rpcz_store.cc:276] 0709 13:01:26.166235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.446586 (+1280351us) service_pool.cc:224] Handling call
0709 13:01:27.449629 (+  3043us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.451459  1557 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:27.451776  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:27.452648  1553 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:27.452787  2351 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:27.374585  6029 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:01:27.454013  2353 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:27.455405  6027 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:27.456691  5427 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:27.404644  5505 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:27.459501  1557 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:27.419649  5999 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:27.463929  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:27.467572  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:27.482272  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:27.482963  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.444012  6035 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:27.503093  6035 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:27.504411  6035 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:27.506556  5852 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:27.508914  5782 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:27.513044  5505 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:27.472993  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 128933us
I20260709 13:01:27.472800  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1162) took 1307 ms. Trace:
I20260709 13:01:27.535400  2351 rpcz_store.cc:276] 0709 13:01:26.165831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.451403 (+1285572us) service_pool.cc:224] Handling call
0709 13:01:27.472723 (+ 21320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.536902  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:27.472822  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.484045  1553 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:27.538647  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2003) took 1003 ms. Trace:
I20260709 13:01:27.539146  1553 rpcz_store.cc:276] 0709 13:01:26.535227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.466398 (+931171us) service_pool.cc:224] Handling call
0709 13:01:27.483779 (+ 17381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.538232 (+ 54453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.538569 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.485993  1557 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:27.462469  5492 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:27.486917  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 632) took 1360 ms. Trace:
I20260709 13:01:27.522861  5494 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:27.518895  5840 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:27.480726  6032 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:01:27.549718  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1166) took 1382 ms. Trace:
I20260709 13:01:27.550307  2351 rpcz_store.cc:276] 0709 13:01:26.167310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.536489 (+1369179us) service_pool.cc:224] Handling call
0709 13:01:27.549623 (+ 13134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.482225  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 991) took 1415 ms. Trace:
I20260709 13:01:27.541216  1554 rpcz_store.cc:276] 0709 13:01:26.126566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.410510 (+1283944us) service_pool.cc:224] Handling call
0709 13:01:27.410827 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.420181 (+  9354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.420256 (+    75us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.460331 (+ 40075us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.475219 (+ 14888us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:27.475985 (+   766us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.476221 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.476310 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.486688 (+ 10378us) raft_consensus.cc:1685] finished
0709 13:01:27.486712 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.486849 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":228,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":43,"raft.queue_time_us":2890,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":4745,"thread_start_us":14379,"threads_started":1,"wal-append.queue_time_us":14875}
I20260709 13:01:27.553968  2352 rpcz_store.cc:276] 0709 13:01:26.066895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.453344 (+1386449us) service_pool.cc:224] Handling call
0709 13:01:27.482159 (+ 28815us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.554718  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:27.556452  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.561800  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:27.562496  1632 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:27.562752  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:27.563411  1635 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:27.568082  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:27.570153  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:27.571332  1702 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:27.572605  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:27.568997  1704 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:27.583587  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:27.585695  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:27.808511  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 782) took 2424 ms. Trace:
W20260709 13:01:27.731575  2347 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 719) took 1770 ms (client timeout 1818 ms). Trace:
I20260709 13:01:27.780253  1635 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
W20260709 13:01:27.990765  2347 rpcz_store.cc:269] 0709 13:01:25.961006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.103752 (+142746us) service_pool.cc:224] Handling call
0709 13:01:27.731275 (+1627523us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b540002bf54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:27.731493 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811244800}
I20260709 13:01:27.991657  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:27.914826  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:27.670238  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:27.585270  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:27.595952  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 629) took 1477 ms. Trace:
W20260709 13:01:27.735818  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2575) took 1933 ms (client timeout 1687 ms). Trace:
I20260709 13:01:27.666501  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:27.995605  1558 rpcz_store.cc:276] 0709 13:01:26.118301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.362437 (+1244136us) service_pool.cc:224] Handling call
0709 13:01:27.362821 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.366413 (+  3592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.366452 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.370389 (+  3937us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.483096 (+112707us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:27.483730 (+   634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.483803 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.595637 (+111834us) raft_consensus.cc:1685] finished
0709 13:01:27.595671 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.595873 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3239,"raft.queue_time_us":5344,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":3980,"thread_start_us":101097,"threads_started":1,"wal-append.queue_time_us":219423}
I20260709 13:01:27.599210  1702 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:01:27.997364  1555 rpcz_store.cc:269] 0709 13:01:25.802594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.802806 (+   212us) service_pool.cc:224] Handling call
0709 13:01:27.735605 (+1932799us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54003c0fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:27.735746 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582577024}
I20260709 13:01:27.736330  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2556) took 4262 ms. Trace:
I20260709 13:01:28.001365  1556 rpcz_store.cc:276] 0709 13:01:23.473636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.778680 (+305044us) service_pool.cc:224] Handling call
0709 13:01:23.778985 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.198201 (+1419216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.736103 (+2537902us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400271fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:27.736266 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3247211392}
W20260709 13:01:27.935461  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:27.852196  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.997767  1557 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:27.735344  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:28.006790  1633 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
W20260709 13:01:27.998865  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 450287us
I20260709 13:01:28.008208  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:28.009786  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:27.670728  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 216171us
I20260709 13:01:27.990203  1634 rpcz_store.cc:276] 0709 13:01:25.383749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.383886 (+   137us) service_pool.cc:224] Handling call
0709 13:01:25.402416 (+ 18530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.761768 (+359352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.808136 (+2046368us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54001b6f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:27.808427 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":737914112}
W20260709 13:01:28.012217  1526 leader_election.cc:336] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.640s (SENT)
I20260709 13:01:27.803844  1632 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:01:28.012777  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:27.998088  1703 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:28.017643  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:27.989943  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:28.023424  2351 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:28.021631  1558 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:01:28.028103  1558 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 7
I20260709 13:01:28.003227  1555 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:28.029501  1558 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 8.8
I20260709 13:01:28.030917  1558 raft_consensus.cc:2917] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Aborting config change with OpId 8.8: Aborted: Op aborted by new leader
I20260709 13:01:28.032608  1558 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:28.021831  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.888909  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:28.164019  1704 raft_consensus.cc:2410] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:01:28.164467  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2579) took 1150 ms. Trace:
W20260709 13:01:28.164739  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 156797us
I20260709 13:01:27.546412  6036 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.176604  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:28.164896  1554 rpcz_store.cc:276] 0709 13:01:27.014012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.993755 (+979743us) service_pool.cc:224] Handling call
0709 13:01:27.994150 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.164117 (+169967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.164147 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.164196 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:28.164225 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.164281 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.164411 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.999287  1709 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:28.179296  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:27.992161  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:28.176196  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2006) took 1072 ms. Trace:
I20260709 13:01:28.182324  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:27.565285  6045 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:01:28.186671  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:28.187836  2347 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:28.188169  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2667) took 1119 ms. Trace:
I20260709 13:01:28.188478  1633 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:28.189100  6036 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.189781  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:27.693462  5909 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.188776  1627 rpcz_store.cc:276] 0709 13:01:27.069099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.096065 (+ 26966us) service_pool.cc:224] Handling call
0709 13:01:27.258948 (+162883us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.304946 (+ 45998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.304985 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.319643 (+ 14658us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.473405 (+153762us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:27.473511 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.473623 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.187790 (+714167us) raft_consensus.cc:1685] finished
0709 13:01:28.187822 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.188022 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24919,"raft.queue_time_us":34593,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":9853,"thread_start_us":107006,"threads_started":1,"wal-append.queue_time_us":115232}
I20260709 13:01:28.191160  5909 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:27.583993  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.192111  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:28.192299  6036 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:28.175828  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:27.547266  5730 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:28.185292  1632 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:01:28.183429  1704 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:28.221133  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.193938  1634 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:28.193410  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:27.744853  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:27.557955  5851 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.602097  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:28.198820  1602 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:27.811873  6017 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.037s	user 0.000s	sys 0.013s
I20260709 13:01:28.199491  1633 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:01:28.199606  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 715) took 3257 ms (client timeout 1872 ms). Trace:
I20260709 13:01:28.206519  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 982) took 3304 ms. Trace:
I20260709 13:01:28.222975  1701 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:28.193248  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2586) took 1094 ms. Trace:
I20260709 13:01:27.583655  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.189186  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:28.362593  1704 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:28.363786  2351 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:28.438124  2347 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:28.438666  1561 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:28.439198  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 992) took 1074 ms. Trace:
I20260709 13:01:28.439765  2347 rpcz_store.cc:276] 0709 13:01:27.364456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.188835 (+824379us) service_pool.cc:224] Handling call
0709 13:01:28.439091 (+250256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.743587  5378 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:28.443297  1708 rpcz_store.cc:269] 0709 13:01:24.941767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.948790 (+  7023us) service_pool.cc:224] Handling call
0709 13:01:28.199351 (+3250561us) spinlock_profiling.cc:243] Waited 50.6 ms on lock 0x7b5400077e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:28.199509 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116321664}
I20260709 13:01:28.434808  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:28.450902  1709 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:28.451131  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 86315.2us
I20260709 13:01:28.448225  2318 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:27.727600  6043 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:28.436249  2353 rpcz_store.cc:276] 0709 13:01:24.902039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.922366 (+ 20327us) service_pool.cc:224] Handling call
0709 13:01:24.941280 (+ 18914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.466620 (+2525340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.197277 (+730657us) spinlock_profiling.cc:243] Waited 2.51 s on lock 0x7b540049b454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:28.206416 (+  9139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5778129152}
I20260709 13:01:27.585863  5427 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:27.602444  5959 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:27.670130  5852 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.453912  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:27.690404  5937 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:27.596052  5501 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:27.679347  5576 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:27.559377  6039 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:28.462102  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1161) took 2312 ms. Trace:
I20260709 13:01:28.511248  2353 rpcz_store.cc:276] 0709 13:01:26.149962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.460705 (+2310743us) service_pool.cc:224] Handling call
0709 13:01:28.461171 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.461506 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.461553 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.461630 (+    77us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:28.461691 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.461820 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.462008 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.448634  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.163198  5968 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:28.514083  5968 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.480990  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:28.514676  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.514819  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:28.515537  1633 raft_consensus.cc:2410] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:28.516358  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:28.516546  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:28.481890  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:27.588050  5492 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:28.440613  6017 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.669s	user 0.004s	sys 0.016s
I20260709 13:01:28.516920  2353 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:27.732969  5935 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:28.539469  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1165) took 2372 ms. Trace:
I20260709 13:01:28.540288  5935 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:28.540850  5935 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.683874  5840 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:28.542126  5840 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:28.197533  5864 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.740051  5494 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:27.606340  5505 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:28.454519  5963 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.001s	sys 0.021s
I20260709 13:01:28.494998  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.449859  5378 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:28.497314  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 485091us
I20260709 13:01:28.495769  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1157) took 2379 ms. Trace:
I20260709 13:01:28.567708  2351 rpcz_store.cc:276] 0709 13:01:26.116487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.363090 (+2246603us) service_pool.cc:224] Handling call
0709 13:01:28.363543 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.494377 (+130834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.495409 (+  1032us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.487323  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:28.494313  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.458418  5423 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.515614  1704 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:28.367444  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:28.574234  1702 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:28.222638  5961 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:27.584640  6041 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.438149  5730 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.539992  2353 rpcz_store.cc:276] 0709 13:01:26.167016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.513083 (+2346067us) service_pool.cc:224] Handling call
0709 13:01:28.516504 (+  3421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.539050 (+ 22546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.539374 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.545957  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:27.608948  5939 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:28.547662  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:28.514739  5938 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.545656  1710 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:28.519217  2347 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:28.458139  5775 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:28.470990  5830 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.558176  1706 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:28.495234  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.552136  6036 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:28.497690  5847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.007s	sys 0.033s
W20260709 13:01:28.571182  1526 leader_election.cc:336] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.641s (SENT)
I20260709 13:01:28.613931  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:28.571684  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1171) took 2070 ms. Trace:
I20260709 13:01:28.614915  2351 rpcz_store.cc:276] 0709 13:01:26.499447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.568781 (+2069334us) service_pool.cc:224] Handling call
0709 13:01:28.569199 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.569445 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.569491 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.569553 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:28.569601 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.569670 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.569824 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:28.522439  5999 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.009s	user 0.012s	sys 0.000s
I20260709 13:01:28.618218  5999 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.577816  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.234391  5909 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:28.586302  1632 raft_consensus.cc:2410] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:28.627498  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.541301  5935 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.588231  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:28.622249  1708 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
W20260709 13:01:28.561247  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:28.557823  5963 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.001s	sys 0.021s
I20260709 13:01:28.584581  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:28.636699  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 65568.9us
I20260709 13:01:28.636996  1704 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:28.637734  1526 leader_election.cc:336] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.648s (SENT)
I20260709 13:01:28.485159  6043 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:28.651204  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 452440us
I20260709 13:01:28.655071  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:28.583858  5423 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.664451  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:28.623101  5847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.007s	sys 0.038s
I20260709 13:01:28.597637  1701 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:28.669901  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2574) took 1067 ms. Trace:
I20260709 13:01:28.670303  1701 rpcz_store.cc:276] 0709 13:01:27.602266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.443165 (+840899us) service_pool.cc:224] Handling call
0709 13:01:28.669836 (+226671us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.434038  5817 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:01:28.602226  5938 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:28.552032  5864 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:28.510015  5852 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:28.621445  5999 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:28.452454  5851 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:28.622509  1707 raft_consensus.cc:2368] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11.
I20260709 13:01:28.546677  5576 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.593446  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1160) took 2470 ms. Trace:
I20260709 13:01:28.673089  5963 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.710206  2347 rpcz_store.cc:276] 0709 13:01:26.122802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.441040 (+2318238us) service_pool.cc:224] Handling call
0709 13:01:28.495654 (+ 54614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.593047 (+ 97393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.593364 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:28.641765  1526 leader_election.cc:336] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.748s (SENT)
I20260709 13:01:28.711751  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:28.643975  1632 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:01:28.633136  1634 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:28.546506  5501 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.606328  5939 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:28.715709  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.048s	user 0.015s	sys 0.016s
W20260709 13:01:28.717290  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.049s	user 0.015s	sys 0.016s
W20260709 13:01:28.594355  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:28.671681  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 679514us
I20260709 13:01:28.672338  1602 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:28.573503  5505 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.639787  5910 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.546895  6039 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.497337  5427 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.665795  5984 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:28.746145  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 104421us
I20260709 13:01:28.729995  5817 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:01:28.707471  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 259340us
I20260709 13:01:28.590586  5961 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.752882  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:28.722425  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50158.8us
I20260709 13:01:28.725044  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:28.753968  1553 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:28.722887  5963 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.696739  5937 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:28.727072  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 724) took 1486 ms. Trace:
I20260709 13:01:28.763543  2347 rpcz_store.cc:276] 0709 13:01:27.241058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.726047 (+1484989us) service_pool.cc:224] Handling call
0709 13:01:28.726438 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.726705 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:28.726737 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.726783 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:28.726821 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.726874 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.727012 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.723407  5939 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:28.728977  5852 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:28.730433  1705 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:28.777761  1709 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:28.731056  1703 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.778538  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2578) took 1175 ms. Trace:
I20260709 13:01:28.623659  6036 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.693197  5840 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:28.736306  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:28.742383  1708 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:01:28.781314  1708 pending_rounds.cc:85] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 4
I20260709 13:01:28.781649  1708 pending_rounds.cc:107] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:01:28.781924  1708 raft_consensus.cc:2889] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:28.744043  1704 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:28.621735  5830 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.706161  1706 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:28.747653  1526 leader_election.cc:336] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.766s (SENT)
W20260709 13:01:28.722074  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.792073  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2582) took 1187 ms. Trace:
W20260709 13:01:28.792196  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.744900  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:28.793680  1632 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:28.793956  1706 rpcz_store.cc:276] 0709 13:01:27.604581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.694070 (+1089489us) service_pool.cc:224] Handling call
0709 13:01:28.705913 (+ 11843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.791725 (+ 85812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.791989 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.753612  6056 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.659852  5378 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.730919  6060 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:28.768656  5963 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:28.769575  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:28.771456  5937 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:28.590909  6041 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:28.779114  1709 rpcz_store.cc:276] 0709 13:01:27.603413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.180465 (+577052us) service_pool.cc:224] Handling call
0709 13:01:28.778462 (+597997us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.786223  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2577) took 1183 ms. Trace:
I20260709 13:01:28.817817  1704 rpcz_store.cc:276] 0709 13:01:27.603145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.584239 (+981094us) service_pool.cc:224] Handling call
0709 13:01:28.786140 (+201901us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.751741  5817 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:28.795961  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:28.762362  1602 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:28.804369  6062 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:28.806020  6036 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:28.806360  5510 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:28.806669  5378 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:28.852284  5510 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.852873  5510 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.855877  5510 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:28.856475  5510 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.813549  6056 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.858372  6062 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:28.858821  6062 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:28.859238  6062 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:28.860469  6056 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:28.816675  6060 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:28.862118  5730 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:28.864264  6060 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:28.868242  6016 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 2.202s	user 0.005s	sys 0.009s
I20260709 13:01:28.869308  6059 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:28.806854  5830 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:28.778057  5939 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:28.810837  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 74640.7us
I20260709 13:01:28.890985  1675 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:28.785121  5775 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:28.738173  5910 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.819185  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.897179  1704 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:28.898896  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.915707  1704 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:28.820756  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:28.822986  6051 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:01:28.824004  6043 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:28.824663  5615 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:28.825672  5576 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:28.827135  6041 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:28.828917  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2671) took 1500 ms. Trace:
I20260709 13:01:28.829495  5938 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:28.829566  6039 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:28.791311  6061 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:28.831239  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1172) took 2304 ms. Trace:
I20260709 13:01:28.813758  5946 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:28.832255  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 986) took 3796 ms. Trace:
I20260709 13:01:28.832574  5851 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:28.833182  5937 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:28.834297  5423 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:28.835785  5501 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:28.836539  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2489) took 1121 ms. Trace:
W20260709 13:01:28.839159  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 76883.3us
I20260709 13:01:28.969447  2348 rpcz_store.cc:276] 0709 13:01:25.035567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.062560 (+ 26993us) service_pool.cc:224] Handling call
0709 13:01:25.063046 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:25.089256 (+ 26210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.089294 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:25.098220 (+  8926us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.817086 (+2718866us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b1800349180. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:01:28.197826 (+380740us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:28.197880 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.197983 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.831936 (+633953us) raft_consensus.cc:1685] finished
0709 13:01:28.831959 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.832190 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18306,"raft.queue_time_us":69332,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":10743,"spinlock_wait_cycles":5115136,"thread_start_us":380476,"threads_started":1,"wal-append.queue_time_us":3551301}
I20260709 13:01:28.973799  1627 rpcz_store.cc:276] 0709 13:01:27.715492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.191389 (+475897us) service_pool.cc:224] Handling call
0709 13:01:28.191721 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.755383 (+563662us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.755419 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.755482 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:28.755520 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.836256 (+ 80736us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b5400197b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:28.836314 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.836465 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1295722240}
I20260709 13:01:28.841009  1703 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:28.842661  1705 raft_consensus.cc:2410] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:01:28.847297  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 99685.2us
I20260709 13:01:28.951225  5968 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:28.849860  5817 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:28.846717  5977 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.833120  5942 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:28.893042  1675 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:28.997714  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:28.895936  5907 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:28.856925  5510 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:28.963855  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 391960us
I20260709 13:01:28.963464  1636 rpcz_store.cc:276] 0709 13:01:27.328372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.332900 (+  4528us) service_pool.cc:224] Handling call
0709 13:01:27.333380 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.340522 (+  7142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.340569 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.358920 (+ 18351us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.549505 (+190585us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:27.549568 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.549668 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.828671 (+1279003us) raft_consensus.cc:1685] finished
0709 13:01:28.828701 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.828848 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":169202,"threads_started":1,"wal-append.queue_time_us":1328518}
I20260709 13:01:28.967177  2353 rpcz_store.cc:276] 0709 13:01:26.527298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.579525 (+2052227us) service_pool.cc:224] Handling call
0709 13:01:28.579892 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.583999 (+  4107us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.584040 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.584089 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:28.584138 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.830959 (+246821us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b5400642dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:28.831019 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.831178 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8793856}
I20260709 13:01:28.901947  6036 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 13:01:28.970697  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:28.978586  1526 leader_election.cc:336] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.867s (SENT)
I20260709 13:01:28.838819  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.003614  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:29.007413  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:29.007755  6059 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:29.008642  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:28.949429  5615 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:29.008685  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:28.998317  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 105386us
I20260709 13:01:29.010952  5910 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:29.011284  1675 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:29.012066  5910 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:29.014861  2348 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:29.008874  5907 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.997857  2347 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:29.013060  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:29.030052  1636 raft_consensus.cc:2368] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13.
I20260709 13:01:29.031167  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:29.032406  1636 raft_consensus.cc:2410] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:28.918020  1709 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:29.032666  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:28.980876  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:29.033912  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 53170.6us
I20260709 13:01:29.034811  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:29.036418  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:29.026665  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.038331  2348 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:28.987453  5494 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:29.039981  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.040822  1706 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:28.997413  5942 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.999982  5851 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:29.053743  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:28.986265  5775 consensus_queue.cc:1059] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:01:29.041658  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:29.028348  5910 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.042171  5946 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:29.057662  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:28.973609  6061 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:28.990963  5959 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:01:29.030169  5501 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.071774  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:28.999573  5977 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.046859  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:29.065841  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 313359us
I20260709 13:01:29.051856  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:29.057315  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.029210  5817 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:29.059376  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.049382  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:28.999718  6029 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:01:29.060128  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.064409  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:29.088129  1555 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:29.089146  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.089548  1553 raft_consensus.cc:2410] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:29.090046  1555 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:29.050055  5942 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:29.065778  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:29.103034  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.104482  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:29.052012  5423 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:29.026983  6063 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:01:29.090866  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.050740  5559 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:01:29.050981  6065 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:29.102429  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.121978  1554 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:29.121793  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:29.122146  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 710) took 4157 ms. Trace:
I20260709 13:01:29.122722  1627 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:29.122582  2346 rpcz_store.cc:276] 0709 13:01:24.964431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.966392 (+  1961us) service_pool.cc:224] Handling call
0709 13:01:24.988019 (+ 21627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.838284 (+850265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.121855 (+3283571us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b540054a454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:29.122073 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1944389504}
I20260709 13:01:29.047214  6066 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:29.103817  1706 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:29.106956  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:29.109175  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.109285  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:29.125062  1555 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:29.125182  1558 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:29.123852  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.127138  2345 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:29.109727  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:29.060433  5910 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:29.129046  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:29.130270  1632 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
W20260709 13:01:29.110826  2319 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.013s	sys 0.007s
I20260709 13:01:29.064146  6056 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:29.091674  1636 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:29.115209  1553 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:29.052789  5968 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:01:29.136264  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.140676  1706 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:29.117736  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.058593  5935 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.111245  5938 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:29.131742  2319 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.014s	sys 0.007s
I20260709 13:01:29.155759  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:29.117671  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 117987us
I20260709 13:01:29.142001  1709 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:29.104844  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.049260  5494 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.143143  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.150738  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.113596  5928 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:29.156967  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:29.168210  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:29.130995  2352 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.011s	sys 0.015s
W20260709 13:01:29.172832  2352 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.011s	sys 0.015s
I20260709 13:01:29.156284  1558 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:29.186522  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:29.188548  1706 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:29.121491  6065 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.189126  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:29.248193  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59147.2us
W20260709 13:01:29.250801  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.173308  2348 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:29.192819  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.133320  6063 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:29.194348  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 191401us
I20260709 13:01:29.143774  5775 consensus_queue.cc:1059] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:01:29.185802  6076 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:01:29.197925  6061 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.186110  5928 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:29.200987  6079 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:29.152734  6043 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:29.216300  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:29.117028  5830 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:01:29.112498  6041 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:01:29.224967  5977 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:29.225947  6029 raft_consensus.cc:3055] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:29.133630  6056 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:29.286243  6029 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:29.287062  6056 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.288414  6056 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:29.194958  5378 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:01:29.185760  6039 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:01:28.181680  1557 rpcz_store.cc:276] 0709 13:01:27.103963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.005143 (+901180us) service_pool.cc:224] Handling call
0709 13:01:28.005398 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.175670 (+170272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.175702 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.175750 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:28.175782 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.175860 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.176120 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169810,"raft.queue_time_us":169986}
I20260709 13:01:29.318218  6039 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:29.318697  6039 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:29.319630  6039 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.321067  6039 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:29.239784  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.240739  2345 raft_consensus.cc:2410] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:29.326421  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:29.259251  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:29.235960  6067 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:01:29.189771  6060 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:29.195182  5938 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:29.279361  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.274399  1709 raft_consensus.cc:2410] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:29.227145  5510 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.282693  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.283754  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:29.275564  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.287217  6029 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.102519  5501 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.238576  1554 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:29.248250  6080 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:01:29.295655  5943 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:29.298833  5961 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.302651  5864 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:29.305980  5830 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.307981  5910 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.262589  5730 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:29.256318  6065 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:29.314488  5967 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:29.271502  5937 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.315677  5427 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:29.190121  6064 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.158s
I20260709 13:01:29.327201  5615 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.328457  5935 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:29.282101  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 233811us
I20260709 13:01:29.329778  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.403906  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:29.404507  1704 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:01:29.325039  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2572) took 1723 ms. Trace:
I20260709 13:01:29.405259  1708 rpcz_store.cc:276] 0709 13:01:27.601730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.660293 (+1058563us) service_pool.cc:224] Handling call
0709 13:01:28.660800 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.782515 (+121715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.324608 (+542093us) spinlock_profiling.cc:243] Waited 81.1 ms on lock 0x7b54002d87d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:29.324953 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186554112}
I20260709 13:01:29.255066  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.333943  1674 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:29.407325  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 73497.5us
I20260709 13:01:29.407433  1706 raft_consensus.cc:2368] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13.
I20260709 13:01:29.226476  6062 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:29.409292  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 154682us
I20260709 13:01:29.342357  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2483) took 1759 ms. Trace:
I20260709 13:01:29.410128  1635 rpcz_store.cc:276] 0709 13:01:27.582892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.177628 (+594736us) service_pool.cc:224] Handling call
0709 13:01:28.178032 (+   404us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:28.585643 (+407611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.585701 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:28.622577 (+ 36876us) log.cc:844] Serialized 52 byte log entry
0709 13:01:28.972463 (+349886us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:29.281808 (+309345us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.341723 (+ 59915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.341860 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.341925 (+    65us) raft_consensus.cc:1685] finished
0709 13:01:29.341941 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.342250 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126915,"prepare.queue_time_us":398189,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":421080,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":23332,"thread_start_us":862825,"threads_started":3,"wal-append.queue_time_us":488710}
I20260709 13:01:29.347254  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.400807  5423 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:29.412396  1558 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:29.413452  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.413254  5423 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.414256  1558 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:29.347548  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.404516  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:29.415731  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.416431  1553 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:29.348203  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.349948  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.351986  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.277733  5928 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:29.417722  1557 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:29.397022  5967 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:29.419015  5967 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.420286  5967 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:29.326862  6070 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:01:29.424186  5907 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:29.356019  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.425079  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2571) took 1823 ms. Trace:
I20260709 13:01:29.365422  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:29.426563  1634 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:29.366221  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.396144  5830 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.427438  2348 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
I20260709 13:01:29.428478  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.359581  6085 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.429556  5427 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:29.429888  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:29.429793  6085 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.372843  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:29.373788  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 90233.8us
I20260709 13:01:29.339787  6043 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:29.396347  5730 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:29.351038  5775 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:29.442250  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.360772  5988 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.385632  6029 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:29.363615  6063 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:01:29.438956  5427 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.453938  5427 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:29.387158  5938 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.376233  5959 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:29.393399  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2578) took 2412 ms. Trace:
I20260709 13:01:29.456724  1560 rpcz_store.cc:276] 0709 13:01:26.981367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.736874 (+755507us) service_pool.cc:224] Handling call
0709 13:01:27.887213 (+150339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.887812 (+   599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.887855 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.888482 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.900592 (+1012110us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:28.900642 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.900730 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.393099 (+492369us) raft_consensus.cc:1685] finished
0709 13:01:29.393132 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.393321 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24278,"raft.queue_time_us":119,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":202185,"thread_start_us":453753,"threads_started":1,"wal-append.queue_time_us":833645}
I20260709 13:01:29.462244  6085 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.467386  6062 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:29.441727  5775 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:29.471988  5775 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:29.397769  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 288094us
I20260709 13:01:29.398761  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 718) took 3476 ms. Trace:
I20260709 13:01:29.377882  6080 consensus_queue.cc:1243] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:29.406631  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.408332  1674 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:29.340668  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.425554  1702 rpcz_store.cc:276] 0709 13:01:27.601403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.585974 (+984571us) service_pool.cc:224] Handling call
0709 13:01:28.586352 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.586744 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.586783 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:28.612336 (+ 25553us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.971600 (+359264us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:28.971688 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.981671 (+  9983us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.424783 (+443112us) raft_consensus.cc:1685] finished
0709 13:01:29.424812 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.424994 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136951,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"thread_start_us":307566,"threads_started":1,"wal-append.queue_time_us":469020}
I20260709 13:01:29.373944  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:29.425221  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.417778  1709 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:29.429136  5830 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.437844  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:29.437456  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:29.440979  2346 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:29.344388  6060 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:29.443214  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:29.441834  6089 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:29.563299  6089 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:29.563791  6089 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:29.564301  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:29.565156  1709 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:29.566442  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:29.566989  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2561) took 1999 ms (client timeout 1748 ms). Trace:
W20260709 13:01:29.567513  1710 rpcz_store.cc:269] 0709 13:01:27.567205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.567359 (+   154us) service_pool.cc:224] Handling call
0709 13:01:29.566770 (+1999411us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b5400078354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:29.566894 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1277426944}
W20260709 13:01:29.568049  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 159838us
I20260709 13:01:29.547798  1704 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:01:29.568285  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.570343  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.571094  1704 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
W20260709 13:01:29.572609  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 135210us
I20260709 13:01:29.573666  1527 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:29.575191  2346 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:01:29.475284  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:29.552425  2352 rpcz_store.cc:276] 0709 13:01:25.922040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.191061 (+2269021us) service_pool.cc:224] Handling call
0709 13:01:28.191411 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.191662 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:28.608059 (+416397us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.398382 (+790323us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.398446 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:29.398498 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.398569 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.398700 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9008,"prepare.queue_time_us":581141,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":126195,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"thread_start_us":1191957,"threads_started":3,"wal-append.queue_time_us":649450}
I20260709 13:01:29.576057  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:29.577000  1702 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:29.558075  1554 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:29.559301  6060 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:29.559887  5910 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:28.441655  1556 rpcz_store.cc:276] 0709 13:01:27.098766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.004059 (+905293us) service_pool.cc:224] Handling call
0709 13:01:28.004414 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.177063 (+172649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.177095 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.177143 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:28.192943 (+ 15800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.193033 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.193184 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":161064,"raft.queue_time_us":162372,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":169838}
I20260709 13:01:29.561858  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2571) took 4769 ms. Trace:
I20260709 13:01:29.562794  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 791) took 2002 ms. Trace:
I20260709 13:01:29.583503  1631 rpcz_store.cc:276] 0709 13:01:27.560960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.561140 (+   180us) service_pool.cc:224] Handling call
0709 13:01:27.589526 (+ 28386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.764956 (+1175430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.764998 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.765051 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:28.765097 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.562521 (+797424us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540017be54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:29.562578 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.562730 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2702589696}
I20260709 13:01:29.584235  1674 leader_election.cc:400] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13.
I20260709 13:01:29.584751  1674 leader_election.cc:403] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:29.575794  2346 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 7
I20260709 13:01:29.481587  2345 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:01:29.576820  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 139056us
I20260709 13:01:29.447207  5615 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:29.441140  5730 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:29.579957  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.410462  5963 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:01:29.575124  1527 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:29.593242  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:29.594007  1710 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:29.585366  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.557998  1558 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:29.582896  1561 rpcz_store.cc:276] 0709 13:01:24.791999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.538651 (+746652us) service_pool.cc:224] Handling call
0709 13:01:25.563923 (+ 25272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.439230 (+2875307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.561547 (+1122317us) spinlock_profiling.cc:243] Waited 2.87 s on lock 0x7b54003304d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:29.561777 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6610027264}
I20260709 13:01:29.585461  2346 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 8.8
I20260709 13:01:29.596441  2346 raft_consensus.cc:2917] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Aborting config change with OpId 8.8: Aborted: Op aborted by new leader
I20260709 13:01:29.596501  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.596969  2346 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:29.589048  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.590374  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:29.598665  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:29.600224  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:29.602125  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 126989us
I20260709 13:01:29.605270  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:29.607427  1601 leader_election.cc:400] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13.
I20260709 13:01:29.610042  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:29.610682  1556 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:29.610531  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:29.612402  2345 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:29.615444  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.618614  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:29.619462  2346 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:29.581167  5907 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:29.578241  5775 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:29.566414  6085 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:29.616967  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.581943  5505 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:29.670231  5505 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.671128  5505 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.672777  1556 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:29.578928  5935 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:29.619988  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:29.676875  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 56984.1us
I20260709 13:01:29.676913  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.678700  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.678895  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:29.617995  6064 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:29.600360  6020 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 2.763s	user 0.006s	sys 0.032s
I20260709 13:01:29.631109  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2583) took 2026 ms. Trace:
W20260709 13:01:29.666146  5939 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.755s	user 0.013s	sys 0.002s
W20260709 13:01:29.637790  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.594028  5852 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13.
I20260709 13:01:29.626883  5910 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:29.640542  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.641260  5378 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:29.642154  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2565) took 2073 ms (client timeout 1766 ms). Trace:
W20260709 13:01:29.643291  2344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 994) took 2253 ms (client timeout 1999 ms). Trace:
I20260709 13:01:29.644492  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:29.473327  5988 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.677912  5982 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:29.651495  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:29.696623  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52257.4us
I20260709 13:01:29.653317  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2491) took 1937 ms. Trace:
I20260709 13:01:29.588501  6093 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:29.467572  6043 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.599344  6063 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:29.621408  6041 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:29.719326  6041 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:29.719643  5730 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.722035  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.723241  1628 raft_consensus.cc:2410] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:29.723594  5730 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.723757  5945 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.724259  5945 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:29.725750  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.668244  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.664186  6087 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:29.581817  5959 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:29.679771  1704 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:29.666829  2346 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:29.629508  1710 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:29.676151  5935 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:29.695679  1707 rpcz_store.cc:269] 0709 13:01:27.568625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.601730 (+ 33105us) service_pool.cc:224] Handling call
0709 13:01:29.630276 (+2028546us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400223154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:29.642045 (+ 11769us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590421760}
I20260709 13:01:29.697903  1629 rpcz_store.cc:276] 0709 13:01:27.716112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.194555 (+478443us) service_pool.cc:224] Handling call
0709 13:01:28.458803 (+264248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.697339 (+238536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.697370 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:28.697905 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:01:29.066054 (+368149us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:29.066099 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.652585 (+586486us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54001b6054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:29.652662 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.652734 (+    72us) raft_consensus.cc:1685] finished
0709 13:01:29.652756 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.653213 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":547808128,"thread_start_us":367793,"threads_started":1,"wal-append.queue_time_us":702585}
I20260709 13:01:29.697098  2353 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:29.680893  5907 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:29.685010  6064 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:29.696071  2344 rpcz_store.cc:269] 0709 13:01:27.389820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.643061 (+2253241us) service_pool.cc:211] Skipping call since client already timed out
0709 13:01:29.643190 (+   129us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:01:29.680910  6036 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:29.703560  5378 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.711447  5501 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:29.754909  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.718062  6095 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:29.719938  6041 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:29.683683  1701 rpcz_store.cc:276] 0709 13:01:27.604873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.694336 (+1089463us) service_pool.cc:224] Handling call
0709 13:01:28.711095 (+ 16759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.800763 (+ 89668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.800799 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.800849 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:28.800892 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.630825 (+829933us) spinlock_profiling.cc:243] Waited 89.4 ms on lock 0x7b540016c954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:29.630890 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.631041 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205641984}
I20260709 13:01:29.659278  6085 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.728320  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:29.792047  2346 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:29.591244  5928 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.728451  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.728389  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:29.702229  6093 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:29.739156  5907 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:29.763990  5942 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.650s	user 0.011s	sys 0.000s
I20260709 13:01:29.766675  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2584) took 2161 ms. Trace:
I20260709 13:01:29.800690  1707 rpcz_store.cc:276] 0709 13:01:27.605192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.741460 (+2136268us) service_pool.cc:224] Handling call
0709 13:01:29.766043 (+ 24583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.766279 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.766307 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.766352 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:29.766388 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.766447 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.766600 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:29.770943  5988 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.774818  6060 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.777606  5505 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.779430  5935 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.781709  2352 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:29.691481  5939 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.745944  6064 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:29.777616  6065 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.787055  5378 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.787732  6089 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.808995  5942 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.800361  6093 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:29.823629  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 725) took 2541 ms. Trace:
I20260709 13:01:29.797825  6095 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.804751  5907 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:29.829288  6073 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.632s	user 0.000s	sys 0.002s
I20260709 13:01:29.830768  5939 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:29.830984  2350 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1168) took 3663 ms (client timeout 1659 ms). Trace:
W20260709 13:01:29.831389  2350 rpcz_store.cc:269] 0709 13:01:26.167900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.219667 (+ 51767us) service_pool.cc:224] Handling call
0709 13:01:29.830726 (+3611059us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54004ecad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:29.830885 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1334668672}
I20260709 13:01:29.832850  6089 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:29.833909  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 723) took 2607 ms. Trace:
I20260709 13:01:29.834393  6065 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.769058  5427 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:29.788347  2344 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:01:29.800186  6056 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.005s
I20260709 13:01:29.838193  2347 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:29.838621  5928 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.798018  6041 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:29.834309  5942 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:29.824275  2348 rpcz_store.cc:276] 0709 13:01:27.282517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.429370 (+2146853us) service_pool.cc:224] Handling call
0709 13:01:29.632198 (+202828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.632503 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.632545 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.632614 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:29.632666 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.632755 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.823308 (+190553us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b5400478950. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:29.823526 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":454353664}
I20260709 13:01:29.834329  2351 rpcz_store.cc:276] 0709 13:01:27.226894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.616250 (+1389356us) service_pool.cc:224] Handling call
0709 13:01:28.616768 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.617103 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.617142 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:28.627067 (+  9925us) log.cc:844] Serialized 26 byte log entry
0709 13:01:29.052916 (+425849us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:29.052965 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.053051 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.828732 (+775681us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:29.833568 (+  4836us) raft_consensus.cc:1685] finished
0709 13:01:29.833592 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.833831 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86818048,"thread_start_us":356742,"threads_started":1,"wal-append.queue_time_us":491514}
I20260709 13:01:29.841648  5707 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:29.861099  6095 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.862847  5427 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:29.698397  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:29.968125  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2656) took 2970 ms. Trace:
I20260709 13:01:29.786005  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2585) took 2180 ms. Trace:
I20260709 13:01:30.053005  1702 rpcz_store.cc:276] 0709 13:01:27.638515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.784976 (+2146461us) service_pool.cc:224] Handling call
0709 13:01:29.785390 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.785648 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.785676 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.785720 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:29.785753 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.785811 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.785924 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:29.798607  1704 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:30.060940  1704 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:29.799265  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 92447.1us
I20260709 13:01:29.796788  2345 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
W20260709 13:01:29.880831  6039 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.544s	user 0.004s	sys 0.008s
I20260709 13:01:29.878366  6060 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:29.883772  6064 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:29.888329  6069 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.675s	user 0.004s	sys 0.000s
I20260709 13:01:30.048002  1628 rpcz_store.cc:276] 0709 13:01:26.997161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.997633 (+   472us) service_pool.cc:224] Handling call
0709 13:01:29.967183 (+2969550us) spinlock_profiling.cc:243] Waited 2.72 s on lock 0x7b54002da5d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:29.967750 (+   567us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6261489536}
I20260709 13:01:29.877943  6078 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:30.076433  6078 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:30.076907  5928 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.078285  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:29.873997  6085 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:29.885814  6066 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.002s	sys 0.006s
I20260709 13:01:30.083016  6064 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:29.971114  1706 raft_consensus.cc:2368] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10.
W20260709 13:01:30.083016  6066 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.003s	sys 0.006s
I20260709 13:01:30.084647  6066 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:30.085506  5961 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:30.086184  5961 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:29.999850  1709 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 13:01:29.941181  1632 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:30.038259  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:30.038554  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:30.008271  5942 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.946117  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:29.986263  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.040573  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:29.934396  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 343888us
I20260709 13:01:29.969091  1634 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:01:29.889770  6083 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.446s	user 0.002s	sys 0.003s
I20260709 13:01:29.972038  1707 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:29.996522  1708 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:29.868853  5707 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:30.008960  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.804854  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2587) took 2165 ms. Trace:
W20260709 13:01:29.933691  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 358283us
I20260709 13:01:30.095592  5942 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:29.945911  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 394100us
I20260709 13:01:30.095427  1701 rpcz_store.cc:276] 0709 13:01:27.639429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.791259 (+2151830us) service_pool.cc:224] Handling call
0709 13:01:29.803842 (+ 12583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.804314 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:29.804360 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.804426 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:29.804486 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.804585 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.804771 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4848,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":2004}
I20260709 13:01:30.097877  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:30.098316  1701 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:30.094127  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1945) took 2368 ms. Trace:
I20260709 13:01:30.098970  5942 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:30.099116  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1962) took 1298 ms. Trace:
I20260709 13:01:30.099282  1708 rpcz_store.cc:276] 0709 13:01:27.723332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.994834 (+2271502us) service_pool.cc:224] Handling call
0709 13:01:29.995518 (+   684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.090690 (+ 95172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.091432 (+   742us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.099929  1635 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:30.101050  1558 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:30.101769  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:30.103065  1635 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:30.102948  5707 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:30.103835  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.104565  5707 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:30.105298  1558 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:01:29.971552  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2586) took 2332 ms (client timeout 1815 ms). Trace:
I20260709 13:01:30.099635  1701 rpcz_store.cc:276] 0709 13:01:28.801013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.097339 (+1296326us) service_pool.cc:224] Handling call
0709 13:01:30.097949 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.098810 (+   861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.099023 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:29.973431  1630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2647) took 4976 ms (client timeout 1596 ms). Trace:
W20260709 13:01:30.051265  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 352661us
I20260709 13:01:30.093713  1636 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
W20260709 13:01:30.106714  1630 rpcz_store.cc:269] 0709 13:01:24.996765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.996979 (+   214us) service_pool.cc:224] Handling call
0709 13:01:29.972525 (+4975546us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54001392d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:29.973074 (+   549us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3827257088}
I20260709 13:01:30.107921  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:30.092417  1627 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:30.099009  1634 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:29.935164   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:29.798380  1710 raft_consensus.cc:2410] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:01:29.958196  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 272312us
I20260709 13:01:30.077909  5928 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:29.875823  6093 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:30.129205  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:30.130990  6093 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:30.132498  5945 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:30.057463  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1949) took 1557 ms. Trace:
I20260709 13:01:30.078774  6078 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:30.090189  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.092931  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1947) took 2026 ms. Trace:
I20260709 13:01:30.051283  5378 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:30.109284  1708 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:30.139806  1701 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:01:30.109838  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.147985  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.107975  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:30.149775  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1963) took 1331 ms. Trace:
I20260709 13:01:30.149667  5378 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:30.149693  1630 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:30.146627  1709 rpcz_store.cc:276] 0709 13:01:28.171674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.998283 (+1826609us) service_pool.cc:224] Handling call
0709 13:01:29.999052 (+   769us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:30.090824 (+ 91772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.091062 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.152491  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:30.106037  1705 rpcz_store.cc:269] 0709 13:01:27.639119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.192967 (+553848us) service_pool.cc:224] Handling call
0709 13:01:29.970490 (+1777523us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b5400246654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:29.971100 (+   610us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1167834368}
W20260709 13:01:30.153227  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2581) took 2548 ms (client timeout 1790 ms). Trace:
W20260709 13:01:30.153713  1703 rpcz_store.cc:269] 0709 13:01:27.604293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.017302 (+413009us) service_pool.cc:224] Handling call
0709 13:01:30.152806 (+2135504us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b5400224554. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:30.152976 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1623760640}
I20260709 13:01:30.156880  5961 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:30.159595  5961 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:30.110209  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.117035  2345 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:01:30.119197  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:30.163399  1627 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
W20260709 13:01:30.164793  6072 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.937s	user 0.001s	sys 0.003s
I20260709 13:01:30.164745  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:30.166383  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:30.167898  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:30.169113  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.087296  6066 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:30.170284  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.170668  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 733) took 1116 ms. Trace:
I20260709 13:01:30.122987  1629 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:30.171161  1706 rpcz_store.cc:276] 0709 13:01:29.054280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.411711 (+357431us) service_pool.cc:224] Handling call
0709 13:01:30.170379 (+758668us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400077e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:30.170580 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239128448}
I20260709 13:01:30.173496  1635 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:30.174476  1674 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:30.123872  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1170) took 3930 ms. Trace:
I20260709 13:01:30.124650  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:30.125603  1636 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:30.128010  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1965) took 1243 ms. Trace:
I20260709 13:01:30.132205  6085 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:30.176501  1710 rpcz_store.cc:276] 0709 13:01:28.882182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.728235 (+846053us) service_pool.cc:224] Handling call
0709 13:01:30.125268 (+397033us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.169238  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:30.081370  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1959) took 1304 ms. Trace:
I20260709 13:01:30.108136  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:30.134799  1702 rpcz_store.cc:276] 0709 13:01:28.500360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.055339 (+1554979us) service_pool.cc:224] Handling call
0709 13:01:30.056072 (+   733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.056767 (+   695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.056809 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.056889 (+    80us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:30.056981 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.057100 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.057310 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:30.139811  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.150797  5378 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:30.120627  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:30.094334  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2588) took 2452 ms. Trace:
I20260709 13:01:30.150514  1708 rpcz_store.cc:276] 0709 13:01:28.816050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.100482 (+1284432us) service_pool.cc:224] Handling call
0709 13:01:30.101084 (+   602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.147230 (+ 46146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.147493 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.181835  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1966) took 1281 ms. Trace:
I20260709 13:01:30.183775  1701 rpcz_store.cc:276] 0709 13:01:28.900474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.110157 (+1209683us) service_pool.cc:224] Handling call
0709 13:01:30.139327 (+ 29170us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.181358 (+ 42031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.181757 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.186630  6061 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:30.187209  6061 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:30.183219  1707 rpcz_store.cc:276] 0709 13:01:27.639745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.802316 (+2162571us) service_pool.cc:224] Handling call
0709 13:01:29.970769 (+168453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.090538 (+119769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.091761 (+  1223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.182343  1704 rpcz_store.cc:276] 0709 13:01:28.775763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.059339 (+1283576us) service_pool.cc:224] Handling call
0709 13:01:30.060195 (+   856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.066225 (+  6030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.079811 (+ 13586us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.175926  2349 rpcz_store.cc:276] 0709 13:01:26.193276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.515350 (+2322074us) service_pool.cc:224] Handling call
0709 13:01:28.635594 (+120244us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b5800003c98. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:01:28.635900 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.808337 (+172437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.808372 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:28.839406 (+ 31034us) log.cc:844] Serialized 26 byte log entry
0709 13:01:29.152065 (+312659us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:29.152111 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.835113 (+683002us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b5400796b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:29.835167 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.889368 (+ 54201us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b24003aab58. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda244739c4 00007fda24471acb 00007fda2462cdc9 00007fda2462ca55 00007fda2462c09d 00007fda2725e10e 00007fda2727588c 00007fda27274bb6 00007fda272747d8 00007fda2727e221
0709 13:01:30.112050 (+222682us) raft_consensus.cc:1685] finished
0709 13:01:30.112099 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.123646 (+ 11547us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":49621,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":79,"spinlock_wait_cycles":425726464,"thread_start_us":283419,"threads_started":1,"wal-append.queue_time_us":527828}
I20260709 13:01:30.208017  1702 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:30.209024  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 727) took 1157 ms. Trace:
I20260709 13:01:30.212553  1702 rpcz_store.cc:276] 0709 13:01:29.051750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.206708 (+1154958us) service_pool.cc:224] Handling call
0709 13:01:30.207345 (+   637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.208616 (+  1271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.208948 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.210328  1701 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:30.209127  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:30.215583  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 732) took 1161 ms. Trace:
I20260709 13:01:30.216279  1701 rpcz_store.cc:276] 0709 13:01:29.053995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.209523 (+1155528us) service_pool.cc:224] Handling call
0709 13:01:30.210007 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.214036 (+  4029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.215481 (+  1445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.212375  1628 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:30.236837  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:30.247670  1703 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:01:30.248102  1703 pending_rounds.cc:85] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 6
I20260709 13:01:30.248390  1703 pending_rounds.cc:107] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:30.248728  1703 raft_consensus.cc:2889] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:30.249969  5910 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:30.250283  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:30.253136  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 49373.3us
I20260709 13:01:30.197901  6061 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:30.256165  6061 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:30.256696  6063 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:30.264004  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 156050us
I20260709 13:01:30.264408  5427 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:30.266244  5427 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:30.266824  5427 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:30.271596  1627 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:30.185941  6041 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:30.241639  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.244540  5999 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.434s	user 0.005s	sys 0.007s
W20260709 13:01:30.246332  2318 leader_election.cc:336] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.566s (SENT)
I20260709 13:01:30.237411  1676 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:30.272903  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2594) took 1257 ms. Trace:
I20260709 13:01:30.265270  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:30.284495  1703 rpcz_store.cc:276] 0709 13:01:29.015586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.246530 (+1230944us) service_pool.cc:224] Handling call
0709 13:01:30.247056 (+   526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.249372 (+  2316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.272766 (+ 23394us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:30.283213  5999 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.516s	user 0.006s	sys 0.007s
I20260709 13:01:30.299762  1628 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:30.306222  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:30.204998  5907 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:30.317145  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 722) took 1266 ms. Trace:
I20260709 13:01:30.320509  1627 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:30.323274  1705 rpcz_store.cc:276] 0709 13:01:29.050156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.281773 (+1231617us) service_pool.cc:224] Handling call
0709 13:01:30.305781 (+ 24008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.316578 (+ 10797us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.316612 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.316665 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:30.316716 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.316796 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.316955 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.327833  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.339596  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.351905  6062 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:30.352694  6062 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.353219  6062 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:30.354724  6078 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:30.356827  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 55667.9us
I20260709 13:01:30.358657  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:30.361783  5494 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:30.362073  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.363365  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 730) took 1310 ms. Trace:
I20260709 13:01:30.363849  1708 rpcz_store.cc:276] 0709 13:01:29.053280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.189233 (+1135953us) service_pool.cc:224] Handling call
0709 13:01:30.210598 (+ 21365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.211199 (+   601us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:30.211680 (+   481us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.362833 (+151153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.362892 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:30.362948 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.363029 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.363274 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62617,"prepare.queue_time_us":10670,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":2057,"thread_start_us":87751,"threads_started":1}
I20260709 13:01:30.365245  5928 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:30.366401  1558 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
W20260709 13:01:30.366925  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 129675us
I20260709 13:01:30.369603  6108 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:30.371480  6108 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:30.373705  6066 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:30.375049  1709 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:30.375866  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2593) took 1360 ms. Trace:
I20260709 13:01:30.376116  1631 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:30.376289  1709 rpcz_store.cc:276] 0709 13:01:29.015233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.212656 (+1197423us) service_pool.cc:224] Handling call
0709 13:01:30.238950 (+ 26294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.374830 (+135880us) spinlock_profiling.cc:243] Waited 57.9 ms on lock 0x7b54001c3ca0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27651dc0 00007fda276dc1c0 00007fda276dd231 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c
0709 13:01:30.375560 (+   730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.375799 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133116800}
I20260709 13:01:30.378346  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2675) took 1895 ms. Trace:
I20260709 13:01:30.379308  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:30.378175  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 132116us
I20260709 13:01:30.363286  1552 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:30.410784  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:30.414605  6041 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:30.417635  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.417959  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.419168  6063 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:30.421339  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:30.421809  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:30.422427  2349 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:30.422689  2345 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:30.423264  1628 pending_rounds.cc:85] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 4
I20260709 13:01:30.423651  1628 pending_rounds.cc:107] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:01:30.423916  1628 raft_consensus.cc:2889] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:30.422353  1602 connection.cc:256] RPC call timeout handler was delayed by -1.0532708060000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
I20260709 13:01:30.429420  1707 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:30.459221  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1971) took 1205 ms. Trace:
I20260709 13:01:30.459841  1707 rpcz_store.cc:276] 0709 13:01:29.253389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.215346 (+961957us) service_pool.cc:224] Handling call
0709 13:01:30.318532 (+103186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.441875 (+123343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.459060 (+ 17185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.460547  1627 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:30.464028  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:30.352046  6036 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:30.477620  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:30.486158  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 98926.3us
I20260709 13:01:30.485514  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:30.466949  5961 raft_consensus.cc:670] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:30.506556  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:30.506505  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 64226.9us
I20260709 13:01:30.520360  1560 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:30.378790  1633 rpcz_store.cc:276] 0709 13:01:28.482983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.597746 (+114763us) service_pool.cc:224] Handling call
0709 13:01:28.598144 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.599879 (+  1735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.599922 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:28.600536 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 13:01:29.013200 (+412664us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:30.038442 (+1025242us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b1800427b40. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda2b0d23c7 00007fda276e49a7 00007fda276f40bf
0709 13:01:30.038798 (+   356us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.039650 (+   852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.039798 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.163654 (+123856us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:30.377820 (+214166us) raft_consensus.cc:1685] finished
0709 13:01:30.377852 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.378116 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7048,"prepare.queue_time_us":352525,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":59827,"raft.run_cpu_time_us":497,"raft.run_wall_time_us":30998,"spinlock_wait_cycles":41577600,"thread_start_us":557262,"threads_started":2,"wal-append.queue_time_us":547978}
I20260709 13:01:30.480171  6036 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.335691  5427 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:30.513932  2353 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:30.515033  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:30.536829  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 72222.2us
W20260709 13:01:30.541508  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 50640.4us
I20260709 13:01:30.556564  1558 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:01:30.556739  6057 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:30.557610  6057 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:30.558202  6057 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:30.559087  1633 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:30.560701  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:30.562120  5847 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:30.562702  5847 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:30.564040  5847 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:30.441200  6093 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
W20260709 13:01:30.570631  1525 leader_election.cc:336] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.798s (SENT)
W20260709 13:01:30.578069  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 312937us
I20260709 13:01:30.579332  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:30.580686  6076 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:30.580740  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:30.581234  6076 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:30.581763  6076 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:30.582923  5427 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:30.589883  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 47726.7us
I20260709 13:01:30.594930  5907 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:30.597891  6108 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:30.534181  6066 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:30.604483  6066 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.605072  6066 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:30.528649  6036 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:30.636963  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 56339us
I20260709 13:01:30.556563  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:30.630822  1631 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:30.664341  5928 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.665376  5928 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:30.646697  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:30.686599  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.692369  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 136069us
W20260709 13:01:30.697779  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.698518  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.701148  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 725) took 1650 ms. Trace:
I20260709 13:01:30.706614  1707 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:30.698155  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:30.707484  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 292676us
I20260709 13:01:30.667912  5928 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:30.716214  2345 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:30.697397  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:30.719501  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:30.694979  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:30.721446  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.679265  6041 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:30.730626  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:30.733057  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.729863  1602 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:30.708549  5494 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:30.749156  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.755640  1559 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:30.701774  1706 rpcz_store.cc:276] 0709 13:01:29.050977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.262675 (+1211698us) service_pool.cc:224] Handling call
0709 13:01:30.263286 (+   611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.264136 (+   850us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:30.264543 (+   407us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.700624 (+436081us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.700707 (+    83us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:30.700785 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.700884 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.701064 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8488,"prepare.queue_time_us":57284,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":16381,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":178,"thread_start_us":349882,"threads_started":1}
I20260709 13:01:30.740869  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:30.748827  2318 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.776s (SENT) [suppressed 48 similar messages]
I20260709 13:01:30.769995  1703 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:30.771862  1705 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:30.772989  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:30.768357  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:30.779422  1525 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.984s (SENT)
W20260709 13:01:30.780941  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 145655us
I20260709 13:01:30.765101  6041 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:30.782132  5830 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:30.783000  6041 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:30.789279  1702 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:30.787801  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:30.793304  1708 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:30.800640  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:30.801654  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.802019  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 173676us
I20260709 13:01:30.802975  1635 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:30.804729  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.806663  5378 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.534s	user 0.006s	sys 0.010s
I20260709 13:01:30.807228  5378 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:30.809353  1602 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:30.811410  1602 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:30.812423  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.812635  1627 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:30.814867  5378 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:30.815685  5378 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:30.774161  5494 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:30.822420  5922 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:30.823135  5922 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:30.823637  5922 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:30.823855  5494 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:30.824383  6120 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.472s	user 0.004s	sys 0.000s
W20260709 13:01:30.825963  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.828876  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:30.829586  1706 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 7.
I20260709 13:01:30.832717  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:30.832929  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:30.834865  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.836606  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:30.836885  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.838022  1557 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:30.838284  1704 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:30.818120  5865 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:30.817924  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 98430.8us
I20260709 13:01:30.839979  5865 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:30.818425  1633 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:30.843696  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:30.844084  6124 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.465s	user 0.004s	sys 0.000s
I20260709 13:01:30.845351  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:30.845436  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:30.848318  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:30.849001  1710 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:30.850091  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:30.776482  5907 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:01:30.850916  1710 raft_consensus.cc:2410] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 13 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:30.851545  5817 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:30.839747  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:30.802271  5707 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.697s	user 0.014s	sys 0.005s
I20260709 13:01:30.863533  1559 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
W20260709 13:01:30.878486  2318 leader_election.cc:336] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.872s (SENT)
I20260709 13:01:30.869940  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:30.885887  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:30.896938  5830 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:30.897627  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:30.897076  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:30.898644  1701 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:30.899524  6062 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.546s	user 0.013s	sys 0.000s
I20260709 13:01:30.900041  6062 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:30.900708  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 147495us
I20260709 13:01:30.902014  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:30.902247  6063 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:30.905022  6063 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:30.905864  6063 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:30.905174  1707 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:30.909866  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:30.915642  2318 leader_election.cc:336] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.888s (SENT)
I20260709 13:01:30.909199  6063 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:30.939779  6076 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:30.891491  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:30.899525  1704 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:30.860102  5378 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:30.964210  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0647925590000114s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23750000000000002s for a response
I20260709 13:01:30.963300  1555 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:30.968734  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:30.969853  1561 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:30.974530  6062 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:30.967311  5378 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:30.981953  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:30.987051  1631 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:30.988597  1630 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:30.992609  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.991982  1674 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:31.007185  1559 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:31.009902  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:31.012035  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:30.973151  6036 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:30.873440  5910 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
I20260709 13:01:30.841322  5865 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:31.011009  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:30.990772  5928 raft_consensus.cc:2764] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:31.024823  1676 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:31.026888  1557 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:31.027956  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.035090  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:31.035090  6093 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.044996  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:31.045207  6093 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:31.045756  2350 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:31.048027  6093 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:31.048815  1555 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:31.050235  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:31.051205  2348 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:31.044268  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:31.060436  1552 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:31.031582  5494 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:31.072140  2353 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:31.073701  2347 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:31.053349  5910 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:31.067248  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:31.078872  2346 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:31.082623  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:31.094501  1628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.027s	sys 0.024s
W20260709 13:01:31.094928  1628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.027s	sys 0.024s
W20260709 13:01:31.079136  5999 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.042s	sys 0.083s
I20260709 13:01:31.042140  6057 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:31.131868  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:31.077170  5910 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:31.133875  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:31.079125  5494 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:31.147794  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:31.139619  1556 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.030s	sys 0.029s
W20260709 13:01:31.149231  1556 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.030s	sys 0.030s
I20260709 13:01:31.149281  5847 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.160440  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.163007  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:31.093844  5427 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:31.133013  5999 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.043s	sys 0.084s
I20260709 13:01:31.170480  1708 raft_consensus.cc:2368] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9.
I20260709 13:01:31.171636  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.172173  1552 raft_consensus.cc:2410] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:01:31.172603  1707 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:31.173524  1701 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:31.174093  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.193632  6063 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:31.177405  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:31.170660  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.185195  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:31.143723  6085 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.187361  1635 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:31.148838  6076 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:31.195731  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:31.188338  6078 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.829s	user 0.011s	sys 0.006s
I20260709 13:01:31.200887  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:31.209029  1707 raft_consensus.cc:2410] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:31.204346  1709 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:31.204370  1706 raft_consensus.cc:2376] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13. Re-sending same reply.
I20260709 13:01:31.212560  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.209848  1705 raft_consensus.cc:2376] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13. Re-sending same reply.
W20260709 13:01:31.218897  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.220767  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.197660  6036 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:31.213496  1704 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:31.211232  2353 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:31.198534  5427 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:31.150226  5847 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:31.246378  1707 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:31.262142  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.259387  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2505) took 1116 ms. Trace:
I20260709 13:01:31.263005  1704 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:01:31.264647  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 453350us
W20260709 13:01:31.265883  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.237350  6063 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:31.266363  2351 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:31.269452  1559 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:31.268349  1708 raft_consensus.cc:2410] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:01:31.269034  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.270478  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:31.271529  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 187357us
I20260709 13:01:31.271910  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:31.220048  6085 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:31.279021  6085 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:31.302106  2350 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:31.240917  5910 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:31.312290  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 308527us
I20260709 13:01:31.265239  5494 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:31.263329  1636 rpcz_store.cc:276] 0709 13:01:30.142646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.202734 (+ 60088us) service_pool.cc:224] Handling call
0709 13:01:30.210696 (+  7962us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:30.211560 (+   864us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:30.328076 (+116516us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.373142 (+ 45066us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b48000318d8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:01:30.735703 (+362561us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:30.778845 (+ 43142us) log.cc:844] Serialized 52 byte log entry
0709 13:01:30.779061 (+   216us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:30.779354 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.238888 (+459534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.239008 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.255413 (+ 16405us) raft_consensus.cc:1685] finished
0709 13:01:31.255450 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.259306 (+  3856us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3620,"prepare.queue_time_us":352140,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":93,"raft.queue_time_us":10480,"raft.run_cpu_time_us":3232,"raft.run_wall_time_us":182999,"spinlock_wait_cycles":19878272,"thread_start_us":874218,"threads_started":3,"wal-append.queue_time_us":533858}
W20260709 13:01:31.316422  6137 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:01:31.310783  6057 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:31.358670  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 47026.8us
I20260709 13:01:31.361331  1561 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
W20260709 13:01:31.355131  5907 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.039s	sys 0.067s
W20260709 13:01:31.365103  1554 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.008s	sys 0.014s
W20260709 13:01:31.365427  5907 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.039s	sys 0.068s
W20260709 13:01:31.365903  6138 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:01:31.366534  1554 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.008s	sys 0.014s
I20260709 13:01:31.369015  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:01:31.371976  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 85850.4us
I20260709 13:01:31.372551  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2602) took 1032 ms. Trace:
I20260709 13:01:31.381357  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2497) took 1238 ms. Trace:
I20260709 13:01:31.382076  1630 rpcz_store.cc:276] 0709 13:01:30.138613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.152128 (+ 13515us) service_pool.cc:224] Handling call
0709 13:01:31.376793 (+1224665us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b540010d6d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:31.377069 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651429632}
I20260709 13:01:31.386063  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:31.322832  5427 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:31.374994  5910 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:31.373167  6061 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.116s	user 0.005s	sys 0.019s
I20260709 13:01:31.389640  1636 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:31.370070  6063 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:31.395884  6139 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 13:01:31.398691  6140 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:01:31.398548  1602 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:31.376334  1554 rpcz_store.cc:276] 0709 13:01:30.342626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.384164 (+ 41538us) service_pool.cc:224] Handling call
0709 13:01:30.414651 (+ 30487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.415427 (+   776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.415479 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:30.416117 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 13:01:30.721884 (+305767us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:31.371862 (+649978us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.372188 (+   326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.372294 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.372344 (+    50us) raft_consensus.cc:1685] finished
0709 13:01:31.372354 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.372488 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":92216,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":54,"raft.queue_time_us":5079,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"thread_start_us":920427,"threads_started":2,"wal-append.queue_time_us":522604}
W20260709 13:01:31.412360  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:31.416898  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 424107us
W20260709 13:01:31.432447  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 401278us
W20260709 13:01:31.430792  6066 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.825s	user 0.020s	sys 0.009s
I20260709 13:01:31.437429  6066 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:31.428398  6059 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.006s	sys 0.017s
W20260709 13:01:31.443651  6059 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.082s	user 0.006s	sys 0.018s
I20260709 13:01:31.450052  2345 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:01:31.454859  6076 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.455698  6076 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:31.457154  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:31.458415  6076 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:31.460086  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:31.461586  6076 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:31.458695  2318 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:31.474511  6151 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:31.475215  6151 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:31.476511  6155 raft_consensus.cc:2764] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:31.442392  5847 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:31.478184  6155 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.478834  6155 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:31.480000  5922 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.010s
I20260709 13:01:31.480535  5922 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:31.481628  6155 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:31.482940  5922 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:31.483188  6155 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:31.483937  6144 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:31.486588  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:31.497507  6076 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:31.499661  6076 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:31.503180  6066 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:31.504106  6076 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:31.524410  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.552975  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2699) took 1229 ms. Trace:
I20260709 13:01:31.433718  5830 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
W20260709 13:01:31.558290  1525 leader_election.cc:336] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.588s (SENT)
I20260709 13:01:31.553426  1628 rpcz_store.cc:276] 0709 13:01:30.324005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.324247 (+   242us) service_pool.cc:224] Handling call
0709 13:01:30.351256 (+ 27009us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.424747 (+ 73491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.424791 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:30.427847 (+  3056us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.095415 (+667568us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:31.095491 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.095616 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.552684 (+457068us) raft_consensus.cc:1685] finished
0709 13:01:31.552731 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.552902 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":573407,"threads_started":1,"wal-append.queue_time_us":754105}
I20260709 13:01:31.559021  1675 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:31.504078  6066 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:31.579931  5559 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 13:01:31.594386  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 195947us
I20260709 13:01:31.551997  5847 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:31.598452  5847 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:31.631036  6149 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:31.631843  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:31.599547  6151 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:31.634982  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 277731us
W20260709 13:01:31.633713  5378 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.656s	user 0.008s	sys 0.008s
W20260709 13:01:31.675148  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.634061  5922 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:31.628898  6066 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:31.684654  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 56034.3us
I20260709 13:01:31.667982  5830 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:31.673560  5961 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:31.695794  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 855852us
W20260709 13:01:31.694679  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:31.690353  5817 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.838s	user 0.011s	sys 0.011s
I20260709 13:01:31.700266  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.675635  5378 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:31.739941  5378 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:31.708503  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.698704  5847 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:31.708832  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:31.750479  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:31.752753  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:31.752009  1527 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:31.756491  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:31.757264  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.706851  5817 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:31.760125  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 321223us
W20260709 13:01:31.763501  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 68329.9us
W20260709 13:01:31.764820  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.765045  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:31.765450  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:31.769222  6061 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:31.769779  6061 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.770145  6061 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:31.773761  1602 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:31.775161  1636 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:31.776536  5999 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:31.777029  5999 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.777420  5999 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:31.780063  5847 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:31.781859  6162 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:31.782677  6162 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:31.785269  6162 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:31.786662  5559 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.787351  5559 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:31.790547  1636 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:01:31.701699  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 278586us
I20260709 13:01:31.689749  5830 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:31.793192  5830 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:31.793820  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:31.794638  5830 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:31.794850  1635 raft_consensus.cc:2410] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:31.692590  5961 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.693455  6057 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:01:31.710765  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.711061  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:31.800436  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 341818us
I20260709 13:01:31.800963  1701 raft_consensus.cc:2376] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11. Re-sending same reply.
I20260709 13:01:31.712042  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:31.712800  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:31.713776  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.714435  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.712867  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.714903  6154 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:31.715449  6157 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:31.716754  6147 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:01:31.720630  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:31.722339  2348 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:31.723090  5946 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:31.796109  5961 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:31.803226  1710 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:31.803251  1702 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:31.811549  1635 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:31.815002  1706 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:31.822623  1709 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:31.822593  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 648) took 1414 ms. Trace:
I20260709 13:01:31.827762  2318 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:31.829326  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:31.831967  1708 raft_consensus.cc:2410] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:01:31.811254  6076 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:31.817581  5817 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.818631  5922 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:31.821997  6059 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:31.822154  5946 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:31.871939  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 60174.7us
I20260709 13:01:31.873168  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 746) took 1045 ms. Trace:
I20260709 13:01:31.874998  1702 rpcz_store.cc:276] 0709 13:01:30.827987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.867608 (+ 39621us) service_pool.cc:224] Handling call
0709 13:01:31.872923 (+1005315us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b540036ddd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:31.873083 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1956891904}
W20260709 13:01:31.878515  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 128154us
I20260709 13:01:31.830210  1556 rpcz_store.cc:276] 0709 13:01:30.415725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.440889 (+ 25164us) service_pool.cc:224] Handling call
0709 13:01:30.498474 (+ 57585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.594465 (+ 95991us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:31.149884 (+555419us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.814919 (+665035us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.815336 (+   417us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:31.815815 (+   479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.816211 (+   396us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.816375 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":219598,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":107987,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":5952,"thread_start_us":1155611,"threads_started":3,"wal-append.queue_time_us":518188}
I20260709 13:01:31.863118  6076 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:31.868029  6059 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:01:31.919831  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 52357.3us
I20260709 13:01:31.936374  5999 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:31.918898  5559 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:31.912730  6151 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.918035  6149 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:01:31.946224  6059 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:31.946913  6166 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:31.887889  6085 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.948385  1705 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
W20260709 13:01:31.954319  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 118469us
I20260709 13:01:31.920799  6076 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:31.969759  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.972069  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:31.853763  6157 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:31.991209  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 284384us
I20260709 13:01:31.978672  6162 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:31.998035  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 68106.3us
I20260709 13:01:31.999284  5961 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.000547  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:32.001982  6167 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:32.001629  1458 catalog_manager.cc:5697] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 6 to 7, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 7 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:32.003037  6167 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.006000  6169 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:01:32.013829  6162 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.017737  6173 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.018638  6173 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:31.883564  5817 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:32.021143  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 56853.1us
W20260709 13:01:32.036972  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 284975us
I20260709 13:01:32.005143  1458 catalog_manager.cc:5697] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 7 to 8, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 8 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.997944  6163 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.087s
W20260709 13:01:32.046873  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 52535us
I20260709 13:01:32.056231  2351 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:32.061033  5559 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:32.062095  6169 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:32.066079  1675 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:32.065632  6057 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:01:32.070937  5910 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.666s	user 0.015s	sys 0.000s
I20260709 13:01:32.081241  1703 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:32.091473  5961 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:32.094573  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.101982  1602 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:32.105087  6085 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.108458  6085 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:32.109972  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:32.110869  1635 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
W20260709 13:01:32.074589  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.114837  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.115471  6173 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:32.110137  1456 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 273) took 24303 ms (client timeout 15000 ms). Trace:
I20260709 13:01:32.116217  6177 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:32.043483  6066 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:32.118752  5559 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:32.119264  5559 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:32.117794  6176 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:32.120229  6176 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.121937  6169 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:32.122444  6169 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:32.123643  6169 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.124923  6176 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:32.125882  6163 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:32.127959  5946 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.128609  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:32.117801  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 63121.5us
I20260709 13:01:32.046875  6093 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:32.133795  6057 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:32.133342  1458 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.990031  6078 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:32.120400  5559 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.049805  6154 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:01:32.155417  6057 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:32.040786  6144 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.005s
I20260709 13:01:32.136432  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:32.128015  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:32.149902  5494 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:32.071899  5999 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:32.120338  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 171669us
I20260709 13:01:32.149215  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:31.975250  6151 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:32.155360  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.125391  6169 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:32.167100  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:32.167183  1526 leader_election.cc:400] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9.
I20260709 13:01:32.176159  1526 leader_election.cc:403] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:01:32.148492  6163 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:32.165922  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1192) took 1161 ms. Trace:
W20260709 13:01:32.183509  2318 connection.cc:256] RPC call timeout handler was delayed by -1.1977380149999703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 13:01:32.184234  2318 connection.cc:256] RPC call timeout handler was delayed by -1.1590380149999646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 13:01:32.184757  2318 connection.cc:256] RPC call timeout handler was delayed by -1.1509380150000084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 13:01:32.187623  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:32.193221  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:32.194036  1561 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:32.196408  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:32.197271  1559 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:32.201396  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:32.202486  2347 raft_consensus.cc:2410] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:32.203902  6183 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:32.188917  1554 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:01:32.161993  6057 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.160249  5559 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.207646  1458 catalog_manager.cc:5697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:32.162695  5946 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:32.158973  6063 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:32.178520  5922 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:32.171742  6154 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:32.162678  6144 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:32.164537  6155 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:32.180132  2344 rpcz_store.cc:276] 0709 13:01:31.004735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.048606 (+ 43871us) service_pool.cc:224] Handling call
0709 13:01:31.049104 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.434311 (+385207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.434428 (+   117us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:31.450086 (+ 15658us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.950643 (+500557us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:31.985228 (+ 34585us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.985653 (+   425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.985794 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:32.165556 (+179762us) raft_consensus.cc:1685] finished
0709 13:01:32.165581 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.165721 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36997,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":431914,"raft.run_cpu_time_us":396,"raft.run_wall_time_us":31497,"thread_start_us":760704,"threads_started":2,"wal-append.queue_time_us":621509}
I20260709 13:01:32.229787  1558 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:32.234902  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 169146us
I20260709 13:01:32.161644  6176 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:32.209654  5907 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:32.239521  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.242956  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.259231  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 110251us
I20260709 13:01:32.237135  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:32.172087  6093 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.279311  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:32.279361  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:32.282635  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 115706us
I20260709 13:01:32.238368  6155 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:32.288247  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.221931  5494 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:01:32.267292  6162 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:32.304819  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:32.290091  1628 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:32.286403  1458 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:32.301313  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:32.308810  1630 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:01:32.116922  1456 rpcz_store.cc:269] 0709 13:01:07.806120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.806370 (+   250us) service_pool.cc:224] Handling call
0709 13:01:32.109805 (+24303435us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:23.980699 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:24.340236 (+359537us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:24.340318 (+    82us) write_op.cc:435] Acquired schema lock
0709 13:01:24.340362 (+    44us) tablet.cc:662] Decoding operations
0709 13:01:24.688107 (+347745us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:24.791227 (+103120us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:24.797769 (+  6542us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:01:25.421778 (+624009us) tablet.cc:701] Row locks acquired
0709 13:01:25.421839 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:01:25.422375 (+   536us) write_op.cc:270] Start()
0709 13:01:25.422711 (+   336us) write_op.cc:276] Timestamp: P: 1783602085422280 usec, L: 0
0709 13:01:25.422760 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 13:01:25.424029 (+  1269us) log.cc:844] Serialized 7922 byte log entry
0709 13:01:26.634974 (+1210945us) op_driver.cc:464] REPLICATION: finished
0709 13:01:27.729275 (+1094301us) spinlock_profiling.cc:243] Waited 74 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda245d00ac 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2b110ef0
0709 13:01:28.693282 (+964007us) write_op.cc:301] APPLY: starting
0709 13:01:28.702293 (+  9011us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:29.469733 (+767440us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:29.469781 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:29.866337 (+396556us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:29.866870 (+   533us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:01:29.866942 (+    72us) write_op.cc:312] APPLY: finished
0709 13:01:29.867712 (+   770us) log.cc:844] Serialized 180 byte log entry
0709 13:01:29.879568 (+ 11856us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:29.882393 (+  2825us) write_op.cc:454] Released schema lock
0709 13:01:29.882516 (+   123us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:29.883153 (+   637us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:29.883197 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1043916,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":1964772,"apply.run_cpu_time_us":48805,"apply.run_wall_time_us":1239267,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87727,"num_ops":20,"prepare.queue_time_us":827012,"prepare.run_cpu_time_us":37817,"prepare.run_wall_time_us":1687559,"raft.queue_time_us":352189,"raft.run_cpu_time_us":22921,"raft.run_wall_time_us":2040918,"replication_time_us":1097959,"spinlock_wait_cycles":170234624,"thread_start_us":2583529,"threads_started":5,"wal-append.queue_time_us":318019,"wal-append.run_cpu_time_us":34874,"wal-append.run_wall_time_us":2509076}]]}
I20260709 13:01:32.308112  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:32.314373  2344 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:01:32.315519  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 273916us
I20260709 13:01:32.317935  2347 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:32.292378  6155 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:01:32.307186  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 189808us
I20260709 13:01:32.350513  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:32.351471  6180 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:32.351987  6180 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.352381  6180 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.352689  6093 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:32.353693  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 70012.7us
W20260709 13:01:32.352675  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.354581  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:32.355160  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:32.355857  2344 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:32.356206  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:32.317588  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:32.259192  6063 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.358865  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:32.360471  6171 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.319660  5427 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:32.364181  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 103651us
I20260709 13:01:32.346367  6162 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:32.364521  5830 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:32.389940  5830 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.390446  5830 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.393432  6162 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.393965  6162 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:32.384960  6063 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:32.401153  6063 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:32.401638  6063 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:32.402026  6063 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.364375  5427 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:32.385830  6066 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.403643  5427 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.403988  6066 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.364163  6085 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:32.406219  6085 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:32.386646  6173 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:32.406723  6085 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:32.407084  6173 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:32.407617  6173 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:32.408228  6061 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.637s	user 0.015s	sys 0.000s
I20260709 13:01:32.408756  6061 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:32.410331  6184 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 13:01:32.388307  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:32.388664  2318 leader_election.cc:336] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.536s (SENT)
I20260709 13:01:32.320093  6182 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:32.427510  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:32.428112  1561 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
W20260709 13:01:32.417462  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 146776us
I20260709 13:01:32.418475  1458 catalog_manager.cc:5697] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 6 to 7, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 7 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:32.429942  6157 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:32.431960  1458 catalog_manager.cc:5697] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:32.439708  6066 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:32.415009  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:32.443718  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:32.445704  5847 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.665s	user 0.014s	sys 0.000s
I20260709 13:01:32.447876  5830 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.449862  1675 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:32.452297  5830 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:32.454027  5922 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:32.454603  5922 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.455065  5922 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.457967  1458 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 2 to 8, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 8 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:32.461925  6175 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 13:01:32.469291  6076 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.470345  6151 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:32.472062  6076 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:32.374675  6171 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.472543  1458 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:01:32.477785  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 89194.4us
I20260709 13:01:32.479640  6061 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:32.479768  2318 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.623s (SENT)
I20260709 13:01:32.480417  6061 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:32.480964  6061 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:32.479837  1458 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:32.475126  6171 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:32.476393  6059 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.476553  6085 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.302245  6176 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.367147  5817 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:32.483927  6167 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:32.474444  6057 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.426425  6144 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:32.466961  6185 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:01:32.488746  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 74687.5us
W20260709 13:01:32.489460  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 64210.9us
I20260709 13:01:32.471634  5427 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.493165  1458 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 8 to 9, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 9 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:32.269444  6163 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.480362  2318 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:32.502239  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 397899us
I20260709 13:01:32.525365  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:32.532584  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 82832.8us
I20260709 13:01:32.509899  6171 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:32.545361  1636 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:32.534575  1458 catalog_manager.cc:5697] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.550486  1602 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:32.553814  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.515427  6144 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:32.536325  6108 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:32.548316  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 104691us
I20260709 13:01:32.553390  6171 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.574575  1602 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:32.576210  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:32.578522  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.579218  1630 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:32.581987  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:32.560789  6144 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:32.598556  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.573078  6163 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.569178  6164 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEARNER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:32.582926  6171 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.607203  6199 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:32.609107  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 256864us
I20260709 13:01:32.588824  5910 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:32.576771  6167 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
W20260709 13:01:32.621403  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 141649us
I20260709 13:01:32.621752  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:32.625540  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:32.626170  6085 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:32.629626  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:32.634140  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.638027  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 54883.1us
I20260709 13:01:32.574781  6061 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.617297  6108 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:32.655850  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:32.656723  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 82262us
I20260709 13:01:32.636413  5427 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:32.655146  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:32.643523  6182 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:32.660161  1602 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:32.660215  1561 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:32.661001  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:32.644449  6199 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:32.661649  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 79817.8us
I20260709 13:01:32.666785  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:32.651611  6180 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.671423  6198 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.671674  6182 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.672624  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:32.672922  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:32.677624  6162 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.680325  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:01:32.656375  6183 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9.
I20260709 13:01:32.710332  2348 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:32.706888  1561 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.710170  1703 raft_consensus.cc:606] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:32.711793  1703 raft_consensus.cc:614] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:32.712864  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 55409.8us
W20260709 13:01:32.713881  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 357888us
I20260709 13:01:32.714322  1602 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:32.686946  6200 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:32.715009  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:32.716593  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:32.716356  6198 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.717999  1559 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:32.714672  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:32.714370  6180 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:32.719648  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:32.720813  6162 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:32.721675  5427 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:32.721728  6108 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:32.724957  6108 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:32.726615  2346 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:32.730027  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:32.737254  2348 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.737084  1602 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:32.709537  6182 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.750146  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:32.751088  1635 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:32.754944  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:32.718436  6057 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:32.755880  2346 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.731434  6199 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:32.729043  6108 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:32.725708  6198 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:32.767716  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:32.732769  6085 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:32.733614  6059 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:32.745724  6200 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:32.728605  6163 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:01:32.784970  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:32.731837  6202 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:32.792807  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:32.799546  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:32.802701  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.794242  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
W20260709 13:01:32.802027  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.799041  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:32.809867  1703 raft_consensus.cc:606] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:32.811453  1703 raft_consensus.cc:614] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:32.807379  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:32.813089  1561 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:32.772287  6061 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:32.807613  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:32.806177  6197 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:01:32.780354  6151 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:32.830206  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 113687us
I20260709 13:01:32.837414  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:32.821703  5559 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.591s	user 0.014s	sys 0.000s
I20260709 13:01:32.842087  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:32.797614  6085 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:32.809295  6200 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:32.789255  6164 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:01:32.851280  1553 raft_consensus.cc:2410] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:01:32.851260  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44008.9us
W20260709 13:01:32.867365  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.838353  6151 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.868036  6151 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.869956  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:32.803781  6066 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:01:32.875015  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:32.879518  2352 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:32.879944  6201 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.880483  6201 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.880860  6200 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.882889  6201 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:32.845512  5865 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:01:32.892566  6176 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:32.893430  6208 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:32.893944  6208 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.894310  6208 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.894143  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:32.898916  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.906905  2346 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:32.908442  2348 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:01:32.913568  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48850.6us
I20260709 13:01:32.841254  6202 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:32.914729  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 48359us
I20260709 13:01:32.917763  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:32.921074  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:32.835332  5999 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:01:32.926326  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5838267321348eb83df088988da6449"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:01:32.927173  1703 raft_consensus.cc:606] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:32.927615  1703 raft_consensus.cc:614] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:32.918810  6155 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.004s
W20260709 13:01:32.928781  6205 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.224s	user 0.001s	sys 0.002s
I20260709 13:01:32.915949  6076 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.930049  6202 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.928519  6155 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.938997  6057 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.939831  6057 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:32.941618  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:32.939225  1630 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:32.942777  1628 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:32.852075  6157 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:01:32.952309  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:32.963266  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.963229  1627 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:32.964314  1636 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:32.876320  6061 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:32.947669  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:32.949011  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:32.972455  1628 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:32.913226  5830 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:32.948185  6173 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.004s
I20260709 13:01:32.952220  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:32.970357  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:32.941601  6163 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.954201  1632 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:32.975754  1630 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:32.954829  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:32.976393  1631 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:32.954952  6155 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:32.955484  6144 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.956102  1634 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:32.930079  6076 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.956957  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:32.959123  1527 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:32.961006  6176 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:32.984364  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:32.916342  6183 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:32.919798  5559 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:01:32.954002  6085 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.999876  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:32.978778  1629 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:32.980285  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 143337us
I20260709 13:01:33.000778  1631 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.001284  1526 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:32.981213  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:32.954836  6063 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.552s	user 0.013s	sys 0.000s
I20260709 13:01:32.982585  1554 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:32.978210  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:01:33.016594  1703 raft_consensus.cc:606] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.017145  1703 raft_consensus.cc:614] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:32.984277  2317 leader_election.cc:336] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.571s (SENT)
I20260709 13:01:32.929391  6168 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:32.985765  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:32.986289  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:33.018774  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 59754.2us
I20260709 13:01:33.019256  2350 raft_consensus.cc:2376] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10. Re-sending same reply.
I20260709 13:01:33.019882  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:32.988341  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:33.020519  1630 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:32.989246  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:32.993654  1561 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:32.996340  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 335361us
I20260709 13:01:33.022257  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:32.974216  5910 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.021023  1636 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:32.969556  5922 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:32.909967  6108 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.033437  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:33.033638  1554 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:33.027454  1557 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:32.974025  6164 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:01:33.038585  2346 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.038990  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:33.042825  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.043125  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.048794  6155 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:33.050012  6203 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:01:33.049711  6155 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.052037  6057 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:33.051891  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:33.053772  1561 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:33.053465  1560 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:33.059190  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:33.059940  1557 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:32.980520  6173 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.063851  1629 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:33.068811  1635 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:33.078480  1631 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:33.079134  2346 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
W20260709 13:01:33.062065  6169 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.871s	user 0.004s	sys 0.010s
I20260709 13:01:33.041371  5494 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.080509  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.082358  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:33.082382  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.095046  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.081059  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:33.061803  5427 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:33.071920  6057 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.052062  5907 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.088186  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:33.111338  1561 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:33.084467  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.090973  2345 raft_consensus.cc:2410] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:01:33.090744  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.080822  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.112991  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.113360  1632 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:33.114207  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:33.114468  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.114903  6214 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:33.115460  6214 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:33.115721  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.116015  5847 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:33.115628  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.095455  1705 raft_consensus.cc:2368] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14.
I20260709 13:01:33.116456  6214 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.115757  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:01:33.080013  6155 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:33.036151  5910 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.085979  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.067490  5830 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:33.027294  6063 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:33.139333  5830 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.141620  6063 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:33.112198  5494 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.143735  5494 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:33.103217  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.147312  1709 raft_consensus.cc:606] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.147826  1709 raft_consensus.cc:614] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:33.101578  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:33.108939  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:33.103278  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.148828  1560 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.117713  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.118934  6151 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.119938  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.154553  1634 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:33.122092  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:33.100297  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.124826  1630 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:01:33.099035  1627 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:33.052481  6168 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.093428  6176 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:33.126937  6162 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:33.175388  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.111189  5427 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.135346  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.135782  6208 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.137538  5910 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:33.150048  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 65821.2us
I20260709 13:01:33.149025  1706 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:33.166878  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.182061  1710 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:33.101155  6223 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.170712  1627 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:33.116505  5847 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:33.133744  6211 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:33.178030  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.178628  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:33.198732  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.210373  1708 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:33.210843  1632 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:33.214855  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:33.220489  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 85233.8us
I20260709 13:01:33.212913  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.221915  1706 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:33.172804  5494 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:33.225231  1634 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: [13.9-16.10]
I20260709 13:01:33.128822  5907 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.088802  6061 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.213258  6144 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:33.230408  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.227847  5494 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:33.234588  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.241048  1708 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:33.229091  1703 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:33.242188  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.242501  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:33.227643  6162 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:33.225214  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 224004us
I20260709 13:01:33.228528  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:33.257887  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.272182  5922 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:33.272332  6222 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:33.245723  5494 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.244917  6066 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:33.287622  6176 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.288173  6066 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:33.290642  5427 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:33.291730  1710 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:33.292289  6168 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:33.296466  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.254659  6076 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:33.286312  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.286259  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.277586  5847 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.288120  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.298548  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:33.281991  6210 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:01:33.280714  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:33.305444  1634 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:33.280018  1552 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:33.212726  6223 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:33.308327  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 80387.2us
W20260709 13:01:33.317334  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 67457.3us
I20260709 13:01:33.318162  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.316897  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.264330  6063 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:01:33.359736  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:33.368422  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 69976.3us
I20260709 13:01:33.270780  6144 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:01:33.369959  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:33.326614  5427 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:33.297300  5817 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:01:33.371384  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:33.394098  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 132745us
W20260709 13:01:33.369670  6171 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.749s	user 0.008s	sys 0.005s
I20260709 13:01:33.401393  1560 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
W20260709 13:01:33.406957  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 247148us
I20260709 13:01:33.356652  6201 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:01:33.404342  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44727.3us
I20260709 13:01:33.357879  6173 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:33.421140  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51322.2us
I20260709 13:01:33.410502  1675 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.412715  2318 leader_election.cc:400] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14.
W20260709 13:01:33.426373  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 203028us
I20260709 13:01:33.407168  6183 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:33.402143  5427 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:33.437558  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.442129  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.444832  1554 raft_consensus.cc:2410] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:33.449692  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.444914  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.436838  6144 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:33.464484  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.420228  6093 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:01:33.445350  6173 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:33.412514  6171 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:33.468326  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 58482.4us
I20260709 13:01:33.446676  6169 raft_consensus.cc:2764] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:33.471256  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.396065  6162 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.475445  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.444885  6151 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:33.475998  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 104697us
I20260709 13:01:33.477267  2317 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:33.484947  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.490381  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.470479  6214 raft_consensus.cc:2764] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:01:33.492940  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64370us
I20260709 13:01:33.492650  1675 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.475442  6177 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:33.474146  5817 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:33.503072  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.474390  6173 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:33.492326  6167 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:33.514955  5907 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:33.529803  6167 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.514123  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.530520  6167 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.531015  1705 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:33.467388  6218 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:01:33.516084  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.492463  6208 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:33.516615  1702 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:33.516896  6246 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:01:33.517699  6238 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:33.518105  6225 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:01:33.518322  5830 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:33.543259  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.520948  6151 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:33.498207  6144 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:33.491811  5559 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.552415  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.553367  1702 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:33.497925  6085 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:33.511583  6233 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:01:33.511267  5427 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.530391  5907 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:33.515887  6241 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:33.532202  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56866.9us
I20260709 13:01:33.564838  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:33.564985  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 87785.3us
I20260709 13:01:33.516420  6224 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:33.537914  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:33.540318  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 56883.4us
W20260709 13:01:33.518942  6228 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:01:33.549772  1706 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:33.555514  6208 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.507596  6093 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 13:01:33.545401  6163 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.556s	user 0.013s	sys 0.000s
I20260709 13:01:33.568077  6241 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:33.566007  6223 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:33.552542  6214 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.509433  6173 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.549798  6232 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:01:33.583339  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.585657  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.604524  1706 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:01:33.592099  1708 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
W20260709 13:01:33.569592  6244 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:01:33.596909  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.598278  6085 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:33.609160  1705 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.611460  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.621095  1708 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:33.610700  6085 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:33.632488  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.646915  6199 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.598430  6241 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:33.649952  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.607807  6151 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:33.650620  1708 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.603159  6163 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:33.602527  6214 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.654071  1705 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:33.653941  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.607539  6232 consensus_queue.cc:1243] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:33.624966  6057 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:33.608577  5494 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:33.657393  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 74171.9us
I20260709 13:01:33.666115  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.600136  6208 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.658843  6220 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:33.659564  6243 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:33.662272  6251 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.685209  6251 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.687755  6251 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:33.660612  6144 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.657919  6085 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.690201  6085 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.692633  6173 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.667876  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:33.649128  6182 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.882s	user 0.013s	sys 0.002s
I20260709 13:01:33.670513  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.700110  1705 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:33.674120  1707 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:01:33.682782  6208 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:33.683147  6214 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:33.702463  6182 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.703197  5494 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:33.705003  6182 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:33.705842  1556 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:33.652421  6199 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.711536  1705 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:33.684508  1702 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:33.692970  1629 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
W20260709 13:01:33.695544  6093 consensus_queue.cc:1175] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" responder_term: 11 status { last_received { term: 8 index: 5 } last_committed_idx: 5 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:01:33.713423  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45665us
I20260709 13:01:33.654920  5830 raft_consensus.cc:697] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:01:33.699448  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.716356  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.717020  1705 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:33.717332  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.671175  1708 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:01:33.701138  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 210959us
I20260709 13:01:33.701961  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:33.666713  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.714493  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.673288  6176 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:33.721853  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55237us
I20260709 13:01:33.722720  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.723739  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.724879  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.725602  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.726172  1702 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.726236  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.730870  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.732625  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.734126  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:33.734685  1705 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:33.717877  6214 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:33.736589  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.737514  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:33.718428  6061 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:33.769306  6061 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.771077  6173 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:33.771807  6061 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:33.771868  6217 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:33.772615  6217 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:33.772315  6167 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.774751  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 661) took 1121 ms. Trace:
I20260709 13:01:33.775009  6201 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:33.775231  1556 rpcz_store.cc:276] 0709 13:01:32.652989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.653123 (+   134us) service_pool.cc:224] Handling call
0709 13:01:33.774470 (+1121347us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b540038a9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:33.774647 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2297013120}
I20260709 13:01:33.744041  1708 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:01:33.777144  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.778024  1708 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:33.779022  6108 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.779731  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.735827  6254 raft_consensus.cc:2764] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:01:33.783524  1555 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:33.754417  6199 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:33.731690  5830 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.755517  6235 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:33.756714  2318 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.725067  6057 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:33.757710  6256 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:33.739049  6241 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.805287  6241 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:33.729724  6168 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:33.767177  6036 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:33.811020  6036 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.767866  6251 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:33.742702  6243 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.814473  6057 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:33.815863  6057 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.817322  1708 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:33.817602  6168 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:33.818133  6168 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:33.820017  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.820915  1708 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:33.742818  6182 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:33.775686  6201 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:33.821903  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.823141  6201 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.824884  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.828011  1708 raft_consensus.cc:2410] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:33.788599  1702 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:33.743813  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.825944  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:33.829648  1705 raft_consensus.cc:2368] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12.
I20260709 13:01:33.738982  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:33.795071  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 57698.5us
I20260709 13:01:33.795877  6078 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:33.796744  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.802402  6085 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:33.782859  6217 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.803072  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.791625  6235 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:33.793390  6254 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:33.833026  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:33.833962  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 95036.1us
I20260709 13:01:33.843087  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:33.836402  1708 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:33.825441  6220 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:33.838760  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:33.845468  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.847929  6176 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:33.848052  6217 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:33.850476  6257 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 13:01:33.850085  6235 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.851851  6173 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:33.852231  6235 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:33.852442  6173 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:33.853564  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:33.848146  1706 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
W20260709 13:01:33.868355  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 71686.7us
I20260709 13:01:33.869254  1702 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:33.853564  6173 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.873383  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:33.836797  6061 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:33.894551  6061 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:33.867617   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:33.895874  6061 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.889319  6168 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.898268   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:33.902510  6168 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:33.834295  6078 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:33.906454  6078 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.908134  6078 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.908869  6066 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:33.909322  6066 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:33.910527  6066 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:33.911686  6066 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:33.878453  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:33.880741  6059 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.930657  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:33.931010  1632 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:33.881067  1634 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:33.883054  6157 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.937959  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:33.854826  6085 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:33.887936  6171 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:33.879436  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:33.892990  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:33.893880  6036 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:33.941996  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 62631.4us
I20260709 13:01:33.942895  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:33.944062  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:33.899956  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:33.903235  6227 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.467s	user 0.000s	sys 0.003s
I20260709 13:01:33.838824  5817 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:33.945880  5817 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.946125  6211 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:33.946635  5817 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.945916  6076 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:33.948153  6076 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.948658  6076 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:33.877264  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:33.949921  1627 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.945350  1633 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:33.951995  6234 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:33.952683  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.842182  6167 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:33.946798  6211 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.935253  1675 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.954380  6211 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.957417  6167 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:33.958537  6167 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:33.960044  6167 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:33.940521  6085 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:33.963233  1601 leader_election.cc:400] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12.
I20260709 13:01:33.975376  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:33.988400  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:33.988933  2344 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:33.991371  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:33.876017  6176 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:33.991708  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:33.999719  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:34.001461  1706 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.994995  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:34.011917  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.012880  1554 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:34.014101  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.014921  1560 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.015763  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:34.012619  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:34.016902  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.028596  2347 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:34.029067  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.033354  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.033943  1627 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:34.038316  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.039955  1553 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:01:34.040593  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.040956  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.043679  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:34.044384  1553 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:34.004061  6036 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:33.978909  6155 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:01:34.047557  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:34.048543  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 56593.8us
W20260709 13:01:34.048979  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 55269.2us
I20260709 13:01:34.073053  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:34.049665  1556 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:34.056877  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.048264  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:34.083333  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:34.015358  6144 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:33.985730  6157 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.026806  6256 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.030485  6063 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.090803  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.126487  1552 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:34.111097  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.128089  1553 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.122225  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:34.152336  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 104138us
W20260709 13:01:34.159890  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 86972.2us
I20260709 13:01:34.103075  6222 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:34.091662  6202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.034s	user 0.041s	sys 0.058s
I20260709 13:01:34.090041  6036 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:34.176595  1554 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:01:34.181604  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:34.185148  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:34.165944  5559 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.609s	user 0.013s	sys 0.001s
I20260709 13:01:34.190387  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:34.125229  5865 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:34.086378  5427 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:01:34.203022  1560 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:34.132437  6144 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:34.184700  6202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.131s	user 0.041s	sys 0.058s
I20260709 13:01:34.228832  1561 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:34.157905  6251 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:34.181092  6222 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.167255  6255 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:34.240658  1552 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:34.288707  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:34.289979  6177 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:34.290438  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.291410  1552 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:34.243551  6036 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.293095  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:34.245620  6212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.017s	sys 0.026s
I20260709 13:01:34.294165  1552 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:01:34.294418  6212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.017s	sys 0.027s
I20260709 13:01:34.250783  6222 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.188297  6176 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.295305  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:34.131141  6256 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.299386  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:34.300616  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:34.253324  6201 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.258306  6144 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.264665  6157 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:34.224596  6254 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:34.317819  1559 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:34.319429  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:34.302038  1552 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:34.275022  6251 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.162732  6063 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.278543  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:34.279842  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:34.337881  1709 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.284451  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2527) took 1111 ms. Trace:
I20260709 13:01:34.223268  5559 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:34.328629  1706 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:34.344086  1552 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:34.340677  1631 rpcz_store.cc:276] 0709 13:01:33.172575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.172798 (+   223us) service_pool.cc:224] Handling call
0709 13:01:33.206652 (+ 33854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:33.228397 (+ 21745us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:33.246315 (+ 17918us) log.cc:844] Serialized 14 byte log entry
0709 13:01:33.438648 (+192333us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:33.438789 (+   141us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:33.439069 (+   280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.282912 (+843843us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b18003293e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419
0709 13:01:34.284106 (+  1194us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b540017b954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:34.284170 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.284363 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15015,"prepare.queue_time_us":40717,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":1070,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":10242,"spinlock_wait_cycles":60728832,"thread_start_us":190764,"threads_started":1,"wal-append.queue_time_us":270450}
I20260709 13:01:34.328225  1553 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:34.339978  1702 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:34.353618  1552 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:01:34.357605  6263 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
W20260709 13:01:34.358506  6266 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.116s	user 0.002s	sys 0.002s
W20260709 13:01:34.338472  6264 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
W20260709 13:01:34.365895  6262 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.379s	user 0.001s	sys 0.000s
I20260709 13:01:34.366576  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2621) took 1224 ms. Trace:
I20260709 13:01:34.368570  1710 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:01:34.320777  6201 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.367183  1557 rpcz_store.cc:276] 0709 13:01:33.141825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.146878 (+  5053us) service_pool.cc:224] Handling call
0709 13:01:33.177814 (+ 30936us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:33.225927 (+ 48113us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:33.225962 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:33.237681 (+ 11719us) log.cc:844] Serialized 26 byte log entry
0709 13:01:33.766016 (+528335us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b180031c120. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:01:33.841090 (+ 75074us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:33.875351 (+ 34261us) log.cc:844] Serialized 14 byte log entry
0709 13:01:33.878800 (+  3449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.878898 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.366160 (+487262us) raft_consensus.cc:1685] finished
0709 13:01:34.366302 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.366498 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122943,"prepare.queue_time_us":26300,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":51542,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":4544,"spinlock_wait_cycles":2516864,"thread_start_us":74840,"threads_started":1,"wal-append.queue_time_us":800570}
I20260709 13:01:34.371888  6201 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:34.372538  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.373390  1557 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:34.374079  6251 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:34.376029  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:34.375828  6211 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.376636  5817 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.410185  1602 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:34.410543  1560 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:34.380249  1710 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:01:34.413574  1561 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:34.351446  6254 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:01:34.435171  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.424813  6276 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:34.402050  1709 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:34.454707  1602 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:34.383991  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:34.440785  6180 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.482s	user 0.014s	sys 0.023s
W20260709 13:01:34.397112  6217 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.000s
W20260709 13:01:34.379347  6241 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.000s
I20260709 13:01:34.407263  1556 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
W20260709 13:01:34.345227  6243 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.008s
W20260709 13:01:34.409755  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.449291  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.385779  6211 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:34.379809  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:34.471683  1554 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:34.435743  6157 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:34.454656  6108 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.665s	user 0.009s	sys 0.006s
W20260709 13:01:34.443538  6057 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.009s	sys 0.016s
I20260709 13:01:34.476759  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:34.460076  6076 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.005s
I20260709 13:01:34.483964  1557 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:01:34.475167  6180 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.513s	user 0.014s	sys 0.024s
I20260709 13:01:34.459013  6208 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:34.488737  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:34.488950  1701 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:34.490267  1553 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.472602  5559 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.471863  6274 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:34.490386  6157 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.483114  6243 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:34.498874  1602 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:34.489730  6057 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.009s	sys 0.016s
I20260709 13:01:34.526463  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:34.526737  1553 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:01:34.500753  6235 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.648s	user 0.013s	sys 0.005s
I20260709 13:01:34.493305  6076 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.492398  6208 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.518210  5817 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:34.544130  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:34.545457  1632 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:01:34.533300  6066 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.616s	user 0.014s	sys 0.006s
I20260709 13:01:34.500157  5559 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.553036  5559 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:34.553937  1556 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:34.554039  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2619) took 1048 ms. Trace:
W20260709 13:01:34.554201  6200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.599s	user 0.013s	sys 0.023s
W20260709 13:01:34.554716  6200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.599s	user 0.013s	sys 0.023s
I20260709 13:01:34.554463  1701 rpcz_store.cc:276] 0709 13:01:33.505788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.505926 (+   138us) service_pool.cc:224] Handling call
0709 13:01:34.553826 (+1047900us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b540028da54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:34.553972 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2194310656}
I20260709 13:01:34.556254  1701 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:34.521780  6274 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:34.558928  6274 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.560868  1557 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:01:34.561273  1557 pending_rounds.cc:85] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 7
I20260709 13:01:34.547955  6275 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:34.562654  1627 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:34.566416  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:34.567149  1633 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:34.567992  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:34.568620  1634 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:34.550770  1636 pending_rounds.cc:85] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 6
I20260709 13:01:34.570103  1636 pending_rounds.cc:107] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:01:34.570405  1636 raft_consensus.cc:2889] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:34.570912  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.572781  1559 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:34.576637  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 422675us
I20260709 13:01:34.576843  5910 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:34.578105  1631 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:01:34.580482  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 419529us
I20260709 13:01:34.582967  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:34.583830  6249 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.584293  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:34.584539  6249 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.588049  6249 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:34.588689  6249 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.591200  6063 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:34.591967  5865 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:34.592692  5865 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:34.593299  6157 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:34.533785  6168 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:34.593978  5865 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.591081  6238 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:34.597500  1701 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:01:34.601317  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:34.535560  6211 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:34.603119  1526 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:34.518978  6254 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.606917  6208 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:34.609320  1710 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:34.611792  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:34.613077  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:34.614746  1631 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:34.615769  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:34.632156  1557 pending_rounds.cc:107] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:01:34.632696  1557 raft_consensus.cc:2889] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:34.622256  1675 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:34.637892  1634 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:34.639189  1558 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:34.600123  6238 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:34.608485  6063 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:34.651278  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:34.654520  1633 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:34.662441  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:34.663168  1631 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:34.663255  2348 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:34.653851  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:34.667639  2346 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:01:34.668171  2346 pending_rounds.cc:85] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 7
I20260709 13:01:34.668439  2346 pending_rounds.cc:107] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 7.8
W20260709 13:01:34.668471  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46315.3us
I20260709 13:01:34.668818  2346 raft_consensus.cc:2889] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:34.670017  1675 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:34.671504  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:34.674222  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:34.630843  5494 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:34.680692  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:34.680084  2350 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:34.645932  6235 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.687917  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:34.648170  5559 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:34.689451  2351 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:34.649681  6180 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:34.655086  6238 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:34.688299  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:34.688699  1634 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:34.616659  6216 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:01:34.683560  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:34.703941  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:34.589264  6249 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.614034  6222 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.710450  1559 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:34.681957  6211 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:34.682421  6171 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:34.717301  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:34.723883  6275 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:34.724359  1701 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:34.724887  6275 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:34.725128  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:34.725427  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:34.726197  1701 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:34.726497  6275 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:34.727098  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:34.685565  6208 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:34.727814  1701 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:34.691205  5910 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:34.731169  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2011) took 1036 ms. Trace:
I20260709 13:01:34.701089  6180 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.732038  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:34.733525  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:34.732544  1709 rpcz_store.cc:276] 0709 13:01:33.694870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.455135 (+760265us) service_pool.cc:224] Handling call
0709 13:01:34.455596 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.619093 (+163497us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:34.619479 (+   386us) log.cc:844] Serialized 14 byte log entry
0709 13:01:34.730338 (+110859us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:34.730420 (+    82us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:34.730494 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.730606 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.731032 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":179230,"prepare.queue_time_us":759,"raft.queue_time_us":153824,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":4024,"spinlock_wait_cycles":14844160,"thread_start_us":109163,"threads_started":1}
I20260709 13:01:34.705994  6200 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:34.709976  5427 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:34.736573  1706 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:34.737655  6180 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:34.749743  6272 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:34.750367  6272 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.750757  6272 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:34.732167  6211 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:34.754772  6288 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:01:34.745129  1559 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:34.754473  6180 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.756421  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2617) took 1252 ms. Trace:
I20260709 13:01:34.757298  6076 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:34.762074  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 81517.7us
W20260709 13:01:34.756809  6287 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:01:34.700481  6235 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:34.807122  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:34.756862  1704 rpcz_store.cc:276] 0709 13:01:33.503796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.504055 (+   259us) service_pool.cc:224] Handling call
0709 13:01:33.532117 (+ 28062us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:33.550129 (+ 18012us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:33.563251 (+ 13122us) log.cc:844] Serialized 14 byte log entry
0709 13:01:33.815906 (+252655us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b1800062500. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:33.892239 (+ 76333us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b1800062500. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:01:34.576086 (+683847us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b1800062500. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:01:34.652580 (+ 76494us) log.cc:844] Serialized 14 byte log entry
0709 13:01:34.675834 (+ 23254us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:34.676397 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:01:34.685843 (+  9446us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:34.685917 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.686034 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.756169 (+ 70135us) raft_consensus.cc:1685] finished
0709 13:01:34.756194 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.756338 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141415,"prepare.queue_time_us":25325,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":100,"raft.queue_time_us":594269,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":3377,"spinlock_wait_cycles":34389760,"thread_start_us":210101,"threads_started":2,"wal-append.queue_time_us":589888}
I20260709 13:01:34.813087  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:34.734689  6216 consensus_queue.cc:1243] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:34.824501  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:34.737680  5910 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:34.806376  5427 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:34.842185  1706 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:34.736549  6063 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 13:01:34.843794  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:34.756901  6211 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:34.823390  6057 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:34.847311  1704 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:34.833974  6076 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:34.810935  6061 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:01:34.814636  6200 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:01:34.852972  6076 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:34.854624  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:34.856307  1706 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:34.827574  6201 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:34.823068  6275 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 13:01:34.862578  6285 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.134s	user 0.001s	sys 0.001s
I20260709 13:01:34.862926  6076 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:34.853091  6290 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.185s	user 0.002s	sys 0.000s
I20260709 13:01:34.828059  6144 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:01:34.855357  6180 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:34.889844  1706 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:01:34.853615  6278 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:01:34.899154  1559 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:34.900348  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:34.901448  1561 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:34.903652  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:34.904445  1706 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:34.914389  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2006) took 1243 ms. Trace:
I20260709 13:01:34.914841  1710 rpcz_store.cc:276] 0709 13:01:33.670536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.412221 (+741685us) service_pool.cc:224] Handling call
0709 13:01:34.428535 (+ 16314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.609787 (+181252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.914111 (+304324us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400078354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:34.914320 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415227648}
I20260709 13:01:34.865991  6201 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.914646  6277 consensus_queue.cc:1059] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 13:01:34.915073  5427 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.847848  5910 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.916638   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:34.941666  1706 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:34.916561  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:34.937996  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 113581us
W20260709 13:01:34.918349  6293 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:01:34.940014  1710 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:01:34.914191  6144 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:34.921893  6281 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 13:01:34.979959  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 769) took 1103 ms. Trace:
I20260709 13:01:34.980762  1706 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:01:34.921332  6296 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:01:34.930110  6235 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:34.940819  6222 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:34.981640  6144 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.927040  6238 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:34.931718  6234 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:34.983402  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:34.938885  6294 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:01:35.006858  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2613) took 2019 ms. Trace:
I20260709 13:01:35.015067  1708 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:34.916002  6200 consensus_queue.cc:1243] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:34.990657  1710 rpcz_store.cc:276] 0709 13:01:33.876413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.916253 (+1039840us) service_pool.cc:224] Handling call
0709 13:01:34.939709 (+ 23456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:34.979563 (+ 39854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.979876 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:35.024664  1555 rpcz_store.cc:276] 0709 13:01:32.987718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.987859 (+   141us) service_pool.cc:224] Handling call
0709 13:01:33.783333 (+795474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:33.783948 (+   615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.006643 (+1222695us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b540038a9d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:35.006790 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1768687872}
I20260709 13:01:35.015468  1559 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:35.033309  1552 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:35.016614  1706 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:01:34.999382  6302 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:01:35.037055  1704 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:35.003801  5847 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:35.007251  6312 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:35.012219  6144 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:35.000955  6201 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:35.014678  6311 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:34.993528  6057 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:35.014838  6061 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:35.063825  1701 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:35.016137  6234 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:35.042553  1710 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:35.044472  1555 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:01:35.012915  6168 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:35.046233  1706 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:35.068573  1706 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:35.070227  1706 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:35.065608  6180 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.071552  6180 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.066248  1555 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:35.074576  1555 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.075019  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:35.076160  1555 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:35.079963  6180 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:35.048280  6238 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:35.058049  1703 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:35.058713  5847 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:35.038259  6309 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:01:35.059556  6176 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:35.047108  1561 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:01:35.043648  6312 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:35.086488  1706 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:35.023376  6272 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.060642  6311 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:35.087998  6163 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.094550  6303 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:35.100355  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 123825us
I20260709 13:01:35.096302  1710 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:35.063961  6061 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:35.110780  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:35.105989  6163 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.115223  6061 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.116539  6061 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:35.116871  6272 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:35.110944  6303 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.107662  6176 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.120108  6303 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:35.122010  6311 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.122653  6308 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:35.123207  6308 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:35.124264  6308 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.126217  6308 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:35.122035  6057 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:35.127363  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 669) took 1095 ms. Trace:
I20260709 13:01:35.127815  1560 rpcz_store.cc:276] 0709 13:01:34.031884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.236372 (+204488us) service_pool.cc:224] Handling call
0709 13:01:34.236859 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.410981 (+174122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.127059 (+716078us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400313654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:35.127287 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398940032}
I20260709 13:01:35.123395  6311 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.132076  6163 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:35.132205  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2014) took 1263 ms. Trace:
I20260709 13:01:35.132823  1701 rpcz_store.cc:276] 0709 13:01:33.868894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.728701 (+859807us) service_pool.cc:224] Handling call
0709 13:01:34.729190 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.064646 (+335456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.131901 (+ 67255us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b540014dcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:35.132133 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768968960}
I20260709 13:01:35.134194  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.134213  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:35.140308  1555 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:01:35.129974  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:35.142653  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.142580  1707 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:35.142155  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 483957us
I20260709 13:01:35.151768  6155 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:35.152165  6036 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:35.153721  5494 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:35.154438  5494 consensus_queue.cc:1243] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:35.154583  6144 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.156996  6144 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:35.165913  2350 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:35.167934  1701 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:35.097661  5847 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.177249  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:35.183532  2346 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:35.181995  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:35.186143  2350 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:35.143939  6243 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.141709  6176 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:35.202687  1707 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:35.204934  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:35.185935  6063 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.142037  6163 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:01:35.214627  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:35.217231  5847 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:35.206971  6155 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:35.206115  6243 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.224718  6155 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.225917  6243 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:35.233282  2347 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEARNER]: Advancing to term 10
I20260709 13:01:35.209653  6063 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.233732  6168 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:01:35.248505  6063 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:35.235118  6267 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.030s	user 0.004s	sys 0.000s
I20260709 13:01:35.250383  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2620) took 1744 ms. Trace:
I20260709 13:01:35.250790  1705 rpcz_store.cc:276] 0709 13:01:33.506146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.832056 (+325910us) service_pool.cc:224] Handling call
0709 13:01:33.832420 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:33.834849 (+  2429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:33.834879 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:33.841835 (+  6956us) log.cc:844] Serialized 26 byte log entry
0709 13:01:33.984071 (+142236us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:33.984134 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.984238 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:35.234404 (+1250166us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:35.250060 (+ 15656us) raft_consensus.cc:1685] finished
0709 13:01:35.250084 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.250316 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2045,"raft.queue_time_us":3622,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":136,"spinlock_wait_cycles":14386560,"thread_start_us":93015,"threads_started":1,"wal-append.queue_time_us":195097}
W20260709 13:01:35.233213  6292 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.525s	user 0.000s	sys 0.003s
I20260709 13:01:35.233342  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:35.254328  2317 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:35.256004  6157 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:35.256497  6157 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:35.256901  6157 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:35.259451  6234 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.263154  1702 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:01:35.263337  6222 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:35.263804  6222 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.263912  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2001) took 1613 ms. Trace:
I20260709 13:01:35.264205  6222 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.264307  1702 rpcz_store.cc:276] 0709 13:01:33.650395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.262429 (+1612034us) service_pool.cc:224] Handling call
0709 13:01:35.262897 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.263585 (+   688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.263845 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:35.265715  6243 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:35.266618  6254 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:35.248807  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:35.275539  2347 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:35.287124  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:35.283489  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.290086  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.288105  1702 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:35.270979  6078 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.288569  6251 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:35.306885  6251 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:35.308362  6251 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.298004  6224 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:35.318710  6224 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.319334  6224 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:35.298105  6059 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:35.323503  6059 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.324070  6059 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.323733  6157 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.286936  6063 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:01:35.303642  6278 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:01:35.334125  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2061) took 1058 ms. Trace:
I20260709 13:01:35.310705  6078 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.316639  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 763) took 2058 ms. Trace:
I20260709 13:01:35.344054  1703 rpcz_store.cc:276] 0709 13:01:33.258049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.261590 (+  3541us) service_pool.cc:224] Handling call
0709 13:01:35.316349 (+2054759us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b5400250dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:35.316547 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4030408192}
I20260709 13:01:35.299815  6155 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.334730  1552 rpcz_store.cc:276] 0709 13:01:34.275710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.354600 (+ 78890us) service_pool.cc:224] Handling call
0709 13:01:34.354906 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.035416 (+680510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.333730 (+298314us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b54001fd454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:35.334026 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1557649792}
I20260709 13:01:35.297441  6234 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:35.359738  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:35.364763  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:35.365439  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:35.366046  1705 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.370364  1556 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:35.370474  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:35.371315  1557 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.375615  1555 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:01:35.380347  1703 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:01:35.380841  1703 pending_rounds.cc:85] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 6
I20260709 13:01:35.381075  1703 pending_rounds.cc:107] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:01:35.381330  1703 raft_consensus.cc:2889] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:35.387437  1705 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:35.389004  1557 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:35.394537  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:35.385407  6155 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.419504  6215 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:35.420042  6215 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:35.423215  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.424691  2318 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.386373  6251 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:35.422259  6326 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:01:35.430714  6326 consensus_queue.cc:1243] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:35.399313  6254 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.425951  6155 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:35.427953  6063 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.442212  6063 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.446357  6063 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:35.432915  6066 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:35.448374  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:35.390543  5946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.479s	user 0.034s	sys 0.029s
I20260709 13:01:35.449103  1703 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:35.429126  6164 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:35.454335  6078 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:35.456637  6326 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:01:35.457065  6157 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:35.450172  6066 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.458982  6066 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:35.434840  6254 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.465790  6254 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:35.465987  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2053) took 1438 ms. Trace:
I20260709 13:01:35.466415  1554 rpcz_store.cc:276] 0709 13:01:34.027843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.202487 (+174644us) service_pool.cc:224] Handling call
0709 13:01:34.202939 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.486022 (+283083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.465725 (+979703us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54003309d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:35.465920 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617474304}
W20260709 13:01:35.468693  5946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.583s	user 0.034s	sys 0.029s
I20260709 13:01:35.466439  6168 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:01:35.451249  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.471750  6215 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:35.473879  6085 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:01:35.475225  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 768) took 1599 ms. Trace:
I20260709 13:01:35.475594  1704 rpcz_store.cc:276] 0709 13:01:33.876020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.902521 (+1026501us) service_pool.cc:224] Handling call
0709 13:01:34.921875 (+ 19354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.037697 (+115822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.474998 (+437301us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400336b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:35.475162 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264180480}
I20260709 13:01:35.477545  6168 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:35.478935  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:35.480305  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.486935  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:35.487777  6256 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:35.488077  6326 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:35.488739  6326 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:35.489573  6249 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:35.489843  6326 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.492040  6085 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:35.492658  6085 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:35.493758  6085 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.494771  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:35.495435  1554 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:35.498118  6274 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:35.498265  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.498831  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:35.499437  6274 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.499939  6274 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.500907  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:35.501649  1634 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:01:35.501837  6198 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:35.503139  6220 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:35.503599  1552 pending_rounds.cc:85] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 9
I20260709 13:01:35.504015  1552 pending_rounds.cc:107] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:01:35.503724  6216 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:35.504395  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:35.505012  1561 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:35.504868  6220 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:35.511126  1635 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:35.588713  6220 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:35.589157  6220 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:35.589622  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.590134  6220 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.591217  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.507429  6078 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:35.593207  1557 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:35.593017  6078 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.593844  6220 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:35.595512  6078 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:35.506856  6168 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:35.597025  6168 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.511126  1630 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:35.601683  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:35.602944  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.498296  1556 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:35.512478  6235 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:35.512720  6217 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:35.513208  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:35.619947  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 133079us
I20260709 13:01:35.514017  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.504024  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:35.515251  1628 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
W20260709 13:01:35.515682  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 405712us
I20260709 13:01:35.515777  6200 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:35.516270  6317 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:35.516505  6276 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:35.516919  6144 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:35.518190  6211 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:35.518977  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.519603  6272 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:35.507225  1629 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:35.525185  6157 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.507187  5494 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:35.503317  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:35.526270  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.527530  2346 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:35.527877  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:35.528213  2345 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:35.529654  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2050) took 1518 ms. Trace:
I20260709 13:01:35.530544  6243 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:35.530659  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:35.504405  1552 raft_consensus.cc:2889] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:35.680248  1558 rpcz_store.cc:276] 0709 13:01:34.011007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.016483 (+  5476us) service_pool.cc:224] Handling call
0709 13:01:35.529401 (+1512918us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b5400207454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:35.529585 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009433984}
I20260709 13:01:35.544592  6274 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:35.546949  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.553222  6215 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.554482  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:35.509987  6324 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:01:35.558768  6254 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:35.574373  6176 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.587163  1555 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:35.678701  6217 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:35.621333  1628 pending_rounds.cc:85] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 8
I20260709 13:01:35.705559  1628 pending_rounds.cc:107] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:01:35.705916  1628 raft_consensus.cc:2889] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:35.631004  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.630651  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:35.631825  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 106804us
W20260709 13:01:35.708794  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 78237.1us
I20260709 13:01:35.650755  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:35.654136  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.672451  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 153577us
I20260709 13:01:35.678764  2348 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:35.681244  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.506142  6303 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:35.651859  6078 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:35.711942  1526 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.481631  6222 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:35.696981  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 150151us
I20260709 13:01:35.697497  1602 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:35.685384  6243 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:35.662477  6200 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.662851  6317 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.678550  5494 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:35.664484  6307 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:01:35.605420  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.662860  6061 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:01:35.718665  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.682688  6157 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.721874  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:35.728318  2346 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:35.728691  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.729259  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:35.777923  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48757.7us
I20260709 13:01:35.780588  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:35.729357  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.732561  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.664258  6235 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.739488  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.787711  6235 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:35.789183  1455 catalog_manager.cc:5697] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:01:35.791498  1455 catalog_manager.cc:5697] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 7 to 9, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:01:35.740304  6059 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.739965  6310 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:01:35.739813  6275 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:35.755749  5559 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:01:35.754194  6217 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:35.766885  6076 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:01:35.739537  6171 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:01:35.760018  6308 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:01:35.717996  6274 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:35.794960  6243 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.808228  6243 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:35.762456  6277 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:01:35.755520  6182 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:35.812208  6182 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.812697  6182 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:35.754427  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.816428  6061 consensus_queue.cc:1243] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:01:35.817138  6061 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:35.817531  6061 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:35.818476  6061 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.819857  6061 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.793445  1455 catalog_manager.cc:5697] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 8 to 9, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 9 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:01:35.823359  6157 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:35.824319  6233 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:35.827574  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:35.828269  1630 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:35.829325  1455 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 7 to 8, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 8 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:35.830850  6254 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:35.729707  6164 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:35.832032  6312 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.832650  6164 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.679997  6211 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.704368  6155 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:35.837052  6155 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.837584  6155 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.838557  6211 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:35.839576  6200 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:35.839733  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.840795  1627 raft_consensus.cc:2410] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:35.719534  1633 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:35.771270  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:35.842029  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 71044us
I20260709 13:01:35.842504  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:35.844550  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 135039us
I20260709 13:01:35.716035  6303 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:35.774842  1458 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 271) took 22494 ms (client timeout 15000 ms). Trace:
I20260709 13:01:35.781993  1632 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:35.813935  1631 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:35.847113  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.797935  5494 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.848181  1632 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:35.848659  5494 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.849301  1632 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:35.849462  1631 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:35.850085  6260 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.185s
I20260709 13:01:35.850669  1632 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:35.851985  1632 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:35.800693  6275 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.853530  6275 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:35.853287  1632 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:35.855419  6256 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:35.802349  6217 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:35.856616  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 117225us
I20260709 13:01:35.803740  6222 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:35.798053  6224 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.860277  6272 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:35.860852  5559 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:35.861299  5559 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.861135  6198 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:35.861709  5559 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:35.802007  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:35.862696  1628 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:35.863502  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:35.864117  2345 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:35.866191  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:35.867591  1631 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:01:35.873364  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 92890.9us
I20260709 13:01:35.879278  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:35.797159  6317 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:35.879565  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:35.891173  1627 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:35.886813  6307 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:35.796931  6059 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:35.892613  6307 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.893288  6307 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.887831  6235 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:01:35.896256  6235 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:35.896673  6235 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:35.897104  6235 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:35.899317  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:35.900663  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.902729  1628 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:01:35.858251  6222 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:35.904304  1631 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:35.904698  1631 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:35.875950  6312 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:35.909801  6177 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:01:35.910370  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:35.911966  6198 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:35.846371  1458 rpcz_store.cc:269] 0709 13:01:13.272997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.307947 (+ 34950us) service_pool.cc:224] Handling call
0709 13:01:35.766975 (+22459028us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:33.117673 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:33.118382 (+   709us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:33.118444 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:01:33.118464 (+    20us) tablet.cc:662] Decoding operations
0709 13:01:33.119743 (+  1279us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:33.119840 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:33.119868 (+    28us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:01:33.121114 (+  1246us) tablet.cc:701] Row locks acquired
0709 13:01:33.121164 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:01:33.121361 (+   197us) write_op.cc:270] Start()
0709 13:01:33.145873 (+ 24512us) write_op.cc:276] Timestamp: P: 1783602093121289 usec, L: 0
0709 13:01:33.171602 (+ 25729us) op_driver.cc:348] REPLICATION: starting
0709 13:01:33.281828 (+110226us) log.cc:844] Serialized 4761 byte log entry
0709 13:01:33.782861 (+501033us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda245d00ac 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b
0709 13:01:34.286611 (+503750us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:34.729670 (+443059us) op_driver.cc:464] REPLICATION: finished
0709 13:01:34.874339 (+144669us) write_op.cc:301] APPLY: starting
0709 13:01:34.874511 (+   172us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:35.126464 (+251953us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:35.126511 (+    47us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:35.140130 (+ 13619us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:35.140504 (+   374us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 13:01:35.140569 (+    65us) write_op.cc:312] APPLY: finished
0709 13:01:35.140960 (+   391us) log.cc:844] Serialized 113 byte log entry
0709 13:01:35.175374 (+ 34414us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:35.189330 (+ 13956us) write_op.cc:454] Released schema lock
0709 13:01:35.197362 (+  8032us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:35.208872 (+ 11510us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:35.209755 (+   883us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":155546,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":138573,"apply.run_cpu_time_us":20268,"apply.run_wall_time_us":323576,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54287,"num_ops":12,"prepare.queue_time_us":214929,"prepare.run_cpu_time_us":51491,"prepare.run_wall_time_us":1177582,"raft.queue_time_us":307449,"raft.run_cpu_time_us":16452,"raft.run_wall_time_us":194500,"replication_time_us":1519062,"spinlock_wait_cycles":163763328,"thread_start_us":1346561,"threads_started":4,"wal-append.queue_time_us":918055}]]}
I20260709 13:01:35.936347  1628 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:35.936769  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:35.913583  6317 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:35.913985  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.940377  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:35.905421  1631 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:35.775170  6201 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.938198  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:35.707526  6176 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.941912  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:35.943113  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 166943us
I20260709 13:01:35.832166  1455 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 13 to 16, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:35.947011  1674 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:35.948041  1455 catalog_manager.cc:5697] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 8 to 9, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 9 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:01:35.915625  1633 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:35.921310  1630 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:35.951320  1633 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:01:35.951694  1633 pending_rounds.cc:85] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 6
I20260709 13:01:35.951920  1633 pending_rounds.cc:107] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:35.952148  1633 raft_consensus.cc:2889] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:35.952445  1556 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:35.953951  1631 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:35.953837  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:35.954890  1675 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:35.956357  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:35.943771  6216 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:35.943923  6059 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:35.987092  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:35.970158  1558 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:35.971421  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:35.800851  6066 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:35.934713  6198 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:35.989760  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:36.006111  1706 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:35.934288  6199 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:01:36.007036  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.991859  1705 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:01:35.991079  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:35.994118  1707 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:35.993165  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:35.995074  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:36.000924  1559 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:36.010032  1702 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:35.978236  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:35.917683  6222 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:36.006204  1455 catalog_manager.cc:5697] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:01:35.994146  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:36.012562  1455 catalog_manager.cc:5697] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.019161  1631 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:01:36.022590  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.022941  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:36.022694  1703 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:36.023864  1708 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
W20260709 13:01:36.003509  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 56622.1us
W20260709 13:01:36.024761  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 494164us
W20260709 13:01:36.029136  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 57873.5us
I20260709 13:01:36.040208  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:36.040298  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:36.035770  2317 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:35.939539  6308 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:01:36.046892  2317 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:35.939851  6277 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:01:35.981719  6317 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:36.056052  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:36.048607  2348 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:01:36.054502  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 99714.4us
I20260709 13:01:35.949385  6274 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:01:36.027586  1455 catalog_manager.cc:5697] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 6 to 8, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 8 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.062860  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:36.067868  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:36.069535  1706 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:01:36.070391  1702 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:01:36.072309  1704 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:01:36.073676  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.075891  2344 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEARNER]: Advancing to term 12
I20260709 13:01:36.100152  2349 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:36.101678  1708 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:36.103461  1455 catalog_manager.cc:5697] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:36.022590  6222 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:36.107148  1706 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:35.979852  6278 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:01:36.044914  6224 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:36.137048  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.116494  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.970482  6162 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:01:35.987578  6063 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:01:35.913785  6144 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.002632  6208 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:01:35.969838  6201 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.055133  6309 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:36.053390  6251 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:36.003078  6249 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 13:01:36.258285   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:36.156097  1707 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:01:36.114770  1455 catalog_manager.cc:5697] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 4 to 7, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.053977  6059 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.053614  6199 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:36.077462  6157 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:36.077812  5847 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.056180  6299 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:36.165727  6328 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:36.165921  6303 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:36.171576  1633 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:36.323283  1633 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:36.177733  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55125.9us
I20260709 13:01:36.177207  6317 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:36.183390  6272 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:01:36.185637  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.186803  6327 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:01:36.194093  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:36.310292  6157 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:36.198804  6332 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:01:36.205312  6330 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:01:36.205197  6198 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:36.161711  6171 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:01:36.115898  6238 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:36.063328  6200 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.216s
I20260709 13:01:36.161595  6278 consensus_queue.cc:1243] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:36.290359  1455 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 9 to 10, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 10 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:36.412919  2351 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.005s	sys 0.023s
W20260709 13:01:36.413610  2351 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.006s	sys 0.023s
I20260709 13:01:36.414327  1455 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 8 to 9, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 9 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:36.327721  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:36.324597  1633 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:36.417971  2353 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:36.418205  1633 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.248862  6251 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:36.158660  5910 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.160s
I20260709 13:01:36.291517  6066 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:36.002703  6319 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 13:01:36.341073  6335 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:01:36.349025  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.442458  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.325188  6078 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.017s	sys 0.008s
I20260709 13:01:36.320523  6299 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:36.378414  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.349601  6168 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.021s	sys 0.044s
I20260709 13:01:36.310570  6059 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.359052  6157 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:36.355763  6303 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:36.455178  6078 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.018s	sys 0.008s
I20260709 13:01:36.416474  1455 catalog_manager.cc:5697] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.392988  5946 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.431921  6198 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:36.426126  1634 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.029s	sys 0.038s
I20260709 13:01:36.435719  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1227) took 1102 ms. Trace:
W20260709 13:01:36.525012  1634 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.029s	sys 0.038s
W20260709 13:01:36.476217  6168 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.021s	sys 0.045s
I20260709 13:01:36.324002  6164 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:01:36.436496  2347 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEARNER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:36.475762  6299 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:36.407658  6243 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.599s	user 0.010s	sys 0.003s
W20260709 13:01:36.469035  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.354573  6224 consensus_queue.cc:1175] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" responder_term: 10 status { last_received { term: 8 index: 5 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [9.6-9.6]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:01:36.477429  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 485428us
W20260709 13:01:36.477988  1627 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.010s	sys 0.007s
W20260709 13:01:36.553079  1627 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.010s	sys 0.007s
I20260709 13:01:36.480587  1633 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:01:36.553877  1633 pending_rounds.cc:85] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 6
I20260709 13:01:36.554275  1633 pending_rounds.cc:107] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:36.554808  1633 raft_consensus.cc:2889] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:36.555505  2344 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:36.436640  6251 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:36.441496  6182 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.628s	user 0.006s	sys 0.007s
I20260709 13:01:36.511678  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2749) took 1012 ms. Trace:
W20260709 13:01:36.359187  6326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.006s	sys 0.021s
I20260709 13:01:36.437671  6278 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:36.516119  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 458791us
I20260709 13:01:36.525125  2350 rpcz_store.cc:276] 0709 13:01:35.333792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.334004 (+   212us) service_pool.cc:224] Handling call
0709 13:01:35.342011 (+  8007us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.355716 (+ 13705us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:35.355753 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:35.356299 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 13:01:35.633105 (+276806us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:35.633179 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.633292 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:36.435405 (+802113us) raft_consensus.cc:1685] finished
0709 13:01:36.435434 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.435635 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20166,"raft.queue_time_us":20030,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":14340,"thread_start_us":276409,"threads_started":1,"wal-append.queue_time_us":542212}
I20260709 13:01:36.517603  6176 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:36.342980  2318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.007s	sys 0.014s
W20260709 13:01:36.571024  2318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.007s	sys 0.015s
W20260709 13:01:36.571436  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 949762us
I20260709 13:01:36.572674  2318 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:36.475903  6076 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.361188  6199 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:36.434450  6277 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:01:36.476444  6335 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:36.541231  5946 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:36.555394  1676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.943s (SENT) [suppressed 19 similar messages]
W20260709 13:01:36.502301  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.561156  1636 rpcz_store.cc:276] 0709 13:01:35.499724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.539932 (+ 40208us) service_pool.cc:224] Handling call
0709 13:01:35.540792 (+   860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.541532 (+   740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.541563 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:35.542224 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 13:01:35.731859 (+189635us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:35.731909 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.731990 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:36.511321 (+779331us) raft_consensus.cc:1685] finished
0709 13:01:36.511357 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.511577 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":188858,"threads_started":1,"wal-append.queue_time_us":459916}
I20260709 13:01:36.517952  6078 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:01:36.542941  6275 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.689s	user 0.006s	sys 0.010s
W20260709 13:01:36.527365  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 497329us
I20260709 13:01:36.530004  1455 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 7 to 10, now has a pending config: VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194), VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 10 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:36.581149  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1029) took 1054 ms. Trace:
W20260709 13:01:36.559633  6155 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.721s	user 0.012s	sys 0.001s
W20260709 13:01:36.554121  6220 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.960s	user 0.007s	sys 0.006s
I20260709 13:01:36.577040  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:36.570753  6276 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.011s	sys 0.012s
W20260709 13:01:36.566094  6211 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
W20260709 13:01:36.565750  6326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.006s	sys 0.022s
I20260709 13:01:36.608515  1455 catalog_manager.cc:5697] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 7 to 12, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 12 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.606614  2347 rpcz_store.cc:276] 0709 13:01:35.526236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.528506 (+  2270us) service_pool.cc:224] Handling call
0709 13:01:36.436225 (+907719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.528474 (+ 92249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.580795 (+ 52321us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b54009902d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:36.581034 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2086728576}
I20260709 13:01:36.611406  1553 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 13:01:36.575199  6215 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.876s	user 0.004s	sys 0.009s
W20260709 13:01:36.615839  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.613349  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 286878us
I20260709 13:01:36.623589  1455 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 5 to 9, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 9 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:36.625660  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.626024  1455 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.579574  6168 raft_consensus.cc:2764] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:36.629186  1455 catalog_manager.cc:5697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:36.634748  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.014s	sys 0.014s
W20260709 13:01:36.635315  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.014s	sys 0.014s
W20260709 13:01:36.620734  6235 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.719s	user 0.007s	sys 0.007s
I20260709 13:01:36.616255  6066 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:36.637722  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:36.638971  2349 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:36.654476  1636 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:36.575135  6309 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:01:36.648726  2347 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:01:36.616621  6061 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:36.654953  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.570192  6182 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:36.635748  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 566583us
I20260709 13:01:36.575778  6278 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:36.620450  6276 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.011s	sys 0.012s
I20260709 13:01:36.675552  1557 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 10. Ops: []
I20260709 13:01:36.660730  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:36.578610  6076 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:36.654796  1455 catalog_manager.cc:5697] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:01:36.666731  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:36.612593  6272 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:01:36.635378  6201 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:36.678807  1629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.015s	sys 0.019s
I20260709 13:01:36.616153  6275 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.621793  6155 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:36.663451  6176 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:36.662616  6061 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:36.677214  6307 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.783s	user 0.013s	sys 0.003s
I20260709 13:01:36.683054  2350 raft_consensus.cc:2410] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:01:36.683001  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:36.678151  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:36.684435  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.692418  6319 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:36.694257  6260 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:01:36.724624  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46591.7us
I20260709 13:01:36.727097  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:36.695478  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.700735  1629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.015s	sys 0.019s
I20260709 13:01:36.634336  6243 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:01:36.702484  1702 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:36.707758  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:36.733752  2347 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
W20260709 13:01:36.734150  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.734164  2347 pending_rounds.cc:85] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 10
I20260709 13:01:36.734750  2347 pending_rounds.cc:107] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:01:36.735025  2347 raft_consensus.cc:2889] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:36.734143  2353 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:36.675841  6238 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:36.654764  6235 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:36.704499  6063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.021s	sys 0.017s
I20260709 13:01:36.729367  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:36.741570  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:36.742020  6326 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.742852  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:36.818928  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 76184.7us
I20260709 13:01:36.824697  1705 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:36.754905  2353 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:36.750574  1701 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:36.840363  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:36.746362  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:36.906309  1702 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
W20260709 13:01:36.904040  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.744488  6063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.022s	sys 0.018s
I20260709 13:01:36.738976  6201 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:36.917774  1555 raft_consensus.cc:1217] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 5.6->[6.7-10.9]   Dedup: 6.7->[8.8-10.9]
I20260709 13:01:36.934863  1555 pending_rounds.cc:85] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 7
I20260709 13:01:36.935253  1555 pending_rounds.cc:107] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:01:36.935611  1555 raft_consensus.cc:2889] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:36.919063  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:36.744542  6312 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.805s	user 0.015s	sys 0.000s
I20260709 13:01:36.937178  1558 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:36.913657  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:36.737540  6220 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:36.942864  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 213564us
I20260709 13:01:36.950295  1635 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:36.832244  6345 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.818199  6164 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:36.953981  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:36.962844  1558 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:36.920576  6059 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:36.956502  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:36.904387  5494 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:36.726830  6307 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.924211  6337 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:36.925793  6215 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.736526  6176 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:36.967936  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:36.977540  1558 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:36.737620  6276 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.205s
I20260709 13:01:36.941965  6344 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.816808  6238 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.726490  6078 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.959227  5427 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.984596  1558 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:36.954594  6309 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:36.951004  6057 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.951615  6201 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:36.979496  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:36.986212  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:36.990347  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:36.998169  1559 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:36.998414  1554 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:36.914151  6326 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.973261  6217 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.978933  6319 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.728366  6335 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.977849  6337 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.962255  6312 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:01:37.019526  1706 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:01:37.019366  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 463967us
I20260709 13:01:36.956421  6220 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.974964  6308 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:01:36.978816  6344 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:36.965456  6345 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:36.976555  5494 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:37.047681  2353 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:36.947914  6061 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.971167  6164 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.983054  6215 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:37.039565  6337 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:37.038923  6319 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:37.009323  5427 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:36.995486  6066 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.051s
W20260709 13:01:37.052742  5559 consensus_meta.cc:287] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.190s	user 0.015s	sys 0.000s
I20260709 13:01:37.003036  6057 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:37.088613  6061 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:37.349541  6340 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.402s	user 0.000s	sys 0.001s
W20260709 13:01:37.349902  6338 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.404s	user 0.002s	sys 0.000s
I20260709 13:01:37.085330  5946 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:37.087924  6272 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:37.041354  6217 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:37.377682  6348 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:01:37.090606  6182 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:37.378327  6336 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.691s	user 0.000s	sys 0.002s
W20260709 13:01:37.380712  6342 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.691s	user 0.001s	sys 0.001s
I20260709 13:01:37.377655  6361 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.347015  6278 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:37.386022  6076 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:37.367475  6357 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.442s
W20260709 13:01:37.451998  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.045s	sys 0.056s
I20260709 13:01:37.385613  5559 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:37.404594  6260 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:01:37.381538  6368 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.405s
W20260709 13:01:37.513406  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.057s	sys 0.056s
W20260709 13:01:37.514074  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 569401us
I20260709 13:01:37.424695  6155 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:37.418262  6275 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:37.441234  6367 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.518399  1559 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: [9.10-11.11]
I20260709 13:01:37.450394  6059 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:37.378482  6365 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.357s
I20260709 13:01:37.449270  6182 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:37.475487  6328 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.003s	sys 0.011s
W20260709 13:01:37.542230  1630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.005s	sys 0.017s
W20260709 13:01:37.538427  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.530684  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2072) took 1594 ms. Trace:
I20260709 13:01:37.544092  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1036) took 1690 ms. Trace:
I20260709 13:01:37.425280  6272 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:37.524308  1704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.012s	sys 0.008s
W20260709 13:01:37.542764  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 455710us
W20260709 13:01:37.550789  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:37.550179  1630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.005s	sys 0.018s
W20260709 13:01:37.550941  1704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.013s	sys 0.008s
I20260709 13:01:37.550594  2345 rpcz_store.cc:276] 0709 13:01:35.868121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.887136 (+ 19015us) service_pool.cc:224] Handling call
0709 13:01:35.887570 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.948328 (+ 60758us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.948364 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:35.948913 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.503180 (+554267us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:36.503249 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.503362 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:37.543792 (+1040430us) raft_consensus.cc:1685] finished
0709 13:01:37.543816 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.544003 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53757,"raft.queue_time_us":54376,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":24003,"thread_start_us":480136,"threads_started":1,"wal-append.queue_time_us":699413}
W20260709 13:01:37.536586  6059 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:37.550383  1553 rpcz_store.cc:276] 0709 13:01:35.936127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.937391 (+  1264us) service_pool.cc:224] Handling call
0709 13:01:35.937771 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.623395 (+685624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.530348 (+906953us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b5400230854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:37.530606 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1548369664}
W20260709 13:01:37.557983  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.558157  1554 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:37.558934  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1037) took 1690 ms. Trace:
I20260709 13:01:37.570456  1553 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:01:37.573253  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.016s	user 0.005s	sys 0.011s
W20260709 13:01:37.555728  6351 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.563s	user 0.002s	sys 0.000s
W20260709 13:01:37.574800  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.017s	user 0.005s	sys 0.011s
I20260709 13:01:37.544857  6377 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:37.553668  6307 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:37.478889  6361 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:37.600939  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 221222us
W20260709 13:01:37.581354  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 1.02849e+06us
I20260709 13:01:37.563403  2351 rpcz_store.cc:276] 0709 13:01:35.868522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.868662 (+   140us) service_pool.cc:224] Handling call
0709 13:01:35.869148 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.911318 (+ 42170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.911352 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:35.911969 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.414162 (+502193us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:36.414223 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.414325 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:37.558625 (+1144300us) raft_consensus.cc:1685] finished
0709 13:01:37.558657 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.558849 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30289,"raft.queue_time_us":44957,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":3793,"thread_start_us":501567,"threads_started":1,"wal-append.queue_time_us":692906}
I20260709 13:01:37.553375  6278 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:37.440212  6201 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:37.545004  6182 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:37.596171  6366 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.539s	user 0.000s	sys 0.001s
I20260709 13:01:37.593997  6328 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.530575  6367 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:37.634173  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:37.638185  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.606950  6326 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:37.639693  6215 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:37.648123  6355 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.276s	user 0.000s	sys 0.002s
I20260709 13:01:37.649184  6379 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.617713  6238 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:37.649633  6353 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:01:37.649894  6379 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:37.652482  6344 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:37.653942  5559 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:37.598101  6377 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:37.617383  6220 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:37.658759  6377 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:37.658986  6361 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:37.645804  6260 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:37.659658  6260 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:37.639750  6365 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:01:37.661778  6078 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:37.662392  6078 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:37.663599  6222 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.516s	user 0.010s	sys 0.005s
I20260709 13:01:37.664143  6367 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:37.664100  6222 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:37.665087  6299 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.107s	user 0.010s	sys 0.002s
I20260709 13:01:37.610754  6155 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:01:37.660527  6260 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:37.620616  6278 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:37.611099  6176 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:37.675136  2348 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:37.709316  2319 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.002s	sys 0.003s
W20260709 13:01:37.709940  2319 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.002s	sys 0.003s
W20260709 13:01:37.711583  2318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.002s	sys 0.002s
W20260709 13:01:37.712109  2318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.002s	sys 0.002s
W20260709 13:01:37.660557  6251 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.089s	user 0.011s	sys 0.000s
W20260709 13:01:37.666632  5847 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.343s	user 0.011s	sys 0.002s
I20260709 13:01:37.718595  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:37.654321  6182 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:37.666061  6352 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.293s	user 0.000s	sys 0.002s
W20260709 13:01:37.671064  6369 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.100s	user 0.001s	sys 0.001s
I20260709 13:01:37.632390  6249 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:37.768263  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:37.665418  6078 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:37.671680  6345 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:37.645946  5427 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:37.779002  1632 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:37.670727  6272 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:37.797851  1632 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:37.701196  6171 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:37.810045  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:37.794267  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:37.745030  6260 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:37.716403  6198 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.175s	user 0.009s	sys 0.005s
I20260709 13:01:37.797806  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1041) took 1895 ms. Trace:
W20260709 13:01:37.720827  6063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.001s	sys 0.004s
W20260709 13:01:37.795992  2344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.011s	sys 0.004s
W20260709 13:01:37.809146  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.813704  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:37.794523  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:37.936558  1705 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:37.815793  1560 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:37.817337  1627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.018s	sys 0.001s
W20260709 13:01:37.938067  1627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.019s	sys 0.001s
W20260709 13:01:37.938226  2344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.017s	sys 0.005s
I20260709 13:01:37.938896  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 838) took 2262 ms. Trace:
I20260709 13:01:37.765534  6317 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.767410  6326 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:37.819756  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.002s	sys 0.015s
W20260709 13:01:37.945117  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.002s	sys 0.015s
W20260709 13:01:37.945457  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 992853us
I20260709 13:01:37.818887  2352 rpcz_store.cc:276] 0709 13:01:35.902610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.902752 (+   142us) service_pool.cc:224] Handling call
0709 13:01:35.939854 (+ 37102us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:35.969690 (+ 29836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.969727 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:36.004435 (+ 34708us) log.cc:844] Serialized 52 byte log entry
0709 13:01:36.563201 (+558766us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:36.563284 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.563408 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:37.797547 (+1234139us) raft_consensus.cc:1685] finished
0709 13:01:37.797580 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.797741 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33094,"raft.queue_time_us":38466,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":18059,"thread_start_us":456881,"threads_started":1,"wal-append.queue_time_us":786527}
I20260709 13:01:37.704572  6391 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.824383  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:37.778679  5847 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:37.819324  6171 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:37.819064  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:37.804143  6157 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.276s	user 0.015s	sys 0.000s
I20260709 13:01:37.766678  6319 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:37.891702  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 90595.7us
I20260709 13:01:37.816349  6379 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:37.818020  5559 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:37.815122  6303 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.291s	user 0.005s	sys 0.009s
I20260709 13:01:37.633523  6328 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:37.829694  6200 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.858397  6059 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:37.830068  6063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.001s	sys 0.004s
I20260709 13:01:37.815255  5427 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.777976  6299 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:37.777279  6251 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:37.815237  6220 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:37.830631  1707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.013s	sys 0.026s
I20260709 13:01:37.799068  6249 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:37.933116  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 841) took 2250 ms. Trace:
I20260709 13:01:38.069737  1629 rpcz_store.cc:276] 0709 13:01:35.682721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.902994 (+220273us) service_pool.cc:224] Handling call
0709 13:01:35.940398 (+ 37404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.988075 (+ 47677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.988113 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:35.988734 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.730021 (+741287us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:36.730071 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.730155 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:37.932856 (+1202701us) raft_consensus.cc:1685] finished
0709 13:01:37.932880 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.933036 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51489,"raft.queue_time_us":41282,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":9626,"thread_start_us":659643,"threads_started":1,"wal-append.queue_time_us":968974}
I20260709 13:01:38.072803  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:37.942967  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 769) took 1304 ms. Trace:
I20260709 13:01:37.871281  6368 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.857795  6144 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:37.832043  6386 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:37.894397  6344 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.939286  1627 rpcz_store.cc:276] 0709 13:01:35.676697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.897787 (+221090us) service_pool.cc:224] Handling call
0709 13:01:35.898163 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.938177 (+ 40014us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:36.557098 (+618921us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.938512 (+1381414us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.938589 (+    77us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:37.938634 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.938716 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.938842 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88137,"prepare.queue_time_us":737824,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":80738,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":11352,"thread_start_us":1972343,"threads_started":3,"wal-append.queue_time_us":883781}
I20260709 13:01:37.798817  6367 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:01:38.016536  6407 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:38.023869  6394 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
I20260709 13:01:37.999507  6317 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:38.026754  6370 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.437s	user 0.002s	sys 0.000s
I20260709 13:01:38.028255  6408 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.028620  6222 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:38.108460  6359 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.430s	user 0.000s	sys 0.002s
I20260709 13:01:38.108235  6408 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.050103  5559 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.110723  6251 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:37.945878  6155 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:38.111904  6155 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.113252  6317 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.114866  6155 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.001525  6307 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.097922  6344 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.115691  6307 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.038411  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:38.091113  6144 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.118733  6171 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.119737  6144 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.120069  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 850) took 1618 ms. Trace:
I20260709 13:01:37.999544  6272 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:38.053367  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 433445us
I20260709 13:01:38.058138  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:38.061384  1707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.027s	sys 0.047s
W20260709 13:01:38.062582  1708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.115s	user 0.026s	sys 0.038s
I20260709 13:01:38.028301  6278 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.073658  2344 rpcz_store.cc:276] 0709 13:01:36.635371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.635609 (+   238us) service_pool.cc:224] Handling call
0709 13:01:36.656757 (+ 21148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.657094 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:37.005250 (+348156us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.938833 (+933583us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.938890 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:37.938948 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.939016 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.939135 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":880228,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":1394,"raft.queue_time_us":90,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":58264,"thread_start_us":1254843,"threads_started":2,"wal-append.queue_time_us":891180}
I20260709 13:01:37.994416  6326 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.086304  6386 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:38.052129  6200 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.086189  6368 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.006923  6391 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.091856  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.048049  6157 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.147554  6200 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.148213  1627 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:38.148454  2345 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:38.091863  1706 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:38.149657  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:38.103381  6407 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.151005  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:38.151274  6328 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:38.059823  5427 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.152175  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.152663  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1230) took 2093 ms. Trace:
I20260709 13:01:38.152763  1627 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:38.152899  6407 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.153645  5427 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:38.154072  6164 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:38.107522  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2768) took 2162 ms. Trace:
I20260709 13:01:38.153111  2349 rpcz_store.cc:276] 0709 13:01:36.059196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.123656 (+ 64460us) service_pool.cc:224] Handling call
0709 13:01:36.124098 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.657664 (+533566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.152409 (+1494745us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b5400540e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:38.152602 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1176761728}
I20260709 13:01:38.009390  6276 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:01:38.149228  6368 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:37.991494  6353 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.120702  1633 rpcz_store.cc:276] 0709 13:01:36.501334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.108239 (+606905us) service_pool.cc:224] Handling call
0709 13:01:37.108705 (+   466us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:37.348700 (+239995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.348748 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:37.374825 (+ 26077us) log.cc:844] Serialized 52 byte log entry
0709 13:01:37.606580 (+231755us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:37.610940 (+  4360us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.611199 (+   259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.611292 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.119692 (+508400us) raft_consensus.cc:1685] finished
0709 13:01:38.119731 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.119971 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":239477,"prepare.queue_time_us":3825,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":258244,"raft.run_wall_time_us":6718,"spinlock_wait_cycles":40735872,"thread_start_us":187853,"threads_started":1,"wal-append.queue_time_us":273565}
I20260709 13:01:38.167863  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.168747  1701 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
W20260709 13:01:38.139246  1708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.192s	user 0.030s	sys 0.042s
I20260709 13:01:38.145416  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 682) took 1229 ms. Trace:
W20260709 13:01:38.103076  6406 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:01:38.139281  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 783) took 1151 ms. Trace:
I20260709 13:01:38.139307  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 846) took 2001 ms. Trace:
I20260709 13:01:38.083978  6249 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:38.160671  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.160671  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:38.100896  1703 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:38.162526  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:38.029650  6337 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.001s
W20260709 13:01:38.093547  6061 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.684s	user 0.013s	sys 0.001s
I20260709 13:01:38.125255  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.054829  6059 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.187830  6211 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.188526  6211 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.189762  6059 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:38.165701  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:38.190864  6211 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:38.173097  6353 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.163187  6164 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:38.194547  6164 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.110842  6408 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.155364  1628 rpcz_store.cc:276] 0709 13:01:35.945045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.956632 (+ 11587us) service_pool.cc:224] Handling call
0709 13:01:35.956985 (+   353us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:36.182056 (+225071us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b48000318d8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:01:36.490088 (+308032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:36.490139 (+    51us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:36.491951 (+  1812us) log.cc:844] Serialized 78 byte log entry
0709 13:01:37.008507 (+516556us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:37.045787 (+ 37280us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.106887 (+ 61100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.106991 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.107146 (+1000155us) raft_consensus.cc:1685] finished
0709 13:01:38.107252 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.107429 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85445,"prepare.queue_time_us":37341,"prepare.run_wall_time_us":15981,"raft.queue_time_us":373762,"raft.run_cpu_time_us":6161,"raft.run_wall_time_us":145897,"spinlock_wait_cycles":20122240,"thread_start_us":890058,"threads_started":2,"wal-append.queue_time_us":974993}
I20260709 13:01:38.145543  6386 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:38.173655  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:38.170136  1555 rpcz_store.cc:276] 0709 13:01:36.916107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.916844 (+   737us) service_pool.cc:224] Handling call
0709 13:01:36.917406 (+   562us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:37.415558 (+498152us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:37.626696 (+211138us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.924514 (+297818us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:37.925922 (+  1408us) log.cc:844] Serialized 52 byte log entry
0709 13:01:37.926124 (+   202us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:37.926431 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.933139 (+  6708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.933239 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.145122 (+211883us) raft_consensus.cc:1685] finished
0709 13:01:38.145149 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.145340 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79047,"prepare.queue_time_us":235384,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":5662,"raft.queue_time_us":212518,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":67411,"spinlock_wait_cycles":24091392,"thread_start_us":792976,"threads_started":3,"wal-append.queue_time_us":437440}
I20260709 13:01:38.176064  1627 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:38.209005  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.210062  1627 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:38.172879  1707 rpcz_store.cc:276] 0709 13:01:36.988097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.988316 (+   219us) service_pool.cc:224] Handling call
0709 13:01:37.006850 (+ 18534us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:37.007323 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.007364 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:37.008121 (+   757us) log.cc:844] Serialized 52 byte log entry
0709 13:01:38.138929 (+1130808us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:38.138973 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.139044 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.139086 (+    42us) raft_consensus.cc:1685] finished
0709 13:01:38.139096 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.139214 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24628,"raft.queue_time_us":35990,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":37712,"thread_start_us":1052607,"threads_started":1,"wal-append.queue_time_us":994402}
I20260709 13:01:38.081349  6220 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.223695  6220 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:38.225431  6235 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.179889  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:38.227785  6299 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.228081  2345 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:38.199668  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2659) took 2199 ms. Trace:
I20260709 13:01:38.229619  1708 rpcz_store.cc:276] 0709 13:01:36.000219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.111260 (+111041us) service_pool.cc:224] Handling call
0709 13:01:36.133515 (+ 22255us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:36.159749 (+ 26234us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:36.732441 (+572692us) log.cc:844] Serialized 14 byte log entry
0709 13:01:38.180993 (+1448552us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:38.184779 (+  3786us) log.cc:844] Serialized 52 byte log entry
0709 13:01:38.184973 (+   194us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:38.185015 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.185149 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.199395 (+ 14246us) raft_consensus.cc:1685] finished
0709 13:01:38.199419 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.199603 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19984,"prepare.queue_time_us":801534,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":4059,"raft.queue_time_us":85641,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":9846,"thread_start_us":1648795,"threads_started":2,"wal-append.queue_time_us":1187603}
I20260709 13:01:38.232331  6353 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:38.234174  6164 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:38.235608  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.236603  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.210717  6408 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:38.238184  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1232) took 2163 ms. Trace:
I20260709 13:01:38.238602  2348 rpcz_store.cc:276] 0709 13:01:36.077905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.390813 (+312908us) service_pool.cc:224] Handling call
0709 13:01:36.391347 (+   534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.699908 (+1308561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.237979 (+538071us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540005bc54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:38.238131 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2951777664}
I20260709 13:01:38.238340  6408 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:38.243710  6334 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.833s	user 0.000s	sys 0.017s
I20260709 13:01:38.199501  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:38.201120  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.203262  1636 raft_consensus.cc:2410] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:38.166853  6256 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:38.203399  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.247113  1554 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:38.170971  1631 rpcz_store.cc:276] 0709 13:01:36.138224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.139616 (+  1392us) service_pool.cc:224] Handling call
0709 13:01:36.199565 (+ 59949us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:36.200210 (+   645us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:36.200258 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:36.201226 (+   968us) log.cc:844] Serialized 52 byte log entry
0709 13:01:36.636297 (+435071us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:36.636348 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.636430 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.138894 (+1502464us) raft_consensus.cc:1685] finished
0709 13:01:38.138928 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.139213 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3956,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"thread_start_us":396902,"threads_started":1,"wal-append.queue_time_us":1367806}
I20260709 13:01:38.170923  6307 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:38.171494  5494 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:38.256554  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.214322  2344 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:01:38.179847  6391 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.163244  6272 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:38.179056  6276 consensus_queue.cc:1243] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:38.218034  5559 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:38.171020  5847 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:01:38.218403  6217 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.197669  6337 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.220366  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2530) took 2711 ms. Trace:
I20260709 13:01:38.220767  6157 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:38.298170  2347 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:38.210703  6249 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.298983  6249 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.246071  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.301930  1627 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:38.303443  1627 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:01:38.305071  1627 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:38.211617  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:38.306953  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 95430.9us
I20260709 13:01:38.308919  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:38.213778  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.317994  1703 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:38.319247  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:38.256908  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:38.319694  6199 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:38.241214  6408 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:38.257791  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.320161  1705 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:38.274559  1633 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:38.321535  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:38.274256  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.323020  1631 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 8.
I20260709 13:01:38.282447  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2644) took 2765 ms. Trace:
I20260709 13:01:38.324275  1552 rpcz_store.cc:276] 0709 13:01:35.501739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.503004 (+  1265us) service_pool.cc:224] Handling call
0709 13:01:35.503393 (+   389us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:35.694299 (+190906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.694334 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:35.694995 (+   661us) log.cc:844] Serialized 52 byte log entry
0709 13:01:36.036578 (+341583us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:36.036627 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.036711 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.242906 (+2206195us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:38.266382 (+ 23476us) raft_consensus.cc:1685] finished
0709 13:01:38.266416 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.266745 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37006,"raft.queue_time_us":2611,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":11807,"spinlock_wait_cycles":80146304,"thread_start_us":308078,"threads_started":1,"wal-append.queue_time_us":629929}
I20260709 13:01:38.283982  1702 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:38.244714  6211 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.287297  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:38.281735  6201 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:01:38.292357  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:38.247180  6256 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:38.293660  6337 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:38.256155  6299 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.296274  6402 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:38.255401  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.253108  6319 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.251029  6061 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:01:38.252969  6353 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:38.312419  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.315891  6272 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:38.319625  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.366276  1555 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:38.331643  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:38.320245  6347 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:38.336086  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.336120  1554 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.287473  1634 rpcz_store.cc:276] 0709 13:01:35.508623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.521320 (+ 12697us) service_pool.cc:224] Handling call
0709 13:01:35.521691 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.827565 (+305874us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:36.525524 (+697959us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.942553 (+417029us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:36.942615 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:36.942724 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.715358 (+772634us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b18002b0000. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419
0709 13:01:38.220116 (+504758us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b5400196754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:38.220171 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.220303 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124837,"prepare.queue_time_us":807108,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":3041,"raft.queue_time_us":124520,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":7948,"spinlock_wait_cycles":719328512,"thread_start_us":989835,"threads_started":2,"wal-append.queue_time_us":454006}
I20260709 13:01:38.351531  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:38.352461  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:38.370072  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.353710  1705 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:38.373009  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:38.390898  1705 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.390504  6256 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.392336  6256 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:38.355126  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:38.352006  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 338440us
I20260709 13:01:38.393151  1701 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:38.301409  6249 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:38.363891  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.398190  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:38.398792  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.399111  6272 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:38.399523  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:38.400926  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:38.401438  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.403043  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.403611  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:38.404268  2351 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:38.348878  5847 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.405073  5847 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.406530  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:38.407371  2353 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:01:38.388068  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56553.8us
I20260709 13:01:38.407847  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.408046  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.338747  6307 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:38.408713  1553 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:38.367041  6402 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:38.353226  6211 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.415382  1701 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:38.399024  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.403461  6249 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:38.406620  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:38.386957  1702 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:38.409906  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:38.418180  1705 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:38.388753  6199 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:38.409909  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:38.389329  2344 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:38.407964  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:38.424275  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:38.408687  1552 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:38.413925  1633 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:38.354409  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.426769  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:38.415982  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.426826  1633 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:38.419296  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.421204  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.428267  1633 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.428957  1705 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:38.421350  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.429612  1633 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.430017  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.430716  1701 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:38.418861  6415 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:38.432724  1552 raft_consensus.cc:2410] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:01:38.432603  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.427985  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:38.434911  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:38.438596  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:38.439215  1702 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:38.440079  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:38.443220  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:38.448421  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.357299  6299 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.448669  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.453904  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.456041  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.448601  6059 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.457942  6059 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.458360  6059 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.459100  6299 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:38.459383  6402 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:38.459908  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:38.460058  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.418072  6220 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.463073  1633 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:01:38.419142  6272 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:38.463546  1633 pending_rounds.cc:85] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 6
I20260709 13:01:38.361341  6319 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.450037  1675 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:38.432907  6076 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:38.368248  6353 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.473640  1633 pending_rounds.cc:107] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:38.486311  1633 raft_consensus.cc:2889] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:38.452694  6278 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:38.438457  6216 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:38.452265  6368 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.487654  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.473752  1553 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:01:38.468498  6220 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.449528  6391 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:38.419575  6347 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:01:38.480587  1554 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:38.448756  6407 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:38.472805  6272 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:38.500588  1709 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:38.503924  1633 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:38.522964  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.523665  1633 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:38.461381  6415 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:38.528085  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.528978  1703 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:38.520612  1702 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:38.452527  6224 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:01:38.506879  1552 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:01:38.506090  6360 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.487s	user 0.002s	sys 0.002s
I20260709 13:01:38.525518  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.503185  6408 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.503499  6368 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.494521  6402 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.528391  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:38.494364  6216 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:38.506942  6391 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.494421  6278 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.507321  6220 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.554602  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.568740  1633 raft_consensus.cc:2410] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:38.549155  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.574062  1702 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:38.553804  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:38.548753  1676 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:38.664819  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.562057  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:38.665381  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 85388.7us
I20260709 13:01:38.665684  1553 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
W20260709 13:01:38.665794  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 117147us
I20260709 13:01:38.667150  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:38.665844  1709 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:38.667222  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:38.558166  6408 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.689545  6337 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:38.688884  6243 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:38.704414  6299 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:38.704703  6243 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:38.705204  6408 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.527859  6272 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:38.544171  6368 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.666960  6278 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:38.544332  6402 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.667089  6249 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:38.724498  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.562911  6216 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:38.534956  6376 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:38.519585  6307 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.156s
I20260709 13:01:38.665205  6391 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.670042  6303 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.732920  6168 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.705222  6243 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:38.707686  5494 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:38.744233  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.744485  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:38.745792  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:38.748894  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.748270  6249 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:38.666605  6144 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:01:38.757911  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:38.756857  6425 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.221s	user 0.002s	sys 0.000s
I20260709 13:01:38.737869  6402 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.733247  6328 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:01:38.798154  6408 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:01:38.758466  6303 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.798897  6420 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:01:38.766172  6085 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:38.802248  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2653) took 2807 ms. Trace:
I20260709 13:01:38.811870  1710 rpcz_store.cc:276] 0709 13:01:35.994853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.022941 (+ 28088us) service_pool.cc:224] Handling call
0709 13:01:36.023343 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.066439 (+ 43096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:36.066490 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:36.124764 (+ 58274us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.617952 (+493188us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:36.618015 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.618118 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.485381 (+1867263us) spinlock_profiling.cc:243] Waited 4.37 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:38.770074 (+284693us) raft_consensus.cc:1685] finished
0709 13:01:38.770100 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.802146 (+ 32046us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42503,"raft.queue_time_us":45916,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":39256,"spinlock_wait_cycles":10049792,"thread_start_us":328469,"threads_started":1,"wal-append.queue_time_us":779122}
I20260709 13:01:38.814029  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.801415  6408 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:38.771796  1676 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:38.798897  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:38.805655  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:38.806474  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.750320  6168 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.811321  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.825682  1703 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:38.826113  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:38.823937  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 52274.9us
I20260709 13:01:38.827113  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.736984  6157 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.066s
W20260709 13:01:38.778684  6421 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:01:38.796244  6346 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.844s	user 0.003s	sys 0.003s
I20260709 13:01:38.763252  6307 consensus_queue.cc:1243] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:01:38.851938  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 53139.9us
I20260709 13:01:38.784261  6211 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:38.854321  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:38.782431  6424 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:01:38.829486  6419 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:38.817086  6408 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:38.829366  1526 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:38.857306  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:38.863015  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.803524  6299 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:38.828830  6182 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:38.767338  6415 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.751775  6376 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.828796  6168 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.799001  6057 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:38.857065  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:38.811770  6085 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:38.850729  6407 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:38.876151  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.903388  1709 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:38.905165  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:38.877215  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 48016.1us
I20260709 13:01:38.857144  6144 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:38.866945  6199 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.868764  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.898449  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.907723  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:38.908324  1709 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:01:38.909739  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:38.880541  6063 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:38.880257  6182 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.915170  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 851) took 2179 ms. Trace:
I20260709 13:01:38.897039  6319 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:38.914112  1709 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:38.917029  1629 rpcz_store.cc:276] 0709 13:01:36.735790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.256881 (+1521091us) service_pool.cc:224] Handling call
0709 13:01:38.257272 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.285311 (+ 28039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.285363 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.286111 (+   748us) log.cc:844] Serialized 26 byte log entry
0709 13:01:38.420460 (+134349us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:38.420706 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.421405 (+   699us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.914965 (+493560us) raft_consensus.cc:1685] finished
0709 13:01:38.914992 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.915111 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5242,"raft.queue_time_us":9128,"spinlock_wait_cycles":61207296,"thread_start_us":126664,"threads_started":1,"wal-append.queue_time_us":212301}
I20260709 13:01:38.904346  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1236) took 2084 ms. Trace:
I20260709 13:01:38.936924  2350 rpcz_store.cc:276] 0709 13:01:36.820210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.924318 (+104108us) service_pool.cc:224] Handling call
0709 13:01:36.951370 (+ 27052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.547814 (+596444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.547852 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.558368 (+ 10516us) log.cc:844] Serialized 26 byte log entry
0709 13:01:37.799840 (+241472us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:37.800444 (+   604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.897688 (+1097244us) spinlock_profiling.cc:243] Waited 93.2 ms on lock 0x7b5400762d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:38.897749 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.897811 (+    62us) raft_consensus.cc:1685] finished
0709 13:01:38.897825 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.904265 (+  6440us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4812,"raft.queue_time_us":566364,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":14546,"spinlock_wait_cycles":214410112,"thread_start_us":689675,"threads_started":2,"wal-append.queue_time_us":422401}
I20260709 13:01:38.913043  6199 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.920053  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:38.920552  6182 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.923760  6361 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:38.908982  6057 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:38.878516  6299 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:38.962746  6299 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:38.969200  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:38.932726  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1038) took 3044 ms. Trace:
I20260709 13:01:38.975767  1633 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:38.990082  6356 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:38.991289  6356 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.972837  2346 rpcz_store.cc:276] 0709 13:01:35.868921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.869871 (+   950us) service_pool.cc:224] Handling call
0709 13:01:35.870349 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.948409 (+ 78060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.948442 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:35.948935 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.506964 (+558029us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:36.507022 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.507118 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.795736 (+2288618us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b24003aaac8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda244739c4 00007fda24471acb 00007fda2462cdc9 00007fda2462ca55 00007fda2462c09d 00007fda2725e10e 00007fda2727588c 00007fda27274bb6 00007fda272747d8 00007fda2727e221
0709 13:01:38.907217 (+111481us) raft_consensus.cc:1685] finished
0709 13:01:38.907243 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.913642 (+  6399us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77617,"raft.queue_time_us":98801,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":11480,"spinlock_wait_cycles":54460800,"thread_start_us":463780,"threads_started":1,"wal-append.queue_time_us":756692}
I20260709 13:01:38.943776  6278 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.947018  6243 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:38.949563  6063 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:38.878182  6249 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.914484  6407 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:38.917577  6417 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.428s	user 0.000s	sys 0.003s
I20260709 13:01:38.955757  1707 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:01:39.019169  1707 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 8
I20260709 13:01:38.956777  6303 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:38.958626  6085 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.959277  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:38.959985  6408 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:38.969887  6235 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:38.974731  6361 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:39.016366  6256 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.623s	user 0.012s	sys 0.006s
I20260709 13:01:39.019528  1707 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 11.9
I20260709 13:01:39.066670  1707 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Aborting config change with OpId 11.9: Aborted: Op aborted by new leader
I20260709 13:01:39.067260  1707 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:39.069634  6429 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:01:39.070138  6199 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:39.071920  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:39.073721  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:39.073838  6361 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.075045  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:39.076645  6361 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:39.051141  6303 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.077541  6303 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:39.079698  6235 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.080938  6303 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:39.081055  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:39.065516  1553 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:39.083209  6326 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:39.083536  6085 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:39.083551  6235 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:39.066030  1560 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:39.087034  6076 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:39.087747  6076 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:39.090833  6243 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:39.091691  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2653) took 1647 ms. Trace:
I20260709 13:01:39.092061  1558 rpcz_store.cc:276] 0709 13:01:37.444163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.447416 (+  3253us) service_pool.cc:224] Handling call
0709 13:01:37.447815 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.573542 (+125727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.573592 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.573642 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:37.573683 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.091428 (+1517745us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540038bdd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:39.091488 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.091629 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288573696}
I20260709 13:01:39.094851  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:39.095552  1558 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
W20260709 13:01:39.090755  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2550) took 3570 ms (client timeout 1804 ms). Trace:
I20260709 13:01:39.097224  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:39.097069  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:39.098129  1633 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:01:39.096910  1635 rpcz_store.cc:269] 0709 13:01:35.520578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.592347 (+ 71769us) service_pool.cc:224] Handling call
0709 13:01:39.090554 (+3498207us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400197654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:39.090675 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2719614720}
W20260709 13:01:39.098960  6386 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.849s	user 0.009s	sys 0.005s
I20260709 13:01:39.099537  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:39.122457  1558 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:39.124284  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:39.126011  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:39.100049  6157 raft_consensus.cc:3055] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:39.129256  1558 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
W20260709 13:01:39.135506  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 278526us
I20260709 13:01:39.140447  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:39.139534  1706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.001s	sys 0.021s
I20260709 13:01:39.103976  6350 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:39.055380  6402 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.142689  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:39.141489  1706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.001s	sys 0.021s
I20260709 13:01:39.163848  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:39.170790  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:39.172528  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:39.173240  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:39.173897  1705 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:39.108316  6326 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:39.178380  1560 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:39.180269  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:39.182941  2347 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1237) took 2182 ms (client timeout 2541 ms). Trace:
W20260709 13:01:39.184362  2347 rpcz_store.cc:269] 0709 13:01:36.985763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.985913 (+   150us) service_pool.cc:224] Handling call
0709 13:01:39.167974 (+2182061us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400690854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:39.168116 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2989440256}
I20260709 13:01:39.121927  6303 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:39.179957  6420 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:39.193681  1702 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:39.138021  6157 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:39.174129  6144 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:39.158107  6350 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:39.185034  6199 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:39.177431  6235 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:39.219900  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.231557  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:39.232434  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:39.127022  6386 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:39.234788  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 327129us
I20260709 13:01:39.196396  6420 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:39.218677  6144 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.262248  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:39.263041  1633 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:39.153795  6407 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:39.229939  6199 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:39.195430  6326 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.181274  6063 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:39.195310  6303 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:39.279381  2344 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:39.232348  6337 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:39.284230  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:39.269275  6220 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.578s	user 0.013s	sys 0.000s
I20260709 13:01:39.242010  6436 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:39.262674  6144 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:39.357779  1701 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:39.359901  6317 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:39.280540  6440 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:39.362414  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:39.269836  6057 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:39.363857  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:39.364216  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:39.364675  2347 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:01:39.365221  6391 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.000s
I20260709 13:01:39.363901  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:39.365975  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:39.365877  6391 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.366808  2346 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:39.368569  6391 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:39.371615  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:39.372393  2348 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:39.373361  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2035) took 1120 ms. Trace:
I20260709 13:01:39.280993  6312 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:39.339128  6350 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.374456  6312 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.375077  6350 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:39.373847  1708 rpcz_store.cc:276] 0709 13:01:38.253436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.253583 (+   147us) service_pool.cc:224] Handling call
0709 13:01:38.275946 (+ 22363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.276302 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.276341 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.277063 (+   722us) log.cc:844] Serialized 26 byte log entry
0709 13:01:38.335229 (+ 58166us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:38.335294 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.335379 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.358138 (+1022759us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b24001df648. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda244739c4 00007fda24471acb 00007fda2462cdc9 00007fda2462ca55 00007fda2462bfb0 00007fda272a2488 00007fda272a2be8 00007fda2727a398 00007fda2727576a 00007fda27274bb6
0709 13:01:39.373048 (+ 14910us) raft_consensus.cc:1685] finished
0709 13:01:39.373072 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.373289 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33184512,"thread_start_us":57733,"threads_started":1,"wal-append.queue_time_us":56896}
I20260709 13:01:39.376806  6312 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:39.377352  1632 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.380260  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:39.373351  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:39.336705  6059 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.878s	user 0.014s	sys 0.000s
W20260709 13:01:39.386652  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 224685us
I20260709 13:01:39.364751  6057 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:39.332037  6368 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.592s	user 0.008s	sys 0.005s
I20260709 13:01:39.283768  6199 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:39.299791  6272 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.001s
I20260709 13:01:39.263092  6235 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 13:01:39.385823  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:39.389555  1709 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:01:39.418499  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:39.422021  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 149727us
I20260709 13:01:39.384377  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:39.400821  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:39.381927  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:39.298117  6278 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:39.427603  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:39.427902  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 46048.8us
I20260709 13:01:39.428880  1602 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:39.430970  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:39.432022  1633 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:39.433120  1675 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:39.435053  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.435858  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:39.385154  6308 consensus_queue.cc:1175] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" responder_term: 11 status { last_received { term: 11 index: 9 } last_committed_idx: 7 last_received_current_leader { term: 11 index: 9 } }
I20260709 13:01:39.404179  6349 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:39.345741  6337 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.490787  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2077) took 3550 ms. Trace:
W20260709 13:01:39.454854  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.491447  1557 rpcz_store.cc:276] 0709 13:01:35.940827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.955409 (+ 14582us) service_pool.cc:224] Handling call
0709 13:01:35.955759 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.677056 (+721297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.490483 (+2813427us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b5400208354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:39.490699 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1654852480}
I20260709 13:01:39.415983  6277 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:39.412683  6436 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:39.412614  6198 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:39.386459  6168 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.344230  6220 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:39.513067  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 228928us
W20260709 13:01:39.451098  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.431809  6275 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:39.493199  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:39.518146  1557 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
I20260709 13:01:39.442358  1709 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:39.419840  6272 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:39.338800  6157 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:39.515794  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:39.510721  6277 raft_consensus.cc:1068] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:39.458448  6256 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:39.514560  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:39.445313  6278 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:39.412385  6368 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.491312  6243 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:01:39.443488  6442 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:39.558512  6444 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:01:39.559234  1555 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:39.560173  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.495190  6162 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:39.560940  6162 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.561398  6162 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:39.517452  6402 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:39.399307  6059 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:39.564639  6326 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:39.566318  6059 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:39.519791  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:39.568054  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:39.568702  1557 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:39.569847  1558 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:01:39.515563  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 170858us
I20260709 13:01:39.571059  6168 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:39.534160  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:39.508296  6198 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.572521  6198 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:39.575960  6182 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.614s	user 0.011s	sys 0.000s
I20260709 13:01:39.437647  6445 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:39.577617  6303 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:39.580111  6337 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:39.581715  6439 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:01:39.538324  6317 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:39.520115  6275 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:39.539552  6200 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:01:39.542219  1560 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:01:39.530238  6454 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:39.546202  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:39.553149  6278 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:39.554064  6438 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:01:39.556196  6177 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:39.584138  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2670) took 1289 ms. Trace:
I20260709 13:01:39.588681  1557 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:39.516160  6349 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.430068  6376 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:39.612948  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 184142us
I20260709 13:01:39.550168  6277 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:39.619870  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 73795.3us
W20260709 13:01:39.629875  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.534968  6254 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:01:39.588160  6440 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:39.593905  6275 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:39.632997  1706 rpcz_store.cc:276] 0709 13:01:38.295030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.452311 (+157281us) service_pool.cc:224] Handling call
0709 13:01:38.481778 (+ 29467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:38.493725 (+ 11947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.493786 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:38.506476 (+ 12690us) log.cc:844] Serialized 52 byte log entry
0709 13:01:39.162848 (+656372us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:39.196850 (+ 34002us) log.cc:844] Serialized 14 byte log entry
0709 13:01:39.213152 (+ 16302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.213243 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.583844 (+370601us) raft_consensus.cc:1685] finished
0709 13:01:39.583878 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.584045 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22258,"prepare.queue_time_us":33750,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":4877,"raft.queue_time_us":22976,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":24918,"thread_start_us":611536,"threads_started":1,"wal-append.queue_time_us":896964}
I20260709 13:01:39.589443  6063 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:39.549741  6256 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:39.576480  6182 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.637372  6177 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.504760  6420 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.682567  6442 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:39.635406  6457 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:01:39.686064  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.642470  6200 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:39.689950  6144 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:39.694178  6459 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:01:39.658239  6440 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:39.679524  6299 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.716s	user 0.013s	sys 0.000s
W20260709 13:01:39.704075  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 268257us
W20260709 13:01:39.752092  6076 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.664s	user 0.017s	sys 0.000s
I20260709 13:01:39.752306  6441 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.688205  6457 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.752686  6076 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.753026  6457 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:39.755050  6198 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.789845  6235 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:39.790450  6256 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:39.790798  6235 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:39.628865  6454 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:39.684515  6177 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:39.809376  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:39.744791  6440 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:39.751538  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:39.801095  6076 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:39.801939  6235 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:39.772176  6442 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:39.812994  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:39.752986  6441 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:39.742168  6272 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:39.810464  1709 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:39.888936  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:39.890530  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:39.891722  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:39.669772  6063 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:39.898720  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:39.899611  1556 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:39.906370  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.875296  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:39.879791  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:39.883430  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:39.923188  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:39.924288  1703 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:39.927911  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 176706us
W20260709 13:01:39.937088  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.996802  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:39.997942  1559 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:39.998936  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:40.000177  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:40.000758  1559 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:40.005535  1556 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
W20260709 13:01:40.006785  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 77832.2us
I20260709 13:01:40.007319  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:40.009248  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:40.010368  1556 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:39.756896  6299 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:40.012445  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:40.014137  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:40.015935  1556 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:40.017424  1556 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:40.018232  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2672) took 1682 ms. Trace:
I20260709 13:01:40.018651  1556 rpcz_store.cc:276] 0709 13:01:38.336160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.016766 (+1680606us) service_pool.cc:224] Handling call
0709 13:01:40.017172 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.017933 (+   761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.018163 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.019260  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:40.020058  1706 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:40.022135  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:39.779706  6452 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:40.025358  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.026700  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.750672  6200 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:40.008020  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:40.072582  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.897818  6220 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:40.148283  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 140383us
I20260709 13:01:40.149756  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:40.004887  6076 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:40.119534  6216 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.368s	user 0.006s	sys 0.009s
I20260709 13:01:40.048745  6168 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:01:40.180230  6350 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.805s	user 0.013s	sys 0.003s
I20260709 13:01:40.177961  6445 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:40.178833  6349 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:40.224104  6312 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:40.211170  6307 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:40.270290  6461 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.507s	user 0.000s	sys 0.001s
I20260709 13:01:40.271098  5847 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:40.272442  1708 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:40.272789  6278 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:40.273072  6199 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.851s	user 0.003s	sys 0.008s
I20260709 13:01:40.272797  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 703) took 1322 ms. Trace:
W20260709 13:01:40.274356  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 124741us
I20260709 13:01:40.274504  1555 rpcz_store.cc:276] 0709 13:01:38.950407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.074759 (+124352us) service_pool.cc:224] Handling call
0709 13:01:40.272571 (+1197812us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b5400313654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:40.272716 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1111464832}
I20260709 13:01:40.255791  6445 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:40.301952  6466 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:01:40.307258  1555 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:40.308475  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2094) took 1842 ms. Trace:
I20260709 13:01:40.308893  1555 rpcz_store.cc:276] 0709 13:01:38.466068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.275549 (+1809481us) service_pool.cc:224] Handling call
0709 13:01:40.306970 (+ 31421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.308073 (+  1103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.308390 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.251317  6168 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:40.236713  6216 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:40.289413  1706 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:40.280833  6408 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:40.327196  6462 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:40.329242  6441 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.282044  6144 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.323607  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:40.276422  6312 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:40.298779  6445 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:40.362339  6299 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:40.283941  6307 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:40.188246  6452 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:40.368052  1556 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:40.354921  1555 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:01:40.358198  6465 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:01:40.372874  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:40.369948  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2095) took 1903 ms. Trace:
I20260709 13:01:40.285663  6199 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:40.366099  6454 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.375360  1555 rpcz_store.cc:276] 0709 13:01:38.466550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.326047 (+1859497us) service_pool.cc:224] Handling call
0709 13:01:40.354454 (+ 28407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.369107 (+ 14653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.369864 (+   757us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.381207  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:40.387647  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2767) took 4560 ms. Trace:
I20260709 13:01:40.389935  6168 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.392747  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 780) took 3651 ms. Trace:
I20260709 13:01:40.397158  6470 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:40.398656  2345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 1046) took 2578 ms (client timeout 1930 ms). Trace:
I20260709 13:01:40.416615  6063 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.403276  1628 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:40.417155  6312 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.414803  6216 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.378577  6144 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:40.437680  1701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.070s	sys 0.143s
W20260709 13:01:40.453413  1701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.070s	sys 0.143s
I20260709 13:01:40.400162  6368 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:40.444276  2345 rpcz_store.cc:269] 0709 13:01:37.820233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.823986 (+  3753us) service_pool.cc:224] Handling call
0709 13:01:40.398441 (+2574455us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54006202d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:40.398574 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344835456}
W20260709 13:01:40.409883  6275 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.735s	user 0.008s	sys 0.005s
I20260709 13:01:40.438061  1704 rpcz_store.cc:276] 0709 13:01:36.741010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.742141 (+  1131us) service_pool.cc:224] Handling call
0709 13:01:36.742474 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.017847 (+275373us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:37.018115 (+   268us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.559686 (+541571us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.559762 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:37.559825 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.392354 (+2832529us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b540016ce54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:40.392414 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.392601 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10105,"prepare.queue_time_us":363103,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":3498,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":12407,"spinlock_wait_cycles":632713472,"thread_start_us":541135,"threads_started":1,"wal-append.queue_time_us":655465,"wal-append.run_cpu_time_us":1510,"wal-append.run_wall_time_us":1014242}
I20260709 13:01:40.433130  1630 rpcz_store.cc:276] 0709 13:01:35.826789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.951148 (+124359us) service_pool.cc:224] Handling call
0709 13:01:35.951544 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.323540 (+371996us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:36.715841 (+392301us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.564232 (+848391us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.564764 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:01:37.564930 (+   166us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:37.564971 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.387021 (+2822050us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b5400138dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:40.387088 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.387161 (+    73us) raft_consensus.cc:1685] finished
0709 13:01:40.387300 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.387523 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2653,"prepare.queue_time_us":355503,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":3375,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":21476,"spinlock_wait_cycles":825920384,"thread_start_us":980013,"threads_started":2,"wal-append.queue_time_us":879859,"wal-append.run_cpu_time_us":2479,"wal-append.run_wall_time_us":1039736}
I20260709 13:01:40.463980  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:40.464999  1630 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
W20260709 13:01:40.450120  6057 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.003s	sys 0.054s
W20260709 13:01:40.447045  6249 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.011s	sys 0.018s
W20260709 13:01:40.453837  6457 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.589s	user 0.015s	sys 0.001s
I20260709 13:01:40.472554  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:40.466786  6168 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:40.480995  6277 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.823s	user 0.006s	sys 0.006s
I20260709 13:01:40.454811  6199 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:40.512419  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2783) took 1002 ms. Trace:
I20260709 13:01:40.463766  6144 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:40.467352  6057 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.004s	sys 0.054s
I20260709 13:01:40.502810  6452 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:40.457993  6470 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:40.513697  6303 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.000s	sys 0.030s
W20260709 13:01:40.587134  6383 raft_consensus.cc:670] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } [suppressed 5 similar messages]
I20260709 13:01:40.592506  6307 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.526787  6277 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:40.511535  6457 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:40.624282  6358 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 3.541s	user 0.009s	sys 0.024s
W20260709 13:01:40.614836  6303 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.010s	sys 0.052s
W20260709 13:01:40.614235  6440 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.735s	user 0.000s	sys 0.013s
I20260709 13:01:40.633903  6457 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.641410  6440 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:40.647099  6328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.014s	sys 0.044s
I20260709 13:01:40.644977  6440 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:40.679368  6478 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:01:40.377159  1559 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:01:40.723110  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.027s	sys 0.026s
W20260709 13:01:40.724052  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.028s	sys 0.026s
W20260709 13:01:40.724624  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 878041us
I20260709 13:01:40.724639  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2688) took 1276 ms. Trace:
I20260709 13:01:40.726418  1559 rpcz_store.cc:276] 0709 13:01:39.447583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.999919 (+552336us) service_pool.cc:224] Handling call
0709 13:01:40.724215 (+724296us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.444190  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.738397  1602 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.000s	sys 0.014s
W20260709 13:01:40.739145  1602 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.000s	sys 0.015s
W20260709 13:01:40.739861  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 906985us
I20260709 13:01:40.514287  1630 rpcz_store.cc:276] 0709 13:01:39.510212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.472070 (+961858us) service_pool.cc:224] Handling call
0709 13:01:40.512309 (+ 40239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.511905  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:40.416523  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.746546  1706 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:40.748510  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.514871  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.739882  1707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.012s	sys 0.010s
W20260709 13:01:40.750017  1707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.012s	sys 0.010s
W20260709 13:01:40.507652  1635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.022s	sys 0.017s
W20260709 13:01:40.752166  1635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.022s	sys 0.017s
I20260709 13:01:40.754194  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2771) took 1315 ms. Trace:
I20260709 13:01:40.754809  1635 rpcz_store.cc:276] 0709 13:01:39.438269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.440277 (+  2008us) service_pool.cc:224] Handling call
0709 13:01:39.494512 (+ 54235us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:39.693228 (+198716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:39.705373 (+ 12145us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:39.760645 (+ 55272us) log.cc:844] Serialized 26 byte log entry
0709 13:01:40.753147 (+992502us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:40.753301 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.753539 (+   238us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.753650 (+   111us) raft_consensus.cc:1685] finished
0709 13:01:40.753671 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.753989 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1257,"raft.queue_time_us":92911,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":22687,"thread_start_us":1057210,"threads_started":2,"wal-append.queue_time_us":791350}
W20260709 13:01:40.757819  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.744866  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:40.763084  1630 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:40.510736  6085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.015s	sys 0.025s
W20260709 13:01:40.767848  6085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.015s	sys 0.026s
I20260709 13:01:40.768460  6057 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:40.766497  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:40.770874  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2787) took 1204 ms. Trace:
I20260709 13:01:40.771577  1635 rpcz_store.cc:276] 0709 13:01:39.566110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.765946 (+1199836us) service_pool.cc:224] Handling call
0709 13:01:40.770698 (+  4752us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.473110  6249 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.011s	sys 0.018s
I20260709 13:01:40.512688  6168 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:40.655067  6328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.014s	sys 0.044s
W20260709 13:01:40.454155  6162 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.892s	user 0.015s	sys 0.000s
I20260709 13:01:40.466216  6275 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:40.486506  6386 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.778430  6162 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:40.782482  1633 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:40.783303  6162 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:40.785197  6307 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:40.831678  6168 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.832456  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2664) took 2592 ms. Trace:
I20260709 13:01:40.819967  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:40.842588  1635 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:40.843225  6200 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:40.844398  1558 rpcz_store.cc:276] 0709 13:01:38.240268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.159184 (+918916us) service_pool.cc:224] Handling call
0709 13:01:39.159635 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:39.570330 (+410695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.831647 (+1261317us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b54001c0054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:40.832200 (+   553us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":942537600}
I20260709 13:01:40.834885  6470 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.844568  6057 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:40.855680  6057 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:40.843472  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2781) took 1353 ms. Trace:
I20260709 13:01:40.856868  1635 rpcz_store.cc:276] 0709 13:01:39.490520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.772944 (+1282424us) service_pool.cc:224] Handling call
0709 13:01:40.843357 (+ 70413us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.859056  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:40.860278  1635 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
I20260709 13:01:40.861424  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2786) took 1295 ms. Trace:
I20260709 13:01:40.861938  1635 rpcz_store.cc:276] 0709 13:01:39.565711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.858087 (+1292376us) service_pool.cc:224] Handling call
0709 13:01:40.861322 (+  3235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.902665  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:40.903463  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:40.904017  1635 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:40.905066  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2794) took 1336 ms. Trace:
I20260709 13:01:40.905762  1635 rpcz_store.cc:276] 0709 13:01:39.568599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.902017 (+1333418us) service_pool.cc:224] Handling call
0709 13:01:40.904985 (+  2968us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.907349  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:40.909948  6470 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.781452  6275 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:40.869849  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2681) took 2376 ms. Trace:
W20260709 13:01:40.871415  1553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2088) took 2407 ms (client timeout 2249 ms). Trace:
I20260709 13:01:40.868458  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2085) took 2861 ms. Trace:
I20260709 13:01:40.873303  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2666) took 2632 ms. Trace:
I20260709 13:01:40.878724  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:40.880402  1636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 849) took 4571 ms (client timeout 2833 ms). Trace:
W20260709 13:01:40.886229  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.891059  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2684) took 2184 ms. Trace:
I20260709 13:01:40.894217  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:40.876385  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2581) took 1908 ms. Trace:
I20260709 13:01:40.897080  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:40.912873  1560 rpcz_store.cc:276] 0709 13:01:38.240617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.188707 (+948090us) service_pool.cc:224] Handling call
0709 13:01:39.189101 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:39.593854 (+404753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.872593 (+1278739us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b5400207954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:40.873087 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":811175296}
I20260709 13:01:40.919791  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:40.934435  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2782) took 1430 ms. Trace:
I20260709 13:01:40.912075  1557 rpcz_store.cc:276] 0709 13:01:38.493387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.827886 (+2334499us) service_pool.cc:224] Handling call
0709 13:01:40.828635 (+   749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.829091 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.829129 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.829206 (+    77us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:40.829269 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.829359 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.869577 (+ 40218us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.913599  1636 rpcz_store.cc:269] 0709 13:01:36.308946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.924318 (+615372us) service_pool.cc:224] Handling call
0709 13:01:40.879794 (+3955476us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540017b454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:40.880139 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2843645312}
W20260709 13:01:40.912302  1553 rpcz_store.cc:269] 0709 13:01:38.463813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.913547 (+449734us) service_pool.cc:224] Handling call
0709 13:01:40.870845 (+1957298us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54002ffb54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:40.871165 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259294464}
I20260709 13:01:40.912055  6307 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:40.949895  1557 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 13:01:40.951774  1635 rpcz_store.cc:276] 0709 13:01:39.504074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.906750 (+1402676us) service_pool.cc:224] Handling call
0709 13:01:40.934306 (+ 27556us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.915921  1702 rpcz_store.cc:276] 0709 13:01:38.706878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.727933 (+ 21055us) service_pool.cc:224] Handling call
0709 13:01:40.890350 (+2162417us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b5400223154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:40.890715 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":994819456}
I20260709 13:01:40.912423  1552 rpcz_store.cc:276] 0709 13:01:38.006618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.438505 (+431887us) service_pool.cc:224] Handling call
0709 13:01:38.505444 (+ 66939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.534832 (+ 29388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.867849 (+2333017us) spinlock_profiling.cc:243] Waited 66.6 ms on lock 0x7b5400337cd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:40.868203 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153127680}
I20260709 13:01:40.965459  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 697) took 2021 ms. Trace:
I20260709 13:01:40.911079  6470 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:40.966274  1557 rpcz_store.cc:276] 0709 13:01:38.939976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.946206 (+2006230us) service_pool.cc:224] Handling call
0709 13:01:40.949258 (+  3052us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:40.958073 (+  8815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.961416 (+  3343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.931393  6057 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.968737  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2793) took 1400 ms. Trace:
I20260709 13:01:40.981433  1634 rpcz_store.cc:276] 0709 13:01:39.568212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.878045 (+1309833us) service_pool.cc:224] Handling call
0709 13:01:40.968638 (+ 90593us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.853205  6445 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.013397  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:41.011286  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:41.019860  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:41.025643  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 226909us
I20260709 13:01:41.021338  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2790) took 1449 ms. Trace:
I20260709 13:01:41.026801  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:41.031038  1635 rpcz_store.cc:276] 0709 13:01:39.567206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.972923 (+1405717us) service_pool.cc:224] Handling call
0709 13:01:41.016421 (+ 43498us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.031939  1630 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:41.030520  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:41.002662  6144 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:40.917466  1629 rpcz_store.cc:276] 0709 13:01:38.967592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.970178 (+  2586us) service_pool.cc:224] Handling call
0709 13:01:38.970651 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.999623 (+ 28972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.999681 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:39.017097 (+ 17416us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.253954 (+236857us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:39.254000 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.875507 (+1621507us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b540017c354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:40.875600 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.875691 (+    91us) raft_consensus.cc:1685] finished
0709 13:01:40.875711 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.876147 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22760,"raft.queue_time_us":94162,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":103,"spinlock_wait_cycles":12744576,"thread_start_us":236423,"threads_started":1,"wal-append.queue_time_us":508618,"wal-append.run_cpu_time_us":2189,"wal-append.run_wall_time_us":1023320}
W20260709 13:01:41.037868  1630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2590) took 1372 ms (client timeout 1740 ms). Trace:
W20260709 13:01:41.044101  1630 rpcz_store.cc:269] 0709 13:01:39.660550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.743828 (+1083278us) service_pool.cc:224] Handling call
0709 13:01:41.032516 (+288688us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.047297  1703 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:41.049692  6277 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.050938  6277 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:41.053613  1704 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.022s	sys 0.038s
I20260709 13:01:41.057843  1553 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:41.058720  6455 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:41.059804  6455 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:41.043977  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 59538.3us
I20260709 13:01:41.058902  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 701) took 2109 ms. Trace:
I20260709 13:01:41.066869  1553 rpcz_store.cc:276] 0709 13:01:38.949783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.973842 (+2024059us) service_pool.cc:224] Handling call
0709 13:01:41.043267 (+ 69425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.058459 (+ 15192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.058802 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.074276  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2795) took 1505 ms. Trace:
W20260709 13:01:41.076062  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.133s	user 0.003s	sys 0.013s
I20260709 13:01:41.087291  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:41.074873  1634 rpcz_store.cc:276] 0709 13:01:39.586645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.997028 (+1410383us) service_pool.cc:224] Handling call
0709 13:01:41.074087 (+ 77059us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.040745  6440 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:41.097282  1554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.006s	sys 0.017s
I20260709 13:01:41.064951  6057 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.092418  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:41.089340  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2695) took 1277 ms. Trace:
I20260709 13:01:41.091761  1708 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:41.087970  6470 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.086308  6445 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.105175  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:41.104005  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:41.104547  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.223s	user 0.003s	sys 0.014s
W20260709 13:01:41.112357  1554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.007s	sys 0.017s
I20260709 13:01:41.117856  1636 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:41.066437  6307 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:41.100564  1710 rpcz_store.cc:276] 0709 13:01:39.811940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.048219 (+236279us) service_pool.cc:224] Handling call
0709 13:01:40.278494 (+230275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.278923 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:40.747801 (+468878us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.017520 (+269719us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.021123 (+  3603us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:41.071468 (+ 50345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.088856 (+ 17388us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.089094 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28434,"prepare.queue_time_us":111467,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":24716,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":20696,"thread_start_us":649795,"threads_started":2}
I20260709 13:01:41.138069  1706 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:01:41.147318  1629 raft_consensus.cc:2410] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 13 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:41.149776  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:41.155193  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2792) took 1587 ms. Trace:
I20260709 13:01:41.155906  1629 rpcz_store.cc:276] 0709 13:01:39.567869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.075906 (+1508037us) service_pool.cc:224] Handling call
0709 13:01:41.155096 (+ 79190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:41.165889  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.173133  1630 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:41.194962  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:41.182862  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.924s	user 0.013s	sys 0.019s
W20260709 13:01:41.201047  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.013s	sys 0.020s
I20260709 13:01:41.206844  6383 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:41.208007  6383 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:41.213624  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.006s	sys 0.016s
W20260709 13:01:41.213922  1555 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.000s	sys 0.017s
W20260709 13:01:41.214254  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.006s	sys 0.017s
I20260709 13:01:41.192426  6162 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:41.215104  6303 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.005s	sys 0.011s
W20260709 13:01:41.215673  6303 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.005s	sys 0.011s
I20260709 13:01:41.215502  6162 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.217691  6440 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:41.218490  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2687) took 1771 ms. Trace:
I20260709 13:01:41.219026  1553 rpcz_store.cc:276] 0709 13:01:39.447268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.099179 (+1651911us) service_pool.cc:224] Handling call
0709 13:01:41.169191 (+ 70012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.217912 (+ 48721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.217949 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.218027 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:41.218065 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.218159 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.218380 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:41.214319  1555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.000s	sys 0.017s
I20260709 13:01:41.213727  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:41.213718  1634 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:41.225071  1630 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:41.225936  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2773) took 1784 ms. Trace:
I20260709 13:01:41.226945  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2784) took 1714 ms. Trace:
I20260709 13:01:41.178774  6168 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:41.227782  6168 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.232312  6415 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.227425  1629 rpcz_store.cc:276] 0709 13:01:39.512644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.168750 (+1656106us) service_pool.cc:224] Handling call
0709 13:01:41.226873 (+ 58123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.233227  6415 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:41.235661  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2774) took 1794 ms. Trace:
I20260709 13:01:41.225971  6162 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.226570  1630 rpcz_store.cc:276] 0709 13:01:39.441021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.224233 (+1783212us) service_pool.cc:224] Handling call
0709 13:01:41.224791 (+   558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.225602 (+   811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.225836 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.238782  1630 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:41.236128  6415 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:41.236091  1629 rpcz_store.cc:276] 0709 13:01:39.441300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.234183 (+1792883us) service_pool.cc:224] Handling call
0709 13:01:41.234779 (+   596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.235212 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.235243 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.235317 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:41.235382 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.235453 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.235575 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.239646  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2775) took 1798 ms. Trace:
I20260709 13:01:41.246739  1630 rpcz_store.cc:276] 0709 13:01:39.441597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.237777 (+1796180us) service_pool.cc:224] Handling call
0709 13:01:41.238387 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.239307 (+   920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.239558 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.221278  1553 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:41.254431  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2097) took 1803 ms. Trace:
I20260709 13:01:41.254447  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:41.221447  6440 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:41.291841  1629 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:01:41.292922  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2776) took 1851 ms. Trace:
I20260709 13:01:41.255937  1553 rpcz_store.cc:276] 0709 13:01:39.447932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.220316 (+1772384us) service_pool.cc:224] Handling call
0709 13:01:41.220898 (+   582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.250556 (+ 29658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.250791 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.312395  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2789) took 1744 ms. Trace:
I20260709 13:01:41.314093  1635 rpcz_store.cc:276] 0709 13:01:39.566857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.168593 (+1601736us) service_pool.cc:224] Handling call
0709 13:01:41.310966 (+142373us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.293349  1629 rpcz_store.cc:276] 0709 13:01:39.441879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.249070 (+1807191us) service_pool.cc:224] Handling call
0709 13:01:41.291468 (+ 42398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.292541 (+  1073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.292829 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:41.303884  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.322803  6076 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.097s	user 0.021s	sys 0.008s
I20260709 13:01:41.323616  6076 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:41.316008  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2690) took 1510 ms. Trace:
I20260709 13:01:41.293637  6260 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:01:41.326921  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 164961us
W20260709 13:01:41.327675  6491 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.211s	user 0.001s	sys 0.001s
I20260709 13:01:41.357678  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:41.381922  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2682) took 2853 ms. Trace:
W20260709 13:01:41.350142  6177 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.423s	user 0.011s	sys 0.013s
W20260709 13:01:41.377708  1674 connection.cc:256] RPC call timeout handler was delayed by -1.1131394729999897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21860000000000002s for a response
I20260709 13:01:41.350102  6057 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.364357  6168 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:41.380431  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 69042.9us
I20260709 13:01:41.328227  6328 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:41.393179  1634 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:41.398685  6177 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:41.401697  6356 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:41.402547  6408 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.045s	user 0.016s	sys 0.014s
I20260709 13:01:41.404078  1631 raft_consensus.cc:2410] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:01:41.403967  6177 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:41.404781  6200 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.407011  6328 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.408532  6200 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:41.366698  1707 rpcz_store.cc:276] 0709 13:01:39.805931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.807590 (+  1659us) service_pool.cc:224] Handling call
0709 13:01:39.807969 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.808195 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:39.808479 (+   284us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.120744 (+1312265us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.172124 (+ 51380us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.182241 (+ 10117us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:41.202861 (+ 20620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.247912 (+ 45051us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.315908 (+ 67996us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18706,"prepare.queue_time_us":96079,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":37541,"raft.queue_time_us":36780,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":2784,"thread_start_us":1304860,"threads_started":2,"wal-append.queue_time_us":1165700}
W20260709 13:01:41.413226  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.414422  6462 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.047s	user 0.020s	sys 0.006s
I20260709 13:01:41.414999  6462 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:41.405408  6470 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:41.416390  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2791) took 1848 ms. Trace:
I20260709 13:01:41.417428  2348 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:01:41.420133  6487 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:01:41.385018  1558 rpcz_store.cc:276] 0709 13:01:38.522859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.850174 (+2327315us) service_pool.cc:224] Handling call
0709 13:01:40.851071 (+   897us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:40.867834 (+ 16763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.867893 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:40.869020 (+  1127us) log.cc:844] Serialized 52 byte log entry
0709 13:01:41.148743 (+279723us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:41.148793 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.148883 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.366061 (+217178us) raft_consensus.cc:1685] finished
0709 13:01:41.366087 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.375858 (+  9771us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15636,"raft.queue_time_us":15886,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":230,"thread_start_us":278227,"threads_started":1,"wal-append.queue_time_us":325833}
I20260709 13:01:41.423115  1558 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:01:41.423902  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2060) took 1146 ms. Trace:
I20260709 13:01:41.423924  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2101) took 1973 ms. Trace:
I20260709 13:01:41.416988  1634 rpcz_store.cc:276] 0709 13:01:39.567534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.395889 (+1828355us) service_pool.cc:224] Handling call
0709 13:01:41.415494 (+ 19605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.415798 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.415851 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.415943 (+    92us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:41.416010 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.416095 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.416280 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.426134  1558 rpcz_store.cc:276] 0709 13:01:39.450595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.422079 (+1971484us) service_pool.cc:224] Handling call
0709 13:01:41.422629 (+   550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.423563 (+   934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.423822 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.428887  1558 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:01:41.429328  1558 pending_rounds.cc:85] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 5
I20260709 13:01:41.429466  6328 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:41.429740  1558 pending_rounds.cc:107] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:01:41.430279  1558 raft_consensus.cc:2889] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:41.431319  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2104) took 1968 ms. Trace:
I20260709 13:01:41.431751  1558 rpcz_store.cc:276] 0709 13:01:39.462454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.427668 (+1965214us) service_pool.cc:224] Handling call
0709 13:01:41.428170 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.431011 (+  2841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.431235 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:41.433081  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 60163.5us
I20260709 13:01:41.416946  6470 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.409690  6260 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:41.442817  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:41.443624  2347 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:41.444896  6260 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:41.424417  1704 rpcz_store.cc:276] 0709 13:01:40.277398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.455884 (+178486us) service_pool.cc:224] Handling call
0709 13:01:40.475418 (+ 19534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.488938 (+ 13520us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:41.054374 (+565436us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.357693 (+303319us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.357999 (+   306us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:41.358971 (+   972us) log.cc:844] Serialized 26 byte log entry
0709 13:01:41.359171 (+   200us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:41.359232 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.359388 (+   156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.423609 (+ 64221us) raft_consensus.cc:1685] finished
0709 13:01:41.423637 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.423822 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69383,"prepare.queue_time_us":159997,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":95,"raft.queue_time_us":85003,"raft.run_cpu_time_us":508,"raft.run_wall_time_us":510,"thread_start_us":717004,"threads_started":2,"wal-append.queue_time_us":226482}
I20260709 13:01:41.436441  6383 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:41.450979  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.451112  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:41.452386  2317 connection.cc:256] RPC call timeout handler was delayed by -2.5061295159999872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25179999999999997s for a response
W20260709 13:01:41.452911  2317 connection.cc:256] RPC call timeout handler was delayed by -2.0508048860000372s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28330000000000005s for a response
I20260709 13:01:41.453840  2317 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:41.454216  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.455121  2353 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:41.456131  6328 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:41.458410  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2778) took 2016 ms. Trace:
I20260709 13:01:41.458953  1635 rpcz_store.cc:276] 0709 13:01:39.442387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.315094 (+1872707us) service_pool.cc:224] Handling call
0709 13:01:41.424919 (+109825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.457905 (+ 32986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.457938 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.458048 (+   110us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:41.458091 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.458160 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.458306 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.461663  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.471101  2352 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:41.476837  2353 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:41.484481  6414 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:41.496634  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.497435  6391 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:41.498371  6391 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.500933  6470 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.501856  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.502038  6391 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.505661  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:41.505901  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2770) took 2068 ms. Trace:
I20260709 13:01:41.506642  1707 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:01:41.506345  1633 rpcz_store.cc:276] 0709 13:01:39.437920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.487650 (+ 49730us) service_pool.cc:224] Handling call
0709 13:01:39.488135 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.783258 (+1295123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.505615 (+722357us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54001383d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:41.505833 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2975010048}
I20260709 13:01:41.508213  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:41.510223  1553 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:01:41.510447  1674 connection.cc:256] RPC call timeout handler was delayed by -1.0117811150000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
W20260709 13:01:41.511013  1674 connection.cc:256] RPC call timeout handler was delayed by -1.0107441540000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1904s for a response
I20260709 13:01:41.520872  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.485141  6414 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:41.521576  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:41.523121  6414 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:41.524595  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.526809  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.530016  6344 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.530916  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.510893  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2099) took 2062 ms. Trace:
I20260709 13:01:41.534379  1553 rpcz_store.cc:276] 0709 13:01:39.449830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.434605 (+1984775us) service_pool.cc:224] Handling call
0709 13:01:41.442078 (+  7473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.510630 (+ 68552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.510836 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.516987  6182 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.544219  2353 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:41.525413  6414 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.551493  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:41.550944  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:41.555514  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:41.559098  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:41.560310  2347 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:41.558117  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:41.563969  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:41.564407  1636 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:01:41.523140  6449 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.730s	user 0.000s	sys 0.003s
I20260709 13:01:41.566996  2352 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:41.582394  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:41.569247  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:41.584528  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.570544  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2788) took 1998 ms. Trace:
I20260709 13:01:41.585783  2353 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:41.585729  1636 rpcz_store.cc:276] 0709 13:01:39.566510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.968197 (+1401687us) service_pool.cc:224] Handling call
0709 13:01:41.565088 (+596891us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:41.573562  1704 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:41.587710  1702 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:41.589131  2349 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:41.590334  1705 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:41.591420  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.592300  2346 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
W20260709 13:01:41.597872  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.611601  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:41.613816  6162 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:41.615489  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.615861  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.616855  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:41.617081  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:41.618213  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.621752  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.621923  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:41.621958  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.624269  6353 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:41.624557  2318 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.707s (SENT) [suppressed 17 similar messages]
I20260709 13:01:41.617815  6157 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:41.624401  6319 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:41.631270  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.638370  6260 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:41.639132  6260 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:41.639339  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2796) took 2006 ms. Trace:
I20260709 13:01:41.639811  1634 rpcz_store.cc:276] 0709 13:01:39.633423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.434527 (+1801104us) service_pool.cc:224] Handling call
0709 13:01:41.470866 (+ 36339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.471321 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:41.471867 (+   546us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.638301 (+166434us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.638964 (+   663us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.639015 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:41.639058 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.639148 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.639279 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":946,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":3542,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":195,"thread_start_us":162970,"threads_started":1}
I20260709 13:01:41.648365  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.655246  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.660337  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.660015  1674 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:41.669335  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.670406  2353 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:41.673272  6353 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:41.673784  6353 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:41.674266  6353 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:41.671422  6168 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:01:41.680915  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:41.682081  2347 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:41.688525  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2572) took 3689 ms. Trace:
I20260709 13:01:41.689054  1627 rpcz_store.cc:276] 0709 13:01:37.999125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.322913 (+323788us) service_pool.cc:224] Handling call
0709 13:01:38.349040 (+ 26127us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.351338 (+  2298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.351370 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.358563 (+  7193us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.179615 (+821052us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b1800421600. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:01:39.536517 (+356902us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:39.536565 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.536663 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.522091 (+1985428us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:41.688085 (+165994us) raft_consensus.cc:1685] finished
0709 13:01:41.688111 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.688313 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12669,"raft.queue_time_us":6560,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":2998,"spinlock_wait_cycles":40527488,"thread_start_us":356674,"threads_started":1,"wal-append.queue_time_us":1320915}
I20260709 13:01:41.722028  6397 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.722908  6397 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.723541  6319 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.724172  6319 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:41.726008  6495 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.532s	user 0.003s	sys 0.000s
I20260709 13:01:41.705394  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:41.726195  6326 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:41.681358  6328 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 13:01:41.731680  6434 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 2.487s	user 0.006s	sys 0.013s
W20260709 13:01:41.740247  6519 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:01:41.725459  6397 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.745304  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 684) took 4811 ms. Trace:
I20260709 13:01:41.746196  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.733228  6326 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:41.758790  6326 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.760891  6326 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:41.754241  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 123189us
I20260709 13:01:41.766336  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:41.756278  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2777) took 2314 ms. Trace:
I20260709 13:01:41.767976  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:41.767130  1630 rpcz_store.cc:276] 0709 13:01:39.442126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.251354 (+1809228us) service_pool.cc:224] Handling call
0709 13:01:41.350837 (+ 99483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.376944 (+ 26107us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.377000 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:41.406301 (+ 29301us) log.cc:844] Serialized 26 byte log entry
0709 13:01:41.488764 (+ 82463us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:41.488832 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.488940 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.755931 (+266991us) raft_consensus.cc:1685] finished
0709 13:01:41.755955 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.756156 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25381,"raft.queue_time_us":25554,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":28926,"thread_start_us":72414,"threads_started":1,"wal-append.queue_time_us":230793}
I20260709 13:01:41.752444  6260 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.756963  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:41.770965  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.773334  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:41.747346  1561 rpcz_store.cc:276] 0709 13:01:36.933808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.934151 (+   343us) service_pool.cc:224] Handling call
0709 13:01:36.956420 (+ 22269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.348910 (+392490us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:01:37.597175 (+248265us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.895428 (+298253us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.896023 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:01:37.896173 (+   150us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:37.896205 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.736349 (+3840144us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400312254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:41.742527 (+  6178us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.744523 (+  1996us) raft_consensus.cc:1685] finished
0709 13:01:41.744640 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.744934 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":215239,"prepare.queue_time_us":327649,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":215844,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":10069,"spinlock_wait_cycles":311473536,"thread_start_us":472031,"threads_started":2,"wal-append.queue_time_us":437435,"wal-append.run_cpu_time_us":2337,"wal-append.run_wall_time_us":1060486}
I20260709 13:01:41.775993  6397 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:41.766640  6521 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:01:41.782733  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2780) took 2318 ms. Trace:
I20260709 13:01:41.783195  1629 rpcz_store.cc:276] 0709 13:01:39.464015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.318925 (+1854910us) service_pool.cc:224] Handling call
0709 13:01:41.411053 (+ 92128us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.427976 (+ 16923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.428013 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:41.428699 (+   686us) log.cc:844] Serialized 26 byte log entry
0709 13:01:41.539609 (+110910us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:41.539640 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.539699 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.782525 (+242826us) raft_consensus.cc:1685] finished
0709 13:01:41.782548 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.782670 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1478,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":145,"thread_start_us":110492,"threads_started":1,"wal-append.queue_time_us":231544}
I20260709 13:01:41.786037  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:41.787201  2350 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:41.788713  5494 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:41.788709  6419 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:41.790244  6419 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:41.790668  6419 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.792358  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:41.793342  6474 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.798607  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 797) took 2397 ms. Trace:
I20260709 13:01:41.804739  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:41.805864  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 134971us
I20260709 13:01:41.807467  2348 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:41.805084  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.799232  1701 rpcz_store.cc:276] 0709 13:01:39.401003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.440590 (+ 39587us) service_pool.cc:224] Handling call
0709 13:01:39.440975 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.676560 (+235585us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:39.676993 (+   433us) log.cc:844] Serialized 14 byte log entry
0709 13:01:40.456004 (+779011us) log.cc:844] Serialized 14 byte log entry
0709 13:01:40.456295 (+   291us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.456365 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:40.456445 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.224137 (+767692us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b1800428f20. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed557 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:41.519273 (+295136us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b1800428f20. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:41.797922 (+278649us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540036ddd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:41.797994 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.798331 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19968,"prepare.queue_time_us":1940,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":4756,"raft.queue_time_us":950,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":16556,"spinlock_wait_cycles":600085120,"thread_start_us":776899,"threads_started":1,"wal-append.queue_time_us":920902,"wal-append.run_cpu_time_us":2148,"wal-append.run_wall_time_us":1053507}
I20260709 13:01:41.824389  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:41.826615  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:41.831157  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.836045  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.839403  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.840026  1630 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:41.843734  6353 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.846191  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:41.847738  2346 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:41.854985  1632 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:41.858330  6168 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:01:41.869244  1628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2785) took 2350 ms (client timeout 1785 ms). Trace:
I20260709 13:01:41.862509  6277 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:41.869832  1628 rpcz_store.cc:269] 0709 13:01:39.519222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.323156 (+803934us) service_pool.cc:224] Handling call
0709 13:01:41.868794 (+1545638us) spinlock_profiling.cc:243] Waited 47 ms on lock 0x7b540017a7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:41.869015 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108072192}
I20260709 13:01:41.872412  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:41.875691  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:41.876751  1630 raft_consensus.cc:2410] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:01:41.876876  6455 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.877938  6155 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.878700  6155 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:41.879894  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2589) took 2223 ms. Trace:
I20260709 13:01:41.880311  1635 rpcz_store.cc:276] 0709 13:01:39.657004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.460516 (+1803512us) service_pool.cc:224] Handling call
0709 13:01:41.870408 (+409892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.878920 (+  8512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.878950 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.879033 (+    83us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:41.879090 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.879164 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.879708 (+   544us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b540017a7d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:41.879828 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941261568}
I20260709 13:01:41.881870  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:41.885558  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.862815  6168 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:01:41.896142  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 66569.4us
I20260709 13:01:41.895931  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:41.898130  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.898250  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 796) took 3390 ms. Trace:
I20260709 13:01:41.899284  6198 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.899480  1675 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:41.901268  6469 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:41.902513  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2678) took 3414 ms. Trace:
I20260709 13:01:41.896189  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:41.904672  2344 rpcz_store.cc:276] 0709 13:01:38.507732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.507961 (+   229us) service_pool.cc:224] Handling call
0709 13:01:38.508237 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.299594 (+791357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.891609 (+2592015us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54006310d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:41.898018 (+  6409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772393344}
I20260709 13:01:41.907780  1629 raft_consensus.cc:2410] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:01:41.902372  6168 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:41.906965  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.906867  1559 rpcz_store.cc:276] 0709 13:01:38.488569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.727952 (+2239383us) service_pool.cc:224] Handling call
0709 13:01:40.729173 (+  1221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.735956 (+  6783us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:40.736845 (+   889us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.017151 (+280306us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.673980 (+656829us) spinlock_profiling.cc:243] Waited 9.45 ms on lock 0x7b180030e500. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:41.901948 (+227968us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b180030e500. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:01:41.902120 (+   172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.902175 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:41.902237 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.902311 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.902452 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":841281,"prepare.queue_time_us":58337,"prepare.run_cpu_time_us":146,"prepare.run_wall_time_us":151,"raft.queue_time_us":13600,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":23642,"spinlock_wait_cycles":29197824,"thread_start_us":263934,"threads_started":1,"wal-append.queue_time_us":519207}
I20260709 13:01:41.915890  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:41.896595  6455 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:41.918033  1630 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:41.921866  5494 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:41.922515  5494 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:41.923185  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2675) took 3572 ms. Trace:
I20260709 13:01:41.923570  1556 rpcz_store.cc:276] 0709 13:01:38.351033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.019759 (+1668726us) service_pool.cc:224] Handling call
0709 13:01:40.020139 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.368679 (+348540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.918644 (+1549965us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b5400311d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:41.922986 (+  4342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799358336}
I20260709 13:01:41.923676  5494 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:41.927678  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 835465us
I20260709 13:01:41.922461  1628 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:41.930097  1635 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:41.905540  6155 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:41.943154  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:41.947211  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 117870us
W20260709 13:01:41.947793  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 48558.7us
I20260709 13:01:41.949661  6277 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:01:41.957340  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:41.960424  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 704) took 2521 ms. Trace:
I20260709 13:01:41.960913  1557 rpcz_store.cc:276] 0709 13:01:39.438827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.996078 (+1557251us) service_pool.cc:224] Handling call
0709 13:01:41.045506 (+ 49428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.046177 (+   671us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:41.253366 (+207189us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.959996 (+706630us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.960055 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:41.960125 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.960224 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.960349 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":585390,"prepare.queue_time_us":353302,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":78,"raft.queue_time_us":586134,"thread_start_us":211520,"threads_started":2,"wal-append.queue_time_us":223571}
I20260709 13:01:41.964521  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:41.965764  6224 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.966444  6224 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.967684  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2690) took 2166 ms. Trace:
I20260709 13:01:41.968875  6224 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:41.968099  1558 rpcz_store.cc:276] 0709 13:01:39.800930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.432617 (+1631687us) service_pool.cc:224] Handling call
0709 13:01:41.481675 (+ 49058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.482180 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:41.482625 (+   445us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.967318 (+484693us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.967374 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:41.967422 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.967495 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.967623 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":333411,"prepare.queue_time_us":964,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"thread_start_us":150687,"threads_started":1,"wal-append.queue_time_us":192319}
I20260709 13:01:41.966614  6182 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:41.946341  6455 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:01:41.946033  6368 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:41.970112  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.969954  1527 connection.cc:256] RPC call timeout handler was delayed by -1.0910933600000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
I20260709 13:01:41.971863  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2677) took 3556 ms. Trace:
I20260709 13:01:41.979748  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:41.971537  6182 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.990893  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:41.992086  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:41.970846  6198 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:41.979771  6368 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.975219  6455 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:41.996573  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:41.998920  1634 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.983518  1554 rpcz_store.cc:276] 0709 13:01:38.415709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.206384 (+1790675us) service_pool.cc:224] Handling call
0709 13:01:40.206839 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.207153 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:40.217564 (+ 10411us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.938971 (+1721407us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.971089 (+ 32118us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.971233 (+   144us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:41.971363 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.971475 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.971796 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":808513,"prepare.queue_time_us":46718,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":109,"raft.queue_time_us":816467,"raft.run_cpu_time_us":418,"raft.run_wall_time_us":21390,"thread_start_us":899489,"threads_started":1,"wal-append.queue_time_us":937783}
I20260709 13:01:41.991894  6182 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:42.001752  6162 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:42.001868  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:42.003439  6182 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:42.004599  1674 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:42.005721  6391 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:42.006521  6391 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.995679  6198 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:42.011826  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:42.012881  1627 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:42.014051  6198 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:41.997529  6455 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:41.997324  6368 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:42.021951  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.027097  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:42.027202  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:42.028185  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 74347.5us
I20260709 13:01:42.030915  6155 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:42.032150  6391 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:42.034320  6328 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:42.035292  6328 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:41.969457  6224 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:01:42.022511  6470 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:42.057564  6480 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:01:42.060496  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1053) took 2696 ms. Trace:
I20260709 13:01:42.061192  6488 raft_consensus.cc:3055] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:42.061894  6488 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:42.065541  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 100651us
I20260709 13:01:42.067947  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:42.070797  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:42.071725  1628 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
W20260709 13:01:42.077183  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 112816us
I20260709 13:01:42.085165  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:42.074194  1630 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:42.077839  6155 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:42.086673  1636 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:42.083418  1634 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
W20260709 13:01:42.080691  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 76216.4us
I20260709 13:01:42.087843  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:42.076864  6540 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:42.089634  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59460.3us
I20260709 13:01:42.100148  1629 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:42.102593  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:42.104878  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2684) took 2658 ms. Trace:
I20260709 13:01:42.105330  1552 rpcz_store.cc:276] 0709 13:01:39.446188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.996320 (+1550132us) service_pool.cc:224] Handling call
0709 13:01:42.103979 (+1107659us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b5800002798. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:01:42.104238 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.104517 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.104542 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:42.104593 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:42.104633 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.104688 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.104805 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5229952}
I20260709 13:01:42.079041  6277 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:01:42.114277  1628 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:42.114529  6540 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:42.115000  6200 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:42.115062  6540 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:42.115411  6529 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.217s	user 0.001s	sys 0.001s
I20260709 13:01:42.116349  6540 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:42.117622  6502 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.795s	user 0.007s	sys 0.000s
I20260709 13:01:42.117897  6540 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.121229  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2096) took 3654 ms. Trace:
I20260709 13:01:42.121866  1555 rpcz_store.cc:276] 0709 13:01:38.466916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.380748 (+1913832us) service_pool.cc:224] Handling call
0709 13:01:40.416657 (+ 35909us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.417127 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:40.417169 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:40.417825 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 13:01:41.223561 (+805736us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:41.223703 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.223826 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:42.116952 (+893126us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:42.120837 (+  3885us) raft_consensus.cc:1685] finished
0709 13:01:42.120858 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.121157 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":2700,"spinlock_wait_cycles":19869440,"thread_start_us":802107,"threads_started":1,"wal-append.queue_time_us":900787}
I20260709 13:01:42.125854  6480 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:42.126727  6353 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:42.127617  6488 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:42.129281  6488 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:42.140100  6530 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:01:42.142297  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 699) took 3193 ms. Trace:
I20260709 13:01:42.146000  6455 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:42.147575  6455 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:42.149775  6535 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:42.150887  6535 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:42.151387  6535 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:42.152491  6535 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:42.142800  1560 rpcz_store.cc:276] 0709 13:01:38.948964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.963479 (+2014515us) service_pool.cc:224] Handling call
0709 13:01:42.141378 (+1177899us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b5800002798. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5eee 0000000000375939 00007fda2c6291e0 00007fda2c6f20c4 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:01:42.141591 (+   213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.141889 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:42.141916 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:42.141961 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:42.142032 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.142095 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.142229 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6343552}
I20260709 13:01:42.155265  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.156610  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:42.061041  2351 rpcz_store.cc:276] 0709 13:01:39.363857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.370739 (+  6882us) service_pool.cc:224] Handling call
0709 13:01:39.371117 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.379142 (+  8025us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:39.411322 (+ 32180us) log.cc:844] Serialized 14 byte log entry
0709 13:01:40.483545 (+1072223us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b18003aba00. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:01:40.949846 (+466301us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.949955 (+   109us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:40.950087 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.059968 (+1109881us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b5400478954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:42.060044 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.060286 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24527,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":7014,"raft.queue_time_us":34021,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":23423,"spinlock_wait_cycles":20234880,"thread_start_us":465714,"threads_started":1,"wal-append.queue_time_us":1678328}
W20260709 13:01:42.185839  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.098668  6260 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.102783  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:42.074671  6162 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:01:42.211999  6260 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:42.212802  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 139530us
I20260709 13:01:42.213438  6308 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:42.214057  6308 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:42.215091  6308 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:42.215756  6260 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:42.216784  6162 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:42.210433  1558 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.216413  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:42.220297  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 146705us
I20260709 13:01:42.220228  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:42.220690  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2779) took 2761 ms (client timeout 1540 ms). Trace:
I20260709 13:01:42.225528  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:42.223658  1631 rpcz_store.cc:269] 0709 13:01:39.459535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.522613 (+ 63078us) service_pool.cc:224] Handling call
0709 13:01:42.168079 (+2645466us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b5400198054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:42.220601 (+ 52522us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4210960640}
W20260709 13:01:42.227149  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 124652us
I20260709 13:01:42.224927  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:42.228525  1630 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:42.223309  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 134292us
I20260709 13:01:42.220345  6182 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:42.251771  1632 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:01:42.252205  1632 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 8
I20260709 13:01:42.248056  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.253700  1560 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:42.249221  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.221246  6162 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:42.252455  1632 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 9.9
I20260709 13:01:42.258476  1632 raft_consensus.cc:2917] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Aborting config change with OpId 9.9: Aborted: Op aborted by new leader
I20260709 13:01:42.258957  1632 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:42.263157  6311 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:01:42.264480  6168 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.232045  6540 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:42.266415  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.252022  6200 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:42.255502  6328 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:42.254693  6224 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:42.274955  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:42.254953  6182 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:42.279280  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.290488  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:42.212095  6155 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:42.291954  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 206995us
I20260709 13:01:42.298211  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:42.298743  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.299263  1630 raft_consensus.cc:2368] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10.
I20260709 13:01:42.300595  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:42.300838  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2545) took 6776 ms. Trace:
I20260709 13:01:42.302122  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:42.302848  1701 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
W20260709 13:01:42.303594  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.301252  1632 rpcz_store.cc:276] 0709 13:01:35.519193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.854935 (+335742us) service_pool.cc:224] Handling call
0709 13:01:35.855248 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.376944 (+3521696us) spinlock_profiling.cc:243] Waited 5.21 ms on lock 0x7b54003401a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda2765806f 00007fda276d0f60 00007fda276d27de 00007fda276dcb49 00007fda276dd416 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62
0709 13:01:41.853503 (+2476559us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b58006062d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276360aa 00007fda276b8447 00007fda276d0fe7 00007fda276d27de 00007fda276dcb49 00007fda276dd416 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4
0709 13:01:42.262760 (+409257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.282395 (+ 19635us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b5400349ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:42.295905 (+ 13510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9385946624}
I20260709 13:01:42.308243  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:42.312515  1703 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:42.314664  6182 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:42.315847  6199 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:42.316191  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 54692us
I20260709 13:01:42.316412  6199 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:42.316578  6182 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.316874  6199 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.292717  6535 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.317948  6162 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:42.315410  6407 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.016s	sys 0.019s
W20260709 13:01:42.319476  6407 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.016s	sys 0.020s
W20260709 13:01:42.319869  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 91903.9us
I20260709 13:01:42.320426  1552 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:01:42.320854  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:42.301764  6200 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:42.316963  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:42.329423  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.329823  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:42.329463  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:42.304481  6319 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.580s	user 0.017s	sys 0.007s
I20260709 13:01:42.333244  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:42.338253  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:42.334497  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.334975  1632 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:42.301560  6224 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:42.347121  1555 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.344519  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:42.351500  1560 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
W20260709 13:01:42.345057  6157 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.720s	user 0.010s	sys 0.014s
I20260709 13:01:42.357815  6392 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:42.360023  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:42.360255  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:42.356525  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.328954  6200 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:42.306243  6198 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:42.367964  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.371654  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.381855  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:42.382715  1560 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:42.386085  6224 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:42.388284  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:42.389003  2350 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:42.389897  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:42.391649  2344 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:42.392124  2344 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:42.392606  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:42.393909  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:42.394381  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:42.394922  2348 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:42.395447  6182 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:42.395921  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:42.396559  2353 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEARNER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:42.396653  2346 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:42.395907  6182 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.398267  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:42.398803  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.393935  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:42.400218  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.341413  6319 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:42.402796  1708 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
W20260709 13:01:42.403223  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 95659.4us
I20260709 13:01:42.404435  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:42.407353  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.413208  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:42.393191  2344 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:42.376097  6326 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.614s	user 0.014s	sys 0.004s
I20260709 13:01:42.419082  2350 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
W20260709 13:01:42.420329  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 99637.9us
I20260709 13:01:42.422626  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:42.423036  2344 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:42.424678  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.426148  1552 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 9: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:42.428079  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:42.414382  6419 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.615s	user 0.019s	sys 0.002s
I20260709 13:01:42.432727  1527 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:42.406337  6182 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.420125  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.453132  1701 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: [8.8-11.10]
I20260709 13:01:42.456027  1557 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:42.448676  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.463038  1552 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:42.435343  6277 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.435595  6328 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:42.452111  2351 raft_consensus.cc:2376] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11. Re-sending same reply.
I20260709 13:01:42.451182  6407 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:42.478826  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.475003  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:42.482470  1707 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:42.440294  6537 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.487313  6341 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.487072  6537 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:42.487970  6341 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:42.488480  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:42.479905  6277 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:42.489578  6442 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:01:42.490764  6328 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:42.491308  6328 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:42.489662  2349 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
W20260709 13:01:42.491462  6544 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:01:42.438489  6419 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:42.496281  1552 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:42.531363  6419 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:42.534456  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.535281  1556 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
W20260709 13:01:42.537614  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 119581us
I20260709 13:01:42.538648  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:42.540233  2317 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:42.541364  6326 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:42.541806  6220 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:42.543452  6326 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:42.515774  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.546449  1557 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
W20260709 13:01:42.557015  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 68688.5us
I20260709 13:01:42.557585  1557 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:42.476024  6541 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:42.490787  6533 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 13:01:42.566973  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 148378us
I20260709 13:01:42.567754  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:42.551388  1554 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
W20260709 13:01:42.573207  6368 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.005s
I20260709 13:01:42.573769  6368 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:42.576066  6368 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:42.576751  2317 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:42.549623  5494 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:01:42.586678  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:42.596431  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.597946  2351 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:42.600582  1553 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:42.602512  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:42.603879  1557 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:42.607326  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 132464us
I20260709 13:01:42.607978  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.610113  1561 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:42.597347  6168 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:42.613344  6277 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:42.601439  6538 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:42.622143  6538 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:42.623570  6538 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:42.603026  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:42.633387  1555 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:42.626983  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:42.651386  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:42.662012  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.664173  6308 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.664892  6308 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:42.665901  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:42.666626  1704 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:42.666960  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:42.667815  6308 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:42.677531  6543 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:01:42.661943  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:42.675433  6168 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:42.671151  6539 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:42.656858  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:42.644735  1558 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
W20260709 13:01:42.708475  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 114306us
I20260709 13:01:42.709869  1560 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
W20260709 13:01:42.711656  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44038.6us
I20260709 13:01:42.696869  6541 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:42.716852  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:42.701016  6547 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:01:42.703593  6328 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:42.713902  6538 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:42.696986  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.639452  6537 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:42.712764  6368 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:42.708024  6326 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:42.714989  6198 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:42.713774  6442 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:42.721644  2344 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:01:42.709553  6539 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:42.738389  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 119495us
I20260709 13:01:42.722160  6488 raft_consensus.cc:2764] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:42.747236  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:42.739347  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 82608.5us
I20260709 13:01:42.719599  6260 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.715937  6540 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:42.750593  6311 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:42.751683  6168 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:42.752105  6539 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:42.761646  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.771883  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:42.744138  2344 pending_rounds.cc:85] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 8
I20260709 13:01:42.773008  2344 pending_rounds.cc:107] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:01:42.773352  2344 raft_consensus.cc:2889] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:42.734357  6541 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:42.775167  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:42.745370  6328 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.780251  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:42.740684  6326 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:42.780403  1556 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:01:42.776412  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.782451  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:42.783838  2317 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:42.780793  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 265067us
I20260709 13:01:42.786507  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:42.787644  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.790861  1553 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
W20260709 13:01:42.801944  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.778402  6311 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:42.829236  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54641.8us
I20260709 13:01:42.837193  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:42.783243  6326 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.782039  6488 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:42.783329  6168 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:42.790784  6539 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:42.857424  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:42.839879  6198 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:42.869812  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 147726us
I20260709 13:01:42.782806  6260 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:42.845674  6442 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:42.845964  6168 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:42.922119  1602 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:42.922161  6168 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:42.918793  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 192606us
I20260709 13:01:42.848704  6224 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:42.932531  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 153996us
W20260709 13:01:42.934245  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 92309.8us
W20260709 13:01:42.940558  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 160622us
I20260709 13:01:42.941634  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:42.918874  6470 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:01:42.959381  1708 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:01:42.950232  1525 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:42.943042  6368 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:42.929128  6540 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:01:42.939389  6277 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:42.951812  6455 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.804s	user 0.012s	sys 0.006s
W20260709 13:01:42.951171  6548 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.236s	user 0.002s	sys 0.000s
I20260709 13:01:42.974516  6542 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:42.966904  1527 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:42.950634  6200 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:42.869278  6541 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:42.965231  6311 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:43.005671  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 164927us
I20260709 13:01:42.998278  6085 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:43.006042  6442 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:01:43.038710  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 91514.6us
I20260709 13:01:43.056011  6568 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:43.056880  6568 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:01:43.058053  6551 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:01:43.058827  6319 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:43.061523  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 124279us
W20260709 13:01:43.064739  6553 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:43.065730  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:43.066303  6260 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:43.066676  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:43.067420  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:43.067924  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:43.069540  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:43.071269  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.073788  6200 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:43.075044  6200 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:43.076305  6470 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:43.077019  6470 consensus_queue.cc:1243] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:43.077536  6470 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:43.080538  6311 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:43.080243  6277 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:43.080960  6311 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:43.081804  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 115054us
I20260709 13:01:43.082011  6311 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:43.083367  6311 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:43.084264  1527 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:43.036689  6085 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:43.085856  6085 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:43.013365  6368 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:43.090411  6368 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:43.091044  6308 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:43.091495  6308 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:43.092352  6308 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:43.094861  6561 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:43.096573  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1252) took 1479 ms. Trace:
I20260709 13:01:43.096984  2349 rpcz_store.cc:276] 0709 13:01:41.616784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.621604 (+  4820us) service_pool.cc:224] Handling call
0709 13:01:43.096360 (+1474756us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b540054a454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:43.096482 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1990356736}
I20260709 13:01:43.099326  6224 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:43.100175  6534 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.785s	user 0.000s	sys 0.003s
I20260709 13:01:42.985697  6540 consensus_queue.cc:1243] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:01:43.070763  6442 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:43.104492  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:43.105613  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.101819  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:43.106083  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 164570us
W20260709 13:01:43.107928  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.021355  6392 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:43.011772  6455 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 13:01:43.121337  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:43.135076  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.138195  1632 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:01:43.138729  1632 pending_rounds.cc:85] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 5
I20260709 13:01:43.139022  1632 pending_rounds.cc:107] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:01:43.139307  1632 raft_consensus.cc:2889] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:43.142203  1702 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:43.128535  6414 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.144800  6414 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:43.147219  6414 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:43.133194  6308 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:43.149837  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 166625us
I20260709 13:01:43.169886  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:43.173884  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:43.175009  1631 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:01:43.176512  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 55291.9us
I20260709 13:01:43.178049  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:43.179685  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:43.126200  6392 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:43.176529  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 91037.1us
W20260709 13:01:43.151257  6182 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.681s	user 0.016s	sys 0.000s
I20260709 13:01:43.134101  6537 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:43.194262  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.201150  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:43.202134  1632 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:43.209211  6326 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:43.211966  6326 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:43.212728  6326 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:43.058115  6568 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:43.190618  6392 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:43.217844  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:43.218875  1631 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:43.222036  6559 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:43.224198  6559 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:43.224352  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.226728  6557 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.200345  6455 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:43.231274  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.231137  6085 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:43.231550  6277 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:43.233340  6199 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.916s	user 0.012s	sys 0.004s
I20260709 13:01:43.234000  6199 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:43.234720  1554 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:43.237454  6199 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:43.237708  6260 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:43.239841  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:43.240733  1555 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:43.241830  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:43.242656  1555 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:43.243603  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:43.243819  6162 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.244426  1555 raft_consensus.cc:2376] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9. Re-sending same reply.
I20260709 13:01:43.245983  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:43.246874  1556 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:43.248175  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:43.250222  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:43.251346  1557 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:43.252789  6353 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:43.252884  1553 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:43.200071  6542 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:43.262168  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.254204  1554 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:43.254947  6157 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:43.255218  6085 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:43.259738  6407 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:43.259960  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:43.266251  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 165428us
W20260709 13:01:43.266848  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 259817us
I20260709 13:01:43.270903  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:43.271656  2350 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:43.272198  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 102469us
W20260709 13:01:43.278308  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 214238us
I20260709 13:01:43.281574  1602 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:43.288343  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:43.288929  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:43.289659  1631 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.289953  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:43.290871  2349 raft_consensus.cc:2410] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:43.257714  6353 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:43.271045  6199 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.264782  6157 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:01:43.362190  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66952.6us
I20260709 13:01:43.298722  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:43.204702  6182 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:43.374174  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 78408.4us
W20260709 13:01:43.378038  2318 leader_election.cc:336] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.593s (SENT)
I20260709 13:01:43.383744  2350 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:01:43.454015  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 76001.7us
W20260709 13:01:43.455076  2318 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.602s (SENT)
I20260709 13:01:43.466727  1559 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:43.466903  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:43.297781  6407 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:43.467730  1706 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.468405  1708 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: [8.8-11.10]
I20260709 13:01:43.455646  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:43.476325  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 194881us
W20260709 13:01:43.473186  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 93043.4us
W20260709 13:01:43.482045  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 361237us
I20260709 13:01:43.379259  6277 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:43.483999  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:43.486984  1602 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:43.491791  1704 raft_consensus.cc:2410] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:43.490422  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.498347  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.379611  6199 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:43.520486  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.511777  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 56747.5us
I20260709 13:01:43.493515  6085 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:01:43.528776  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:43.530450  1704 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:43.457453  6564 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.128s
I20260709 13:01:43.502118  6442 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 13:01:43.550680  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 56590.9us
I20260709 13:01:43.502477  6572 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:43.502439  6575 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:43.521240  6556 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:43.553918  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 68463.2us
I20260709 13:01:43.569952  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.477273  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:01:43.585316  1703 raft_consensus.cc:606] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:43.586079  1703 raft_consensus.cc:614] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:43.550951  6573 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.565124  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.532301  6328 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:43.599603  6577 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:43.610579  6577 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:43.585101  6198 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.709s	user 0.013s	sys 0.001s
I20260709 13:01:43.613879  6198 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:43.577502  2317 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.562892  1602 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:43.508615  6157 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:43.588937  6539 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.733s	user 0.009s	sys 0.006s
W20260709 13:01:43.586984  6311 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.006s
I20260709 13:01:43.624437  1704 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:43.571424  6579 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.613262  6577 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:43.624811  1631 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:43.629818  1707 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:43.524619  6353 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:43.639904  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 77082.7us
W20260709 13:01:43.642637  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 72808.2us
I20260709 13:01:43.643626  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:43.623066  6168 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.697s	user 0.015s	sys 0.000s
I20260709 13:01:43.648100  6277 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:43.621517  6557 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:43.648777  6353 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.645670  6573 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:43.651463  6157 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:43.654352  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 97357.7us
I20260709 13:01:43.639098  6328 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:43.658553  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2127) took 1125 ms. Trace:
I20260709 13:01:43.658273  6328 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:43.658977  1559 rpcz_store.cc:276] 0709 13:01:42.533169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.533558 (+   389us) service_pool.cc:224] Handling call
0709 13:01:42.533977 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.467448 (+933471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.658257 (+190809us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b54002530d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:43.658486 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2144545664}
I20260709 13:01:43.661288  1674 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.662844  1674 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:43.663676  6539 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:43.665980  6539 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:43.616879  6407 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:43.667562  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:43.668740  6557 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:43.669904  6199 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:43.669528  6168 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:43.671644  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.666810  6539 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:43.693323  6576 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:01:43.702988  1704 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:43.678642  1602 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:43.693394  1558 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:43.628916  6575 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:43.695079  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:43.675740  1553 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:43.676815  6199 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:43.643703  6579 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:43.674286  6557 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:43.716893  1707 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:43.711191  6368 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.014s	sys 0.068s
I20260709 13:01:43.712589  6568 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:43.733618  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 187290us
I20260709 13:01:43.701747  1632 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:43.707176  1706 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:43.727871  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:43.764662  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 86156.4us
W20260709 13:01:43.775218  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47469.8us
I20260709 13:01:43.775417  1602 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:43.775765  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.776053  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.776831  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:43.766228  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 205942us
I20260709 13:01:43.767638  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:43.778594  2317 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:43.760356  6311 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:43.782593  1701 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:43.784395  1602 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:43.773098  6368 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.014s	sys 0.068s
I20260709 13:01:43.788256  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:43.791734  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.772663  6568 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:43.805006  6568 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:43.806034  6583 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:01:43.792965  6470 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:43.809583  6311 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:43.812587  6311 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:43.814529  6594 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:01:43.815307  6392 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.598s	user 0.008s	sys 0.004s
I20260709 13:01:43.776610  6442 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.782071  6277 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:43.786701  6575 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:43.834805  1555 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:43.786100  6569 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:43.839226  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.839709  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 168128us
I20260709 13:01:43.882257  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:43.838442  6162 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.004s
W20260709 13:01:43.886911  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 109062us
W20260709 13:01:43.887429  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47940.4us
I20260709 13:01:43.925625  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:43.888213  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 96528us
I20260709 13:01:43.889245  6572 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:43.834455  6574 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:43.922502  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2125) took 1390 ms. Trace:
I20260709 13:01:43.927026  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.929068  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.840837  6470 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:43.776608  6557 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:01:43.900452  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 121974us
I20260709 13:01:43.873389  6575 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.841337  6442 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:43.903101  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:43.838172  6277 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:43.815940  6392 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:43.916083  6200 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:43.916769  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:43.840929  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:43.896796  6260 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:43.933717  6569 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.941879  1635 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:43.944502  2318 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:43.927238  1552 rpcz_store.cc:276] 0709 13:01:42.532353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.566632 (+ 34279us) service_pool.cc:224] Handling call
0709 13:01:42.589451 (+ 22819us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:42.590035 (+   584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.590080 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:42.628581 (+ 38501us) log.cc:844] Serialized 26 byte log entry
0709 13:01:42.923329 (+294748us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:43.013051 (+ 89722us) log.cc:844] Serialized 14 byte log entry
0709 13:01:43.123762 (+110711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.123873 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:43.909847 (+785974us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b1800479440. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:43.921111 (+ 11264us) raft_consensus.cc:1685] finished
0709 13:01:43.921292 (+   181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.922382 (+  1090us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":485,"prepare.queue_time_us":201140,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":11488,"raft.queue_time_us":3405,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":19416,"spinlock_wait_cycles":39495936,"thread_start_us":397663,"threads_started":3,"wal-append.queue_time_us":472968}
I20260709 13:01:43.946089  1705 raft_consensus.cc:2410] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:01:43.921955  6168 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:43.832412  6570 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:43.956655  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:43.980721  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.983088  2317 leader_election.cc:400] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10.
I20260709 13:01:44.008441  2317 leader_election.cc:403] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:01:44.012104  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 227836us
W20260709 13:01:43.972450  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.999167  6574 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:43.946089  6577 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:44.023798  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 94942us
I20260709 13:01:44.024849  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:43.952559  6277 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:43.969926  6602 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:44.002920  6200 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:43.946672  6575 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.031544  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:44.022845  6155 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:44.028712  6607 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:44.033872  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:01:44.044647  1709 raft_consensus.cc:606] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:44.043099  1561 raft_consensus.cc:2376] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11. Re-sending same reply.
I20260709 13:01:44.035274  6392 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:44.040192  6590 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 13:01:44.047611  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 96318.8us
I20260709 13:01:43.999585  6260 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:44.049615  6581 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:01:44.049949  6260 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:44.051024  6541 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:01:44.042219  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 97850.4us
I20260709 13:01:44.042219  6407 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:44.041513  6579 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:44.015990  6606 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:44.168170  6606 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:44.168704  6606 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:44.029510  6561 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.169899  6561 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:44.172778  6561 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:44.062585  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 49673.7us
I20260709 13:01:44.174362  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.175181  1635 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:44.175568  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:44.001219  6569 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:44.177135  6607 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:44.110177  6470 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:44.178735  6569 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:44.174404  6200 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:01:44.179622  6470 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.022640  6570 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:44.180235  6607 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.029215  6577 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.182823  6570 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:44.183758  6577 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:44.042837  6162 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:01:44.055953  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:44.159176  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 125826us
W20260709 13:01:44.164036  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 115560us
I20260709 13:01:44.186776  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:44.190476  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.191273  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.063468  6155 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 13:01:44.163359  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 109554us
I20260709 13:01:44.165802  6597 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:44.166499  6592 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.157s	user 0.000s	sys 0.005s
I20260709 13:01:44.175652  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:44.195147  1630 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:44.033324  6168 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10.
I20260709 13:01:44.185940  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.194185  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:44.197746  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 142535us
I20260709 13:01:44.199797  6611 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.200830  6341 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:44.201100  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:44.201543  6341 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:44.189033  6157 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.012s
I20260709 13:01:44.201969  6568 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.202010  6341 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:44.204063  6341 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.204015  6407 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.199847  1635 raft_consensus.cc:2410] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:44.207027  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.208153  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.208933  1561 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:44.208210  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:44.210376  1633 raft_consensus.cc:2410] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:01:44.210825  1554 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:01:44.211431  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.219012  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:44.211766  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:44.220486  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:44.220355  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:44.222391  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.222473  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.223500  1561 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:44.224814  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:44.225054  1629 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:01:44.219702  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:44.226019  1557 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:44.227731  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:44.228351  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.228559  2344 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 7.
W20260709 13:01:44.229707  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.231899  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:44.232867  2353 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:44.233736  1527 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:44.227735  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.235139  1560 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:44.236733  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.225414  1629 pending_rounds.cc:85] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 5
I20260709 13:01:44.238415  1629 pending_rounds.cc:107] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:01:44.238804  1629 raft_consensus.cc:2889] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:44.239259  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:44.240674  1627 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:44.240887  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:44.244707  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.245872  1552 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:44.247977  2351 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:01:44.248589  2351 pending_rounds.cc:85] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 9
I20260709 13:01:44.248831  2351 pending_rounds.cc:107] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:01:44.249130  2351 raft_consensus.cc:2889] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:44.253063  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.211491  6570 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.255210  1633 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:44.255626  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:44.255580  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:44.218895  6569 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:44.255589  2344 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:44.259125  1704 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:44.219162  6561 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.266541  6610 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:44.267143  6610 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.266995  6561 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:44.267611  6610 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:44.269681  6605 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.263399  6260 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.270335  6605 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.270594  6260 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.272929  6260 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:44.263623  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.273557  6260 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:44.273775  6570 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:44.274423  1631 raft_consensus.cc:2410] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:01:44.275583  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.276218  6570 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:44.276772  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.276925  6570 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:44.278568  2351 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:44.238138  6602 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.282007  6602 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.283217  6561 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:44.284103  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:44.284411  6602 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:44.229391  6579 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:44.266404  6613 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:44.285351  6579 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:44.269899  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:44.286083  6589 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:44.264335  6392 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:44.293782  6610 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:44.285791  6613 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:44.296018  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 72565.6us
I20260709 13:01:44.296154  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:44.286458  6579 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.298605  2346 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:44.298753  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:44.299083  6608 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:44.299503  6608 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:44.289812  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:01:44.299923  6608 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:44.300310  1709 raft_consensus.cc:606] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:44.300740  1709 raft_consensus.cc:614] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:44.265839  2345 raft_consensus.cc:2410] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:44.285440  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:44.267161  2348 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
W20260709 13:01:44.294823  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 61139us
I20260709 13:01:44.303010  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.267552  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.304092  1633 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 10: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:44.309962  1636 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.010s	sys 0.032s
W20260709 13:01:44.314502  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.314911  2352 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:01:44.318759  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:44.321918  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.306111  6570 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:44.316403  2352 pending_rounds.cc:85] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 9
I20260709 13:01:44.327445  2352 pending_rounds.cc:107] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:01:44.290712  6569 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.290365  6557 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:44.326742  1561 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:44.327821  2352 raft_consensus.cc:2889] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:44.308725  6311 raft_consensus.cc:2764] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:44.285605  6609 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:44.305783  6392 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:44.325872  6610 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:44.328842  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.352617  1629 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:44.293238  6606 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.356153  6606 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:44.326326  6588 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:01:44.306304  6575 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.359611  6568 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:44.333204  6200 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:44.350369  6602 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:01:44.362192  6570 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.329741  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:44.329783  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:44.364845  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.288697  6574 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:44.309285  6561 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:44.335628  1554 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:44.313632  6611 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:44.307397  6613 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:44.338017  1552 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41150
I20260709 13:01:44.339011  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:44.369369  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.370128  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.336987  6569 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:44.368238  1552 raft_consensus.cc:606] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Received request to transfer leadership
I20260709 13:01:44.374915  1633 raft_consensus.cc:2368] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13.
I20260709 13:01:44.366108  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:44.368969  1553 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:44.381037  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:44.382500  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 162901us
I20260709 13:01:44.384425  1628 raft_consensus.cc:2410] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:01:44.385222  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.294274  6589 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:44.352353  6392 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:44.375849  6561 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:44.377286  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.413298  1629 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:44.402174  6611 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:44.402550  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.403859  6157 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:44.404629  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:44.405766  6591 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:44.406272  6589 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:44.408275  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:44.389865  6568 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:44.381950  6609 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:44.375669  6602 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:44.446072  1629 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:44.448477  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 225218us
I20260709 13:01:44.454387  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.455489  1629 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
W20260709 13:01:44.460140  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 160051us
I20260709 13:01:44.383620  6610 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
W20260709 13:01:44.446094  6556 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.859s	user 0.009s	sys 0.007s
I20260709 13:01:44.443497  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2809) took 1317 ms. Trace:
I20260709 13:01:44.426894  6157 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:44.425803  6575 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:44.476436  1674 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:44.445243  6602 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.420668  1634 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:44.441983  6606 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:44.443755  6568 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.474372  6368 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.445070  6414 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:44.438612  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:44.502475  2350 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:44.481074  1636 rpcz_store.cc:276] 0709 13:01:43.126217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.126379 (+   162us) service_pool.cc:224] Handling call
0709 13:01:43.164840 (+ 38461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.191635 (+ 26795us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:43.761192 (+569557us) log.cc:844] Serialized 14 byte log entry
0709 13:01:44.442957 (+681765us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:44.443038 (+    81us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:44.443105 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.443222 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.443410 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16980,"prepare.queue_time_us":423134,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":79112,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":105,"thread_start_us":1144287,"threads_started":3,"wal-append.queue_time_us":592135}
I20260709 13:01:44.487270  6556 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:44.505755  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.486976  6157 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:44.501129  6610 consensus_queue.cc:1243] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:44.501852  6602 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.512804  6568 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.505064  6606 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.569294  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:44.480484  6277 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:44.528162  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 157996us
W20260709 13:01:44.600337  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 144942us
I20260709 13:01:44.600543  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.601184  1636 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:44.614877  6622 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.615506  6622 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.616175  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:44.617942  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:44.504278  6617 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 13:01:44.588107  6606 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.638095  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.639178  1636 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:44.527086  6368 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.646301  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.613024  6579 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:44.664850  6612 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
W20260709 13:01:44.665963  6614 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:01:44.522954  6561 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:44.640558  6613 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:44.680932  6613 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.681371  6613 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:44.675767  1627 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:44.676894  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:44.680277  6556 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:44.597831  6608 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.730816  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:44.731846  1558 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.741375  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:44.743744  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.747716  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.708433  6407 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.713892  6622 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:44.674486  6579 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.766543  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.604880  6609 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:44.770437  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 179612us
I20260709 13:01:44.782011  2317 leader_election.cc:400] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13.
I20260709 13:01:44.782562  2317 leader_election.cc:403] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:01:44.763180  6561 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:44.791414  1554 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
W20260709 13:01:44.800406  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.820425  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 202715us
I20260709 13:01:44.836838  1558 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:01:44.816489  6622 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.845994  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.766202  6407 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:44.855540  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 73691.8us
I20260709 13:01:44.792721  6613 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.866331  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:44.864073  1554 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
W20260709 13:01:44.865927  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.868394  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.872300  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:44.828981  6607 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:01:44.836354  6623 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:01:44.854806  6277 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:44.878837  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:44.880580  1703 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:44.878903  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:44.882174  1707 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:01:44.879706  1708 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:44.859560  6622 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.883507  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:44.885533  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:44.887971  6555 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.710s	user 0.026s	sys 0.001s
W20260709 13:01:44.888463  6625 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 13:01:44.888850  6620 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.184s	user 0.000s	sys 0.002s
W20260709 13:01:44.891376  6626 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:01:44.898034  6368 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:44.901721  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.903507  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.904832  6643 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:01:44.886593  6572 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.954s	user 0.006s	sys 0.009s
I20260709 13:01:44.897473  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.908164  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:44.927047  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:44.928170  1707 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:01:44.933694  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:44.934127  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:44.934805  1554 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:44.893687  6591 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.895041  6277 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.905791  6608 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:44.905438  6643 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:44.945628  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.949079  6630 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:44.950013  1630 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:44.949916  6630 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:44.953197  1630 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 13:01:44.953895  6632 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:01:44.932716  6572 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:44.959648  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:44.962301  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:44.962785  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.973981  1636 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:44.977345  6407 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:44.978065  6613 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:44.979338  1636 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:44.982007  2346 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:44.982626  6622 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:44.990878  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 802) took 2680 ms (client timeout 1602 ms). Trace:
W20260709 13:01:44.991534  1702 rpcz_store.cc:269] 0709 13:01:42.289861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.290058 (+   197us) service_pool.cc:224] Handling call
0709 13:01:44.970320 (+2680262us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54002e95d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:44.970454 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1627668224}
I20260709 13:01:44.996570  6659 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:44.997632  6659 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:44.998302  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.999513  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2086) took 2701 ms. Trace:
I20260709 13:01:44.999675  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:45.000442  6659 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.000977  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:44.945070  6277 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.008050  6277 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:45.000198  1710 rpcz_store.cc:276] 0709 13:01:42.297956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.330643 (+ 32687us) service_pool.cc:224] Handling call
0709 13:01:42.331077 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:42.341986 (+ 10909us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:42.496424 (+154438us) log.cc:844] Serialized 14 byte log entry
0709 13:01:42.950777 (+454353us) log.cc:844] Serialized 14 byte log entry
0709 13:01:42.964483 (+ 13706us) log.cc:844] Serialized 14 byte log entry
0709 13:01:42.998732 (+ 34249us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.007154 (+  8422us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.007349 (+   195us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:43.007420 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.007515 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.828902 (+1821387us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b24001f90e8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda244739c4 00007fda24471acb 00007fda2462cdc9 00007fda2462ca55 00007fda2462c09d 00007fda2725e10e 00007fda2727588c 00007fda27274bb6 00007fda272747d8 00007fda2727e221
0709 13:01:44.999207 (+170305us) raft_consensus.cc:1685] finished
0709 13:01:44.999242 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.999430 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67484,"prepare.queue_time_us":75200,"prepare.run_cpu_time_us":147,"prepare.run_wall_time_us":154,"raft.queue_time_us":65983,"raft.run_cpu_time_us":603,"raft.run_wall_time_us":38432,"spinlock_wait_cycles":62359808,"thread_start_us":447212,"threads_started":2,"wal-append.queue_time_us":558021}
I20260709 13:01:44.977399  6643 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.014662  1707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:01:45.017220  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:45.017499  6572 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:45.017993  2351 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 9.
I20260709 13:01:45.030493  1702 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:45.030511  1554 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:45.025740  6659 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:45.039091  6311 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:45.039630  6311 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:45.040532  6311 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.041590  6311 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:45.046236  6567 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:45.046958  6567 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:01:45.049278  6634 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.299s	user 0.000s	sys 0.002s
I20260709 13:01:45.051026  6638 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:01:45.039458  6659 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:45.053373  6652 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.098s	user 0.002s	sys 0.000s
I20260709 13:01:45.058422  6573 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.068538  1556 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:45.019152  6572 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:01:45.082441  6630 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:45.085929  2353 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:45.033185  6606 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.098623  6606 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:45.104174  6540 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:45.127197  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:45.128203  1560 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:45.137898  1554 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:45.123853  6540 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:45.146826  6628 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:45.149683  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2083) took 2854 ms (client timeout 1537 ms). Trace:
I20260709 13:01:45.149300  6540 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:45.150139  1705 rpcz_store.cc:269] 0709 13:01:42.295299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.295437 (+   138us) service_pool.cc:224] Handling call
0709 13:01:45.149424 (+2853987us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540014dcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:45.149603 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3763621120}
I20260709 13:01:45.151592  6182 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:45.154502  6587 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.384s	user 0.004s	sys 0.005s
I20260709 13:01:45.154969  1635 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:45.155198  6605 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:45.157876  1560 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:45.161504  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:45.168045  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.171751  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:45.177568  1560 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:01:45.178951  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:45.169885  6182 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:45.188148  6182 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.189567  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:45.189858  6182 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:45.190843  6200 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:45.191463  6200 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:45.176095  6605 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:45.194253  6568 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.592s	user 0.019s	sys 0.000s
I20260709 13:01:45.194829  6568 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:45.195755  6631 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.373s	user 0.000s	sys 0.001s
I20260709 13:01:45.197515  6568 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:45.197788  6308 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:45.197517  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:45.199659  2348 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:45.201177  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 827) took 1834 ms. Trace:
I20260709 13:01:45.201648  2349 rpcz_store.cc:276] 0709 13:01:43.367396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.367742 (+   346us) service_pool.cc:224] Handling call
0709 13:01:43.384248 (+ 16506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.455537 (+ 71289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.455586 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.494275 (+ 38689us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.625912 (+131637us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:43.625961 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.626036 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:45.153817 (+1527781us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:45.200818 (+ 47001us) raft_consensus.cc:1685] finished
0709 13:01:45.200855 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.201089 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10991,"raft.queue_time_us":54821,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":24336,"spinlock_wait_cycles":9706240,"thread_start_us":93939,"threads_started":1,"wal-append.queue_time_us":202606}
I20260709 13:01:45.189621  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:45.177906  6198 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:01:45.209009  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:45.209756  2349 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 9.
W20260709 13:01:45.211581  6569 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.802s	user 0.014s	sys 0.008s
I20260709 13:01:45.212148  6569 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.214913  6442 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:45.186219  6392 consensus_meta.cc:287] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.747s	user 0.019s	sys 0.002s
I20260709 13:01:45.226498  1602 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:45.235364  6569 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:45.241005  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:45.236934  6611 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:45.246050  6611 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.246503  6611 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:45.253684  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 53577us
W20260709 13:01:45.279058  6653 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.297s	user 0.005s	sys 0.000s
I20260709 13:01:45.238406  6591 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13.
I20260709 13:01:45.230619  6392 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.255131  6659 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:45.298233  6157 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.723s	user 0.009s	sys 0.010s
I20260709 13:01:45.314097  6157 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.295805  6326 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:45.315303  6326 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.316758  6157 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:45.317719  6326 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:45.314465  6591 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:45.319237  1631 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:01:45.231092  6567 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.230443  6663 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:01:45.322139  2347 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:01:45.295943  6182 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:45.335204  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.338740  6198 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.343291  6198 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.345803  6645 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:45.347608  6085 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:45.347561  6645 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:45.348325  6085 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:45.350575  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.354091  6085 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:45.360337  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:45.360973  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:45.361452  1552 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:45.367893  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:45.368330  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2593) took 1193 ms. Trace:
I20260709 13:01:45.368937  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:45.369293  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.368813  6573 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:45.370523  6573 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:45.369655  1558 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:45.371028  6573 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:45.374058  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 717) took 2136 ms. Trace:
I20260709 13:01:45.374887  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2594) took 1181 ms. Trace:
I20260709 13:01:45.375259  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 831) took 1143 ms. Trace:
I20260709 13:01:45.375659  2352 rpcz_store.cc:276] 0709 13:01:44.231861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.241200 (+  9339us) service_pool.cc:224] Handling call
0709 13:01:44.307962 (+ 66762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.350845 (+ 42883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.375070 (+1024225us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b54004bf0d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:45.375188 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100634496}
I20260709 13:01:45.369441  1634 rpcz_store.cc:276] 0709 13:01:44.175148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.175290 (+   142us) service_pool.cc:224] Handling call
0709 13:01:45.368063 (+1192773us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54002db9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:45.368199 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479388032}
I20260709 13:01:45.377789  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:45.378391  6629 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.513s	user 0.002s	sys 0.000s
I20260709 13:01:45.378651  1634 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:45.379612  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:45.380774  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.381873  6540 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:45.381985  6085 consensus_queue.cc:1059] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:45.382234  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 732) took 1056 ms. Trace:
I20260709 13:01:45.383208  6085 raft_consensus.cc:3055] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:45.383751  6085 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:45.386636  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.386945  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2723) took 1472 ms. Trace:
I20260709 13:01:45.387351  1704 rpcz_store.cc:276] 0709 13:01:43.914525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.917600 (+  3075us) service_pool.cc:224] Handling call
0709 13:01:43.917920 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.273682 (+355762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.386720 (+1113038us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54001e8f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:45.386886 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":777372544}
I20260709 13:01:45.388046  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.388681  6579 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:45.374490  1555 rpcz_store.cc:276] 0709 13:01:43.237429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.245367 (+  7938us) service_pool.cc:224] Handling call
0709 13:01:43.264109 (+ 18742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.885414 (+621305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.373797 (+1488383us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54002ffb54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:45.373990 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311739776}
I20260709 13:01:45.383625  2317 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:45.390187  6579 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:45.324314  6591 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.383173  1561 rpcz_store.cc:276] 0709 13:01:44.325529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.351667 (+ 26138us) service_pool.cc:224] Handling call
0709 13:01:44.352086 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:44.352360 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:44.352443 (+    83us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:44.363597 (+ 11154us) log.cc:844] Serialized 26 byte log entry
0709 13:01:44.587219 (+223622us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:44.587270 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.587368 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:45.377578 (+790210us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:45.381881 (+  4303us) raft_consensus.cc:1685] finished
0709 13:01:45.381910 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.382166 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6050560,"thread_start_us":223199,"threads_started":1,"wal-append.queue_time_us":450176}
I20260709 13:01:45.392926  6591 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:45.393594  1559 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:45.394665  6611 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:45.396136  6602 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.807s	user 0.005s	sys 0.014s
I20260709 13:01:45.396625  1630 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:45.396941  6611 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.397666  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.398298  1630 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:01:45.399468  6643 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.399649  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:45.399917  6200 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.400408  2353 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:45.400113  6643 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.402812  2352 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:45.405581  5494 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:45.406214  5494 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.406626  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:45.407609  6260 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.408403  5494 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:45.408311  6260 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.408855  6613 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:45.387868  6561 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.567s	user 0.007s	sys 0.010s
I20260709 13:01:45.413909  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:45.415205  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.409947  1636 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 10: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:45.407646  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:45.375299  1628 rpcz_store.cc:276] 0709 13:01:44.193745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.194217 (+   472us) service_pool.cc:224] Handling call
0709 13:01:45.374691 (+1180474us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b540010c2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:45.374825 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364875136}
I20260709 13:01:45.416445  6597 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:45.410585  1632 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:45.474257  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.395058  1559 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:45.417834  6571 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:45.476761  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 63007.6us
I20260709 13:01:45.422472  1459 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 7 to 8, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 8 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:45.477146  6571 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.478045  1559 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:45.478426  1559 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:45.479226  1559 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.479821  6571 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.374645  6392 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:45.431908  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 64116.4us
I20260709 13:01:45.388828  6605 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:45.481616  6597 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:45.481848  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:45.482178  6597 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:45.418080  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1265) took 1195 ms. Trace:
I20260709 13:01:45.471841  1557 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:45.483042  2349 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:45.420986  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 728) took 1203 ms. Trace:
I20260709 13:01:45.484174  1556 rpcz_store.cc:276] 0709 13:01:44.217845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.218094 (+   249us) service_pool.cc:224] Handling call
0709 13:01:45.420730 (+1202636us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b5400304dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:45.420900 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1930312576}
W20260709 13:01:45.474397  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66824.3us
I20260709 13:01:45.475091  1704 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:45.381384  1634 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:45.485855  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.483096  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:45.487612  2346 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:45.488665  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:45.482882  2350 rpcz_store.cc:276] 0709 13:01:44.223046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.223176 (+   130us) service_pool.cc:224] Handling call
0709 13:01:44.223470 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.519902 (+296432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.417804 (+897902us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b5400762d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:45.418011 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641006464}
W20260709 13:01:45.489189  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.489817  1526 connection.cc:256] RPC call timeout handler was delayed by -1.6040327380000008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
I20260709 13:01:45.489979  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:45.410218  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 880) took 1097 ms. Trace:
I20260709 13:01:45.430138  6260 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.428934  6407 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:45.480388  1559 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:45.418171  6643 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.418457  6319 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:45.409343  6613 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:45.490828  2350 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:45.486752  1709 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:45.503096  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:45.503767  1558 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:01:45.504753  1555 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:45.497659  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.430002  6574 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:01:45.499491  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.503944  1709 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:45.498957  2353 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:45.385135  6085 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.418581  6619 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:45.530046  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.482719  6597 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:45.507601  1557 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41150
I20260709 13:01:45.495594  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:45.539422  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.490999  1633 rpcz_store.cc:276] 0709 13:01:44.312649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.394660 (+ 82011us) service_pool.cc:224] Handling call
0709 13:01:45.409989 (+1015329us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b54002dcdd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:45.410151 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752886144}
I20260709 13:01:45.540827  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.507357  1552 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:01:45.421233  6538 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.419000  6637 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:45.510397  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:45.532140  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:45.531414  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:45.562111  1706 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:45.530946  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.492939  6455 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:45.488591  6199 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:45.396633  6602 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.496876  6407 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:45.536309  6260 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:45.509196  6319 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.444659  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.546990  1552 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:45.540180  1704 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 8.
I20260709 13:01:45.562918  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.558211  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:45.577621  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:45.578472  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:45.505430  6613 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:45.567775  1705 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:45.579954  2350 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:45.581969  1556 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:45.582231  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.582470  1556 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:45.541606  6574 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:45.582509  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:45.584749  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:45.560885  6541 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.560902  6353 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.586989  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:45.570664  6611 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.568336  6455 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:45.588878  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:45.591157  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.598649  1675 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:45.603472  1702 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:45.595177  1556 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.606312  1552 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:45.585848  6569 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:45.585736  6162 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:45.592527  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.593405  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.566875  6645 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:45.583158  6368 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:45.574857  6407 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:45.585417  6085 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:45.566475  6619 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.580175  6199 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.611826  1556 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:45.570575  6538 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.615140  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.613291  1633 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:01:45.600652  6455 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:45.625844  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:45.589825  6200 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:45.631898  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.633538  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:45.634219  1633 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:45.635663  6200 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:45.636502  6200 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.639032  6200 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:45.589178  6392 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:45.639863  6392 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.640250  6392 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:45.642274  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.613044  6162 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.644368  6162 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:45.573805  6637 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.625022  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:45.646772  6419 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:45.647234  6419 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:45.647610  6637 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:45.647636  6419 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:45.583763  6260 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.609135  6649 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:45.609604  6618 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:45.596060  6353 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.592121  1706 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:45.596218  6541 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.671597  1559 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:45.607947  6659 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:01:45.680122  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 875) took 1374 ms. Trace:
I20260709 13:01:45.680619  1632 rpcz_store.cc:276] 0709 13:01:44.306216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.774830 (+468614us) service_pool.cc:224] Handling call
0709 13:01:45.410380 (+635550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.474814 (+ 64434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.679876 (+205062us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54002dcdd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:45.680050 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460393216}
I20260709 13:01:45.614261  6368 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:45.658649  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:45.690246  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2711) took 1088 ms. Trace:
I20260709 13:01:45.690629  1555 rpcz_store.cc:276] 0709 13:01:44.601907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.389975 (+788068us) service_pool.cc:224] Handling call
0709 13:01:45.504570 (+114595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.505174 (+   604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.681057 (+175883us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54001ce150. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:45.690166 (+  9109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262754688}
I20260709 13:01:45.691217  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:45.601408  6611 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.675051  1706 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:45.612197  6630 consensus_queue.cc:1059] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:01:45.684495  6538 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:45.671764  6573 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.686868  6353 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.687968  6162 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:45.726374  6677 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:01:45.656417  6618 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.727293  6618 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:45.728075  6162 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:45.729179  6319 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:45.731714  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 735) took 1201 ms. Trace:
I20260709 13:01:45.732134  1554 rpcz_store.cc:276] 0709 13:01:44.529845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.249442 (+719597us) service_pool.cc:224] Handling call
0709 13:01:45.249877 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.250326 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.250356 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.253424 (+  3068us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.565730 (+312306us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:45.565780 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.565858 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:45.731458 (+165600us) raft_consensus.cc:1685] finished
0709 13:01:45.731481 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.731647 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":2633,"thread_start_us":283845,"threads_started":1,"wal-append.queue_time_us":448031}
I20260709 13:01:45.695266  1556 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:45.734023  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:45.734745  1554 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:45.702675  1701 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:45.736536  6619 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.736634  6573 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.671061  6200 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:45.737694  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.737820  6200 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.705287  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:45.738811  6260 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:45.739501  6260 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:45.740463  6200 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:45.741185  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.741811  6619 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:45.742784  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1268) took 1512 ms. Trace:
I20260709 13:01:45.743093  6619 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:45.743243  2344 rpcz_store.cc:276] 0709 13:01:44.230489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.236231 (+  5742us) service_pool.cc:224] Handling call
0709 13:01:44.240949 (+  4718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.265339 (+ 24390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.742495 (+1477156us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b5400796b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:45.742714 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31469440}
I20260709 13:01:45.744339  6538 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:45.744765  6538 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:45.745245  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:45.746001  1553 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:45.745749  6538 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.747157  6538 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:45.747388  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:45.748133  1553 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:45.750758  6155 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.751397  6155 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.752038  1704 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:45.752393  6368 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:45.620987  6574 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.737866  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.716579  1708 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:45.674901  6341 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:45.665274  6570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.842s	user 0.008s	sys 0.030s
I20260709 13:01:45.755715  1627 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:45.734404  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:45.750802  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:45.674826  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:45.755069  6574 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:45.762440  1633 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:45.728150  6541 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:45.746107  6630 raft_consensus.cc:3055] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:45.711994  1552 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:45.758687  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:45.709877  1709 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:01:45.719682  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:45.772857  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:45.773102  1632 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:45.756711  1556 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:45.689764  6602 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:45.779659  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:45.781904  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:45.759902  6570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.008s	sys 0.030s
I20260709 13:01:45.798092  1709 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:45.807572  1556 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:45.810294  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:45.805037  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 323233us
I20260709 13:01:45.814774  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.817250  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:45.771011  6648 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:45.820137  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.822057  1710 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:45.821604  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.823482  1634 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:45.825598  1554 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:45.771821  6630 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:45.830848  1709 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:45.783001  6326 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:01:45.794234  6541 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:45.832455  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.833510  6455 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.860095  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.806296  6155 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.838300  6597 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.839581  6198 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.840216  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:45.842769  6419 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:45.844120  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:45.777124  6606 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
W20260709 13:01:45.878940  1560 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.003s	sys 0.005s
W20260709 13:01:45.879367  1560 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.003s	sys 0.005s
I20260709 13:01:45.849370  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:45.857568  6591 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.859191  6611 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:45.856667  6392 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.837828  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:45.863715  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:45.867489  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.887905  1705 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:45.867695  6085 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:45.869130  1701 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:45.845502  1552 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
W20260709 13:01:45.863523  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46385.9us
I20260709 13:01:45.877858  2345 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:45.820639  6260 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:45.849076  1707 raft_consensus.cc:606] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:45.880595  1632 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:45.882616  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:45.867444  6541 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.889040  2350 raft_consensus.cc:2368] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10.
I20260709 13:01:45.869930  1634 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:01:45.898568  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.887297  1526 connection.cc:256] RPC call timeout handler was delayed by -1.3196912070000053s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
I20260709 13:01:45.890241  2353 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:45.891816  1557 raft_consensus.cc:606] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Received request to transfer leadership
I20260709 13:01:45.902554  1557 raft_consensus.cc:614] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:45.901881  1526 connection.cc:256] RPC call timeout handler was delayed by -1.1540327380000122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2096s for a response
W20260709 13:01:45.903935  1526 connection.cc:256] RPC call timeout handler was delayed by -1.1361277650000261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
I20260709 13:01:45.868798  6648 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:45.884178  6606 consensus_queue.cc:1243] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:45.906664  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.909052  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:45.907336  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:45.911063  2346 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:45.920301  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:45.923720  1634 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:45.911850  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2139) took 1387 ms. Trace:
W20260709 13:01:45.916669  1561 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.039s	sys 0.040s
W20260709 13:01:45.928011  1561 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.039s	sys 0.040s
I20260709 13:01:45.928117  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:45.935006  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 439515us
I20260709 13:01:45.948545  1627 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:45.958925  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:45.903343  6541 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:46.025125  1553 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:45.961450  6407 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:45.998188  2317 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:46.021685  6570 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:46.044448  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:46.101663  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 57367.5us
I20260709 13:01:45.943601  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:45.952180  1526 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:45.894147  6260 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.910100  6679 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:01:45.910544  6630 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.057660  1558 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:45.948119  6155 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.926806  6611 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.886027  6591 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.021175  1602 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:45.956024  1634 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:45.947244  6085 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:46.009140  1632 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
W20260709 13:01:46.151192  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 62603.9us
W20260709 13:01:46.178067  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 231496us
I20260709 13:01:46.070909  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:46.178953  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:46.028203  6564 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
W20260709 13:01:46.068127  6678 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:01:46.181531  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 889) took 1023 ms. Trace:
W20260709 13:01:46.088414  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 90387.4us
I20260709 13:01:46.181974  1636 rpcz_store.cc:276] 0709 13:01:45.158117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.387748 (+229631us) service_pool.cc:224] Handling call
0709 13:01:46.181256 (+793508us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b54003cdcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:46.181439 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48298880}
I20260709 13:01:46.088816  2346 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:46.089977  6606 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:46.090225  6686 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:01:46.091750  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.224s	user 0.025s	sys 0.023s
W20260709 13:01:46.093986  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.097020  6540 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:46.098371  6542 consensus_queue.cc:1059] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:46.153409  6455 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:46.097419  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:46.222455  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 125106us
I20260709 13:01:46.151569  6392 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:46.223824  6182 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:46.224740  6540 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:46.225356  6540 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:46.226430  6540 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:46.029947  6648 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.155668  1631 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:46.230048  1631 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:46.232388  6539 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:01:46.233326  6680 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.291s	user 0.001s	sys 0.001s
I20260709 13:01:46.160943  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2718) took 1528 ms. Trace:
I20260709 13:01:46.234184  6311 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:46.153324  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.235231  6571 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:46.235559  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.236454  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.160972  2345 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:46.238976  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:46.239699  2345 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:46.240772  2349 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:46.241089  2345 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: [10.9-11.10]
I20260709 13:01:46.178994  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:46.245884  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 66967.5us
I20260709 13:01:46.245420  1459 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 14, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:46.248142  6198 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:46.247571  1459 catalog_manager.cc:5697] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 4 to 7, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 7 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:46.248824  6198 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:46.179840  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:46.250070  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 70353us
I20260709 13:01:46.251340  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:46.086619  6597 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:46.253098  6198 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.254045  6597 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:46.158648  6613 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.573s	user 0.013s	sys 0.000s
I20260709 13:01:46.234532  1561 rpcz_store.cc:276] 0709 13:01:44.632699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.391721 (+759022us) service_pool.cc:224] Handling call
0709 13:01:45.392104 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.392974 (+   870us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.393010 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.410227 (+ 17217us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.928514 (+518287us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:45.928587 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.928716 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.160608 (+231892us) raft_consensus.cc:1685] finished
0709 13:01:46.160642 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.160859 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":517776,"threads_started":1,"wal-append.queue_time_us":626668}
W20260709 13:01:46.194321  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.197853  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:46.167812  6260 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:46.171094  6605 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:46.152760  6199 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:46.261313  6326 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:46.269529  1636 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:46.202039  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:46.188865  6611 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.262766  6613 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:46.272047  2350 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:46.272632  6611 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:46.205366  6542 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:46.273211  6542 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:46.190840  6155 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:46.276276  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.209231  6690 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:01:46.278523  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:46.174814  6569 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.279872  1703 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:46.280118  6569 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:46.215358  6689 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:01:46.187300  6606 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:46.283447  6606 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:46.181365  6684 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:46.219602  6673 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.625s	user 0.005s	sys 0.000s
I20260709 13:01:46.285872  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:46.286217  6419 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:46.286839  1553 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:46.220299  6618 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:46.185665  6687 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:46.234614  6311 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:45.995327  6319 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.260568  2346 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:46.254650  6597 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:01:46.262168  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.098891  6567 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:46.274593  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:46.267709  6605 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.287705  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1087) took 1062 ms. Trace:
I20260709 13:01:46.289304  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:46.264665  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:46.293618  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:01:46.335335  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 70766us
W20260709 13:01:46.294371  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.427s	user 0.025s	sys 0.024s
W20260709 13:01:46.302299  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.335824  2345 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:46.336207  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 1.46981e+06us
I20260709 13:01:46.235762  6571 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:46.172748  6570 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:46.323328  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.323055  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:46.279378  1559 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:46.326078  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41150 (request call id 1) took 1841 ms. Trace:
I20260709 13:01:46.326833  6155 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:46.321089  6319 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:46.331745  6692 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.114s	user 0.003s	sys 0.000s
I20260709 13:01:46.332675  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43592 (request call id 37) took 1418 ms. Trace:
I20260709 13:01:46.344003  2318 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:46.344470  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:46.292433  6684 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:46.348047  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:46.356489  1558 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:46.356778  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:46.355500  1557 rpcz_store.cc:276] 0709 13:01:44.484958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.507417 (+1022459us) service_pool.cc:224] Handling call
0709 13:01:46.325833 (+818416us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b5400389ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d4853 00007fda2c6a260a 00007fda25c287d4 00007fda25c28762 00007fda25c28691 00007fda25c2860c 00007fda25c278c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:46.325965 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":812784000}
I20260709 13:01:46.274212  6542 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:46.334491  2348 rpcz_store.cc:276] 0709 13:01:45.225669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.238700 (+ 13031us) service_pool.cc:224] Handling call
0709 13:01:45.244213 (+  5513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.255113 (+ 10900us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:45.255148 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.295261 (+ 40113us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.339496 (+ 44235us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:45.339547 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.339623 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.218828 (+879205us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:46.287259 (+ 68431us) raft_consensus.cc:1685] finished
0709 13:01:46.287290 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.287565 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10371,"raft.queue_time_us":42935,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":1900,"spinlock_wait_cycles":3425664,"thread_start_us":13534,"threads_started":1,"wal-append.queue_time_us":271231}
I20260709 13:01:46.314381  6613 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:46.321324  6597 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:46.395572  6353 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:46.398739  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:46.409638  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 65725.8us
I20260709 13:01:46.409986  1559 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:46.398952  1710 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 8.
I20260709 13:01:46.410373  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:46.345140  6442 raft_consensus.cc:2764] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:46.404027  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:46.413929  1557 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:46.398032  1707 rpcz_store.cc:276] 0709 13:01:44.913956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.969311 (+ 55355us) service_pool.cc:224] Handling call
0709 13:01:46.332442 (+1363131us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b5400077e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d4853 00007fda2c6a260a 00007fda25c287d4 00007fda25c28762 00007fda25c28691 00007fda25c2860c 00007fda25c278c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:46.332555 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1916716416}
I20260709 13:01:46.414331  6341 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:45.924863  1560 rpcz_store.cc:276] 0709 13:01:44.524951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.178949 (+653998us) service_pool.cc:224] Handling call
0709 13:01:45.205923 (+ 26974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.206184 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:45.328715 (+122531us) log.cc:844] Serialized 14 byte log entry
0709 13:01:45.889109 (+560394us) log.cc:844] Serialized 14 byte log entry
0709 13:01:45.889729 (+   620us) log.cc:844] Serialized 14 byte log entry
0709 13:01:45.911439 (+ 21710us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:45.911502 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:45.911556 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.911638 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.911788 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25044,"prepare.queue_time_us":64200,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":3999,"raft.queue_time_us":28008,"spinlock_wait_cycles":41815040,"thread_start_us":640045,"threads_started":2}
I20260709 13:01:46.416970  1559 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: [12.8-14.9]
W20260709 13:01:46.418655  6693 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
I20260709 13:01:46.423002  6570 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:46.428309  6630 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:46.341702  6605 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:46.438125  6442 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:46.439271  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:46.470711  1707 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:46.446476  6407 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:46.449913  6326 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:46.455627  2345 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:46.457504  1676 leader_election.cc:400] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10.
I20260709 13:01:46.473006  1676 leader_election.cc:403] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:46.436832  1560 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:46.474418  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:46.475119  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:46.476217  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:46.477674   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:01:46.478242  6569 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:46.485826  6679 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:46.486577  6679 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:46.487790  6679 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.489987  6419 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:46.490095  6591 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:46.491389  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:46.434429  6353 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:46.494056  6319 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:46.400458  6684 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:46.495864  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.467213  1558 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
W20260709 13:01:46.469926  1555 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.036s	sys 0.072s
W20260709 13:01:46.500268  1555 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.036s	sys 0.072s
I20260709 13:01:46.500423  1554 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:46.502193  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:46.509842  1704 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:46.510963  1558 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:46.512173  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:46.518074  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:46.519761  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:46.521587  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:46.495385  6442 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:46.503517  6618 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:46.529858  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:46.532553  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 349591us
I20260709 13:01:46.536084  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:46.523460  6569 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:46.537035  1560 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:46.538239  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:46.540397  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:46.541923  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2725) took 1006 ms. Trace:
I20260709 13:01:46.544615  1706 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:46.522012  6617 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:46.530519  6613 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:46.559553  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:46.559623  6613 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.547581  6260 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:46.560316  6613 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:46.560913  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 890) took 1401 ms. Trace:
I20260709 13:01:46.562363  6260 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:46.548333  6683 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.332s	user 0.005s	sys 0.000s
W20260709 13:01:46.548645  6688 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.506s	user 0.001s	sys 0.002s
I20260709 13:01:46.549311  2351 raft_consensus.cc:2410] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:46.561384  1630 rpcz_store.cc:276] 0709 13:01:45.159742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.400063 (+240321us) service_pool.cc:224] Handling call
0709 13:01:45.400447 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.493247 (+ 92800us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:45.493290 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.500890 (+  7600us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.879973 (+379083us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:45.880021 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.560526 (+680505us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b54001b6f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:46.560576 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.560620 (+    44us) raft_consensus.cc:1685] finished
0709 13:01:46.560632 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.560852 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16775,"raft.queue_time_us":20327,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":84240640,"thread_start_us":378662,"threads_started":1,"wal-append.queue_time_us":523214}
I20260709 13:01:46.554885  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:46.558059  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:46.571540  6567 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:46.530328  6455 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:01:46.555226  1703 rpcz_store.cc:276] 0709 13:01:45.539250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.560773 (+ 21523us) service_pool.cc:224] Handling call
0709 13:01:45.570239 (+  9466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.280335 (+710096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.541595 (+261260us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b540016c454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:46.541848 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1631472896}
I20260709 13:01:46.521817  6659 consensus_queue.cc:1059] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:01:46.553381  6541 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:46.533571  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:46.516624  6571 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.580736  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:46.577322  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:46.575874  6542 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:46.585852  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:46.594969  1557 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:46.565137  6569 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:46.573721  6326 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:01:46.576444  6567 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:46.588773  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:46.589098  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:46.589148  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:46.584649  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51232.1us
I20260709 13:01:46.597841  1558 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:46.597802  1703 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:46.598449  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:46.549957  6442 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.603211  1560 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:46.604182  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:46.605345  1703 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:46.588374  6645 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:46.593552  6602 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:46.622922  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:46.633865  1704 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:46.602404  6597 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.640082  6597 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:46.640885  6597 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:46.641573  6697 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:46.596745  6569 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.629232  1558 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:46.651921  6645 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:46.620074  6684 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:46.666225  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:46.673650  6645 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:46.673659  6567 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.682153  6541 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:46.683365  6319 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:46.683995  6319 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:46.684540  6682 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.489s	user 0.002s	sys 0.000s
I20260709 13:01:46.617398  6606 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.687109  6606 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:46.691355  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:46.692322  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:46.693359  6419 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:46.693823  6419 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.694168  6419 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:46.694234  6659 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:46.694738  6659 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.695070  6659 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:46.697613   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:46.719259  1707 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:46.720678  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:46.724391  1557 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:46.652375  6341 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:46.715873  1675 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:46.736625  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:46.738119  2352 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:01:46.738572  2352 pending_rounds.cc:85] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 7
I20260709 13:01:46.738837  2352 pending_rounds.cc:107] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:01:46.739107  2352 raft_consensus.cc:2889] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:46.727744  6697 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:46.741062  6619 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:46.741451  6619 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.741586  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2616) took 1536 ms. Trace:
I20260709 13:01:46.741880  6619 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:46.742432  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 721324us
I20260709 13:01:46.742041  1629 rpcz_store.cc:276] 0709 13:01:45.205176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.481013 (+275837us) service_pool.cc:224] Handling call
0709 13:01:45.481351 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.490412 (+  9061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.490447 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.497055 (+  6608us) log.cc:844] Serialized 26 byte log entry
0709 13:01:46.064146 (+567091us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:46.081374 (+ 17228us) log.cc:844] Serialized 14 byte log entry
0709 13:01:46.451609 (+370235us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b18003de7c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db
0709 13:01:46.535819 (+ 84210us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:46.741023 (+205204us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b18003de7c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:01:46.741212 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.741290 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.741336 (+    46us) raft_consensus.cc:1685] finished
0709 13:01:46.741352 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.741522 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13472,"prepare.queue_time_us":89045,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"raft.queue_time_us":11225,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":3534,"spinlock_wait_cycles":119956736,"thread_start_us":486421,"threads_started":1,"wal-append.queue_time_us":582762}
I20260709 13:01:46.747730  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:46.748850  1552 raft_consensus.cc:2410] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:01:46.751065  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:46.752380  1635 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:46.754016  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2732) took 1194 ms. Trace:
I20260709 13:01:46.755417  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 1083) took 1570 ms. Trace:
I20260709 13:01:46.755896  2351 rpcz_store.cc:276] 0709 13:01:45.185306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.185758 (+   452us) service_pool.cc:224] Handling call
0709 13:01:46.737321 (+1551563us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400643a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:46.755338 (+ 18017us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3092061568}
I20260709 13:01:46.762414  1559 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:46.754516  1706 rpcz_store.cc:276] 0709 13:01:45.560016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.715959 (+155943us) service_pool.cc:224] Handling call
0709 13:01:46.544341 (+828382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.555509 (+ 11168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.750394 (+194885us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b540016c450. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:46.753939 (+  3545us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1898611584}
I20260709 13:01:46.771117  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2735) took 1163 ms. Trace:
I20260709 13:01:46.771703  1701 rpcz_store.cc:276] 0709 13:01:45.582974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.584521 (+  1547us) service_pool.cc:224] Handling call
0709 13:01:46.745770 (+1161249us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54002e95d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:46.745905 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266262784}
I20260709 13:01:46.703245  1459 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 10 to 11, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), no longer has a pending config: VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194), VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 11 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:01:46.723383  6570 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10.
I20260709 13:01:46.685211  6319 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.774706  6611 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:46.742044  6613 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.778918  6697 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:46.805567  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2848) took 1132 ms. Trace:
I20260709 13:01:46.818282  1631 rpcz_store.cc:276] 0709 13:01:45.673642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.230978 (+557336us) service_pool.cc:224] Handling call
0709 13:01:46.231369 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.231642 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:46.583095 (+351453us) log.cc:844] Serialized 14 byte log entry
0709 13:01:46.805162 (+222067us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:46.805218 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:46.805273 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.805348 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.805510 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":82239,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":11748,"thread_start_us":572771,"threads_started":2}
I20260709 13:01:46.775149  6368 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:01:46.780174  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:46.796185  6705 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:46.828701  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.819868  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.781078  6645 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 13:01:46.846077  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 250110us
I20260709 13:01:46.778239  6200 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:01:46.869565  1702 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:46.869627  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:46.875358  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 136610us
I20260709 13:01:46.817854  6611 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:46.876062  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:46.823799  6319 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:46.841461  6085 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:46.887084  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 142868us
I20260709 13:01:46.889802  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:46.891222  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.892493  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:46.841950  6341 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.907991  2351 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:46.888370  6442 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:46.840935  6407 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:01:46.891109  6085 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:46.938177  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.940062  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:46.884145  6573 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:01:46.940515  1701 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:46.940876  1705 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
W20260709 13:01:46.942242  6707 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 13:01:46.943190  6701 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 13:01:46.866852  6705 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.893342  6567 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:01:46.941226  6573 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:46.958853  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.889731  6605 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:46.962790  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:46.964986  6721 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:46.965749  6442 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:46.966598  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:46.967937  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:46.968906  6326 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.969784  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:46.971850  6567 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:46.924206  6085 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:46.990458  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:46.988909  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:46.992362  1630 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:01:46.977401  1633 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
W20260709 13:01:46.978483  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 86039.4us
I20260709 13:01:46.924813  6720 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:46.924563  6574 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.988906  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:46.989778  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:46.999372  1631 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:46.994980  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:46.924968  6199 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:01:47.001727  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.003371  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.004132  1636 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:46.961001  6573 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:46.989125  6630 consensus_meta.cc:287] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.560s	user 0.006s	sys 0.006s
I20260709 13:01:46.988502  6407 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:46.986732  6567 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:46.985788  6721 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:47.015964  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 894) took 1666 ms. Trace:
I20260709 13:01:46.961290  6697 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:01:46.962765  6564 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:01:47.020750  1627 rpcz_store.cc:276] 0709 13:01:45.349763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.087744 (+737981us) service_pool.cc:224] Handling call
0709 13:01:46.096091 (+  8347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.096344 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:46.298575 (+202231us) log.cc:844] Serialized 14 byte log entry
0709 13:01:47.015590 (+717015us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.015646 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:47.015699 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.015769 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.015899 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":226404,"prepare.queue_time_us":161958,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":1926,"raft.queue_time_us":248208,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":6769,"thread_start_us":690539,"threads_started":3,"wal-append.queue_time_us":434831}
I20260709 13:01:46.998613  6720 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:47.036435  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.038026  1627 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:47.040208  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 164210us
I20260709 13:01:47.042074  1675 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:47.017094  6260 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:47.021888  6407 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:46.994092  6326 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:47.040182  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 144278us
I20260709 13:01:47.044636  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.050196  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:47.050451  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:47.010242  6574 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:47.041294  6702 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.232s	user 0.001s	sys 0.001s
I20260709 13:01:47.022748  6721 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:47.058007  1636 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:46.963500  6611 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:47.066151  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.061496  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:47.054993  6407 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:47.068709  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 67054.9us
I20260709 13:01:47.071411  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.074954  1627 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:47.067855  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.054661  6613 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.077040  1636 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.077170  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.058388  6199 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:47.081909  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.061264  6630 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:47.075645  6706 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:01:47.105021  6199 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.122105  1636 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:47.122756  6199 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.125427  6199 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:47.125921  6199 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:47.065346  6617 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:47.061262  6732 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:47.127291  6732 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:47.132454  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 50685us
I20260709 13:01:47.068413  6731 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:47.134711  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 90236.8us
W20260709 13:01:47.101265  6704 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.209s	user 0.001s	sys 0.001s
I20260709 13:01:47.140475  1628 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:01:47.144996  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.146335  6611 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:47.147766  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:47.114650  6708 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:01:47.055850  6573 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.048034  6442 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:47.152315  6711 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.227s	user 0.003s	sys 0.000s
I20260709 13:01:47.114356  6739 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:47.143576  6731 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.068789  6567 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.135483  6728 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:47.153192  6611 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.103089  6630 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.102222  6724 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:01:47.149338  6613 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:01:47.171124  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2622) took 1955 ms. Trace:
I20260709 13:01:47.160463  6617 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:47.180940  2350 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:47.182157  1635 rpcz_store.cc:276] 0709 13:01:45.215866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.022727 (+806861us) service_pool.cc:224] Handling call
0709 13:01:46.745327 (+722600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.752837 (+  7510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.170907 (+418070us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b5400349ed0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:47.171064 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1630488832}
I20260709 13:01:47.184031  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.165105  6723 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:47.184512  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2836) took 1692 ms. Trace:
I20260709 13:01:47.187549  1632 rpcz_store.cc:276] 0709 13:01:45.491830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.198621 (+706791us) service_pool.cc:224] Handling call
0709 13:01:46.263635 (+ 65014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.961783 (+698148us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:46.999045 (+ 37262us) log.cc:844] Serialized 14 byte log entry
0709 13:01:47.184210 (+185165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.184257 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:47.184296 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.184358 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.184461 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":495841,"prepare.queue_time_us":41283,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":515501,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":15093,"thread_start_us":385862,"threads_started":2}
I20260709 13:01:47.162994  6199 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.168076  6731 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.172899  6611 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.195766  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:47.209688  2348 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.197036  2344 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:47.180379  6705 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.192153  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:47.202139  1674 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.178593  6713 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.181361  6732 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.232115  2351 raft_consensus.cc:2410] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:47.191717  6567 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:47.211494  6726 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:01:47.195005  6574 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:47.249387  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47379.5us
I20260709 13:01:47.251806  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:47.191947  6738 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:47.235806  6648 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.004s	user 0.016s	sys 0.023s
I20260709 13:01:47.209404  6728 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.256733  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:47.187573  6341 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:01:47.258857  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 366171us
I20260709 13:01:47.286582  6732 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.291281  1632 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:47.291090  6732 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.291424  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.292449  1635 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:47.292536  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.293152  1632 raft_consensus.cc:2376] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14. Re-sending same reply.
I20260709 13:01:47.293251  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:47.293830  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.293931  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:47.294755  1632 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:01:47.234045  6260 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:47.311247  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:47.135210  6720 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:47.312191  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.286803  6648 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.055s	user 0.021s	sys 0.025s
I20260709 13:01:47.251315  6733 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.250110  6697 consensus_queue.cc:1243] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:47.254706  6605 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.329974  6758 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:47.330775  6758 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:47.333029  6659 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.638s	user 0.007s	sys 0.004s
I20260709 13:01:47.335777  6260 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:47.340401  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.258149  6326 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:47.286393  6541 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:47.347512  2347 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:47.347954  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:47.348723  1631 raft_consensus.cc:2368] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13.
W20260709 13:01:47.300912  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 116559us
I20260709 13:01:47.315291  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.350122  1627 raft_consensus.cc:2368] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:01:47.301350  6455 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:47.318933  6716 log.cc:927] Time spent T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:01:47.295730  6574 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:47.317368  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:47.351949  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.293546  6732 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.237092  6713 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.356381  6326 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:47.403618  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 269855us
I20260709 13:01:47.378708  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.377782  6541 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:47.361366  6758 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.356560  6720 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.406900  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.409113  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:47.409137  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.350225  6570 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:01:47.348466  6341 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.411585  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.413546  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:47.416064  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.417232  1674 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:47.399219  6754 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:47.346105  6659 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.421084  1635 raft_consensus.cc:2368] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10.
I20260709 13:01:47.426126  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:47.426014  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:47.407493  6738 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.375308  6574 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.353181  6733 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:01:47.428367  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.391767  6085 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.438892  6570 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:47.439330  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:47.441144  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:47.442483  1635 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:47.407310  6732 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:01:47.449329  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.449776  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:47.428470  6754 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.446192  6567 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:47.446338  6648 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.346122  6617 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.453347  2349 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:47.407372  6326 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.455080  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.466104  1627 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:47.439651  6630 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:47.443928  6719 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:47.444356  6730 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:47.444422  6752 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:47.453707  6756 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.478761  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.443818  6753 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:47.447592  6745 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:47.495203  2348 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:47.498871  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.499569  1635 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:47.499759  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.468955  6736 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.454077  6761 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:01:47.528024  6085 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.520514  6745 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:47.530001  6574 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.530789  6574 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:01:47.530710  6745 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.531821  6574 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:01:47.532554  6619 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.751s	user 0.006s	sys 0.005s
I20260709 13:01:47.499188  6753 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:47.461907  6567 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.533301  6753 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.534842  6199 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.535290  6567 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.499514  6766 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:47.536130  6766 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:47.536530  6766 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.538436  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:47.539199  1555 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:47.510123  6751 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:01:47.514680  6713 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:47.496728  6752 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:47.552809  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:47.562064  1627 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:47.502070  6730 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:47.574203  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:47.466171  6648 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.576713  1553 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:47.533062  6619 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.584995  6619 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:47.586685  6754 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.561820  6752 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:47.589735  6326 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.595551  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:47.595715  6319 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.704s	user 0.007s	sys 0.004s
I20260709 13:01:47.596287  1556 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.578955  6353 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:47.598023  6679 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:47.598479  6679 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:47.598595  6419 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.904s	user 0.013s	sys 0.000s
I20260709 13:01:47.599478  6679 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.599308  6419 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.600520  6679 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:47.501075  6756 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.601270  1705 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:01:47.607229  6659 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.607829  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 835) took 1793 ms. Trace:
I20260709 13:01:47.608232  2347 rpcz_store.cc:276] 0709 13:01:45.814176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.814430 (+   254us) service_pool.cc:224] Handling call
0709 13:01:45.840956 (+ 26526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.347921 (+1506965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.607563 (+259642us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b540048ab54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:47.607765 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3464287488}
I20260709 13:01:47.608523  6730 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:47.579578  6765 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.078s	user 0.002s	sys 0.000s
W20260709 13:01:47.604707  6542 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.002s	user 0.007s	sys 0.006s
I20260709 13:01:47.602283  6571 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.522416  6736 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.635564  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.644872  1556 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:47.646471  1635 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:47.647231  1632 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:47.647568  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:47.623025  6721 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.006s
I20260709 13:01:47.606928  6419 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.606097  6341 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:47.660887  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.661624  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.643193  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:47.660694  1555 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:47.623346  6761 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:47.637946  6728 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:47.674283  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.679369  1627 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.623680  6713 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:01:47.648548  6570 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:47.697242  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 218587us
I20260709 13:01:47.699604  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.675129  6542 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:47.671404  6602 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:47.711614  1627 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:47.663503  6721 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.704707  6542 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.681344  6761 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.715090  6574 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:47.726699  6768 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:01:47.667972  6731 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.751183  1631 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
W20260709 13:01:47.741447  1525 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.641s (SENT) [suppressed 18 similar messages]
I20260709 13:01:47.715090  6618 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:47.750598  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2109) took 1407 ms. Trace:
I20260709 13:01:47.706203  6602 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:47.731297  1627 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:47.748492  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.787062  1628 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:47.750034  6542 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:47.792203  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43838.7us
I20260709 13:01:47.785693  1708 rpcz_store.cc:276] 0709 13:01:46.342840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.343210 (+   370us) service_pool.cc:224] Handling call
0709 13:01:46.390868 (+ 47658us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:46.391448 (+   580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:46.391489 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:46.399805 (+  8316us) log.cc:844] Serialized 52 byte log entry
0709 13:01:46.697026 (+297221us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:46.697080 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.698210 (+  1130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:47.743703 (+1045493us) spinlock_profiling.cc:243] Waited 9.72 ms on lock 0x7b24001df5b8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda244739c4 00007fda24471acb 00007fda2462cdc9 00007fda2462ca55 00007fda2462bfb0 00007fda272a2488 00007fda272a2be8 00007fda2727a398 00007fda2727576a 00007fda27274bb6
0709 13:01:47.746522 (+  2819us) raft_consensus.cc:1685] finished
0709 13:01:47.746557 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.750511 (+  3954us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":852,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3792,"spinlock_wait_cycles":22364032,"thread_start_us":281075,"threads_started":1,"wal-append.queue_time_us":380225}
W20260709 13:01:47.788763  6407 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.706s	user 0.011s	sys 0.002s
I20260709 13:01:47.797261  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.781232  6326 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:47.807572  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66177.2us
I20260709 13:01:47.808421  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.789582  6618 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:47.794672  6768 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:47.746824  6723 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:01:47.750831  6767 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:01:47.812077  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.805716  6754 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:47.811729  6085 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.808758  6574 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:47.811968  6732 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:01:47.812413  6770 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:01:47.828691  6455 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:47.816491  6761 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.829874  6442 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:01:47.831035  6442 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:47.830808  6311 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.831504  6442 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:47.821635  6754 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.832320  6311 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.832479  6442 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.833355  6731 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.834384  6754 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.817150  6767 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.798112  6777 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:47.823354  6739 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.824098  6564 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:47.839365  6564 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:47.809152  6407 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:47.839951  6648 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.840283  6564 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.841586  6407 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.745959  6570 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.824533  1630 raft_consensus.cc:2410] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:01:47.846040  6736 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.838559  6777 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:47.823101  6766 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.852939  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.852959  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.860168  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:47.856155  6564 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:47.865059  1507 auto_rebalancer.cc:893] Removing replace marker failed: Replica d62ce354d09a4be59c29094425d09bc1 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:47.865124  6739 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:47.865808  1507 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:01:47.830911  6756 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:47.866140  1507 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:01:47.862210  6697 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.867130  6721 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.866812  6697 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:47.869082  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.868975  6752 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.822000  6085 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.870291  6744 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:47.870666  6611 consensus_meta.cc:287] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.002s
I20260709 13:01:47.870898  6744 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.871089  6611 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.872054  6085 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.873104  6744 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.869385  6697 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.881822  6752 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:47.885048  6753 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.869596  6731 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:47.886349  6731 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.888060  6722 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.888386  6341 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:47.888911  6731 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.888698  6722 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.863336  1602 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:47.872987  6777 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.890317  1635 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.862668  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1278) took 2385 ms. Trace:
I20260709 13:01:47.873617  1636 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 13:01:47.825095  6602 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.874804  6684 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:47.875314  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:47.907891  2347 raft_consensus.cc:2410] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:47.810690  6326 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.908784  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.908908  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:47.909636  1628 raft_consensus.cc:2368] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13.
I20260709 13:01:47.909920  6763 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:47.910509  6326 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.911190  6754 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:47.911499  6763 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:47.877599  2345 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:47.829171  6455 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:47.877748  6766 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.880278  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:47.880288  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:47.913764  2344 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:47.913990  2348 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:47.865958  6739 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:47.907130  6684 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:47.915218  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:47.820370  6738 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:47.874066  6767 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.932210  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:47.893719  2352 rpcz_store.cc:276] 0709 13:01:45.477787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.482699 (+  4912us) service_pool.cc:224] Handling call
0709 13:01:46.737692 (+1254993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.739599 (+  1907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.862454 (+1122855us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400643a50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:47.862607 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2842052992}
I20260709 13:01:47.861706  1633 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:47.944334  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.825639   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:47.901054  6777 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.901536  6618 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.902369  6721 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:47.957968  1459 catalog_manager.cc:5697] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 7 to 9, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 9 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:47.902563  6198 consensus_queue.cc:1059] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:01:47.903983  6748 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.649s	user 0.003s	sys 0.000s
I20260709 13:01:48.002164  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:47.908316  6756 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:47.916497  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.858870  6442 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:47.916755  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:48.003398  6739 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:47.919756  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.004452  1552 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:48.004369  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:47.920121  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.005892  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:48.006559  1633 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:48.006690  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.007524  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:47.915875  6684 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:48.008747  6684 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.009438  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:48.010819  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:48.011449  2352 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.910532  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:47.922488  1553 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:01:48.014761  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:48.015900  1628 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:01:48.015954  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.017055  1553 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:47.947422  1635 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:01:48.018348  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 101683us
I20260709 13:01:47.920810  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:47.893778  6407 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.019619  6407 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:47.951620  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2118) took 1485 ms. Trace:
I20260709 13:01:47.873152  6611 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:48.021653  1702 rpcz_store.cc:276] 0709 13:01:46.466106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.559299 (+ 93193us) service_pool.cc:224] Handling call
0709 13:01:47.951323 (+1392024us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b540036ddd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:47.951532 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710640000}
I20260709 13:01:47.980624  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 897) took 2490 ms. Trace:
I20260709 13:01:47.888932  6341 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:01:47.985743  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 53623.2us
W20260709 13:01:47.986790  6779 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:01:47.987473  6368 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:48.024003  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:47.994544  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:47.994369  6455 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:48.024021  6341 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.027081  6455 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.027473  6341 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.028488  6573 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.957397  6767 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.032588  6767 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:47.920722  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.007544  1552 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:48.017309  1628 pending_rounds.cc:85] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:01:48.034622  1628 pending_rounds.cc:107] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:01:48.034897  1628 raft_consensus.cc:2889] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:48.034940  1627 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.018492  6756 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:47.933022  6679 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:47.987149  1557 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:48.045465  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.049170  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:48.049293  1552 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:48.050033  1557 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:48.050550  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.050889  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.051368  1552 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
W20260709 13:01:47.981079  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.034145  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51904.2us
I20260709 13:01:48.036996  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:48.052799  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:48.053133  1553 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:48.022694  1634 rpcz_store.cc:276] 0709 13:01:45.490049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.196693 (+706644us) service_pool.cc:224] Handling call
0709 13:01:46.205191 (+  8498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:46.745246 (+540055us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:46.745277 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:46.745852 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 13:01:46.910682 (+164830us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:46.910736 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.931877 (+1021141us) spinlock_profiling.cc:243] Waited 69.5 ms on lock 0x7b5400139cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:47.931917 (+    40us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:47.931955 (+    38us) raft_consensus.cc:1685] finished
0709 13:01:47.931964 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.980509 (+ 48545us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":470948,"raft.queue_time_us":525458,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":2528,"spinlock_wait_cycles":159827840,"thread_start_us":103281,"threads_started":1,"wal-append.queue_time_us":263399}
W20260709 13:01:48.039602  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 54746.8us
I20260709 13:01:48.057976  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:48.051519  1557 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:48.062171  1557 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:48.064652  6611 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.078912  6542 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:48.065086  1701 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:48.080893  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:48.066753  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.070636  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:48.066355  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:48.074901  2345 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:48.089416  1557 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.075528  2347 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:48.079891  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:48.074098  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:48.065867  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.090940  1708 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.073740  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:48.076500  2351 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:48.072188  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:48.090132  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.077982  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:48.093375  1706 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:48.093219  1629 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:48.074254  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:48.098618  1628 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:01:48.099269  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:48.099879  1560 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:48.101352  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.101958  1706 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:48.113847  2352 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:48.115815  2349 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:48.130996  1552 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:01:48.138911  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:48.139736  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:48.140573  2352 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:48.145612  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.076706  6602 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.146757  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:48.079456  6611 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.070340  6736 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:48.148705  1706 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:01:48.152192  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.153295  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:48.148057  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 124143us
I20260709 13:01:48.166625  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:48.167203  1627 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:48.155866  1705 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:01:48.167878  1705 pending_rounds.cc:85] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 9
I20260709 13:01:48.168169  1705 pending_rounds.cc:107] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:01:48.168470  1705 raft_consensus.cc:2889] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:48.131057  6733 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.174526  1707 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:48.177764  1553 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:48.157982  2318 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:48.158105  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:48.158027  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.183084  1552 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:01:48.159049  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.078589  6407 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:48.160501  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.160744  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.185206  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:48.163290  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.189625  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:48.190295  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 167547us
I20260709 13:01:48.190836  2352 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:48.191337  2317 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:48.170033  1557 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:48.170892  6591 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:01:48.173297  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:48.196178  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.174877  6721 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:48.191771  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:48.071380  6767 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 13:01:48.198419  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:48.230530  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.232893  1602 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:48.233314  1560 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:48.234793  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:48.200453  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.195626  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:48.203934  1704 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:48.153578  6747 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:48.205518  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.147368  6679 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:48.199568  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:48.227340  6727 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:48.168666  6722 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:48.255024  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 88521us
I20260709 13:01:48.277665  1557 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:48.278851  1525 leader_election.cc:400] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13.
I20260709 13:01:48.280198  1525 leader_election.cc:400] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:01:48.280515  1525 leader_election.cc:403] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:01:48.255611  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:48.281579  1705 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:48.252971  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.258699  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.238636  1553 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:48.264693  1558 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:01:48.267011  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.238255  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.269872  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.271636  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.195545  6756 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:48.286499  1459 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 8 to 11, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 11 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:48.274479  1710 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 13:01:48.307658  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 119288us
I20260709 13:01:48.291589  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:48.309165  1710 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:01:48.310264  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 77461.9us
I20260709 13:01:48.314211  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:48.309322  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.319026  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.175952  6743 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.100s
W20260709 13:01:48.320557  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 169448us
I20260709 13:01:48.322575  1602 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:48.324740  1552 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:01:48.334052  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54050.4us
I20260709 13:01:48.336014  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:48.333101  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:48.193852  6769 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.237668  6085 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:01:48.359217  1710 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:48.339866  1602 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:48.339913  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.361070  1702 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:48.254966  6648 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.267570  6567 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.361896  6648 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.360631  6769 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.267814  6736 raft_consensus.cc:697] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:48.364543  6648 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:48.318353  6756 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:48.203092  6326 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.118s
I20260709 13:01:48.195248  6733 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.254635  6597 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.240212  6770 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:48.350911  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.347514  1701 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:48.318529  6200 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:48.328430  6311 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:48.349967  1525 leader_election.cc:400] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10.
I20260709 13:01:48.329150  6761 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:48.305698  6753 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:48.311383  6752 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:48.404268  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.407315  1525 leader_election.cc:403] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:01:48.407405  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 93303.9us
I20260709 13:01:48.388219  6723 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:48.408391  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 75360.7us
I20260709 13:01:48.444424  1675 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:48.446355  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:48.343551  6613 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:01:48.366108  6768 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.420527  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:48.422758  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.459208  1553 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:48.423385  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:48.367130  6758 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.390043  6756 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.330375  6630 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:01:48.437527  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:48.321658  6747 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:01:48.424584  6200 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:48.385769  6770 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.339291  6732 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:01:48.367467  6720 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:01:48.418290  6619 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:48.468719  1702 raft_consensus.cc:2393] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:48.412032  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:48.493798  1703 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.475528  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:48.467545  6730 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.481845  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.485669  1631 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:48.486259  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.513881  6733 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:48.514163  6619 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.514683  6619 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.460472  6542 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:48.515383  6542 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:48.517791  6754 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:48.518112  6786 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:01:48.518247  6754 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:48.519253  6754 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:48.520557  6754 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:48.487887  6777 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.001s
I20260709 13:01:48.442775  6753 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:48.522293  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:48.522900  6753 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:48.523634  6753 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.462898  6723 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.525187  6571 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:48.526198  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.457209  6679 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:48.415452  6727 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:48.530999  6727 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:48.489073  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:48.533602  1635 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:48.512451  6725 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.431727  6752 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:48.451066  6085 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10.
W20260709 13:01:48.490900  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 141147us
I20260709 13:01:48.364959  6736 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.541889  6725 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:48.574484  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.577812  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2759) took 1058 ms. Trace:
I20260709 13:01:48.565197  1701 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:48.561719  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:48.582402  1561 rpcz_store.cc:276] 0709 13:01:47.519732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.523015 (+  3283us) service_pool.cc:224] Handling call
0709 13:01:47.940917 (+417902us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b5800002798. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:01:47.941211 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.941536 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:01:48.240081 (+298545us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.574825 (+334744us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:48.574886 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:48.574936 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.577587 (+  2651us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.577746 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":570967,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":597,"spinlock_wait_cycles":41126784,"thread_start_us":564426,"threads_started":2}
I20260709 13:01:48.595157  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:48.584450  1635 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:01:48.592710  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:48.589154  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.579981  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2760) took 1059 ms. Trace:
I20260709 13:01:48.608831  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:48.615437  1552 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:48.617273  1633 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:01:48.600260  1710 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
W20260709 13:01:48.621954  1708 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.000s
I20260709 13:01:48.616381  1559 rpcz_store.cc:276] 0709 13:01:47.520144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.523379 (+  3235us) service_pool.cc:224] Handling call
0709 13:01:47.987769 (+464390us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b5800002798. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:01:47.987987 (+   218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.988211 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:48.237113 (+248902us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.579482 (+342369us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:48.579559 (+    77us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:48.579627 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.579725 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.579899 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":524648,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":32,"raft.queue_time_us":2960,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":12602,"spinlock_wait_cycles":39152896,"thread_start_us":581085,"threads_started":2}
I20260709 13:01:48.599674  6368 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:48.646142  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:48.609975  6697 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:48.649928  1708 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:48.652175  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:48.642624  6455 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.606s	user 0.007s	sys 0.006s
I20260709 13:01:48.632107  6085 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:48.665704  2353 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.062s	sys 0.053s
W20260709 13:01:48.688015  2353 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.062s	sys 0.053s
I20260709 13:01:48.654655  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:48.690007  2317 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:48.685072  6770 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:48.664793  1703 raft_consensus.cc:2410] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:48.607668  6198 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
W20260709 13:01:48.693848  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.643801  6793 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.610455  6407 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:01:48.696593  6591 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:48.700187  6564 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.835s	user 0.008s	sys 0.004s
I20260709 13:01:48.700455  6731 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:48.701541  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.704072  1706 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:48.705333  6721 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:48.710394  6618 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.686318  6766 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:48.651494  6759 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.616518  6758 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
W20260709 13:01:48.723162  1554 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.028s	sys 0.072s
I20260709 13:01:48.653530  6368 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:48.646984  6200 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:48.640929  6747 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:01:48.679544  6606 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:48.733783  1634 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:01:48.757925  1634 pending_rounds.cc:85] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 8
I20260709 13:01:48.758298  1634 pending_rounds.cc:107] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:01:48.758589  1634 raft_consensus.cc:2889] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:48.758994  1634 pending_rounds.cc:107] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:01:48.759259  1634 raft_consensus.cc:2889] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:48.734093  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.720587  6793 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.767491  2344 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:01:48.750098  1554 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.028s	sys 0.072s
I20260709 13:01:48.721755  6442 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:48.717365  6085 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:48.724803  6591 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.727872  6648 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:48.783393  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:48.728909  6731 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:48.787925  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:48.742024  6721 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:48.792168  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 345845us
I20260709 13:01:48.791561  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:48.793711  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:48.765820  6368 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:48.782361  1705 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:48.801560  1704 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:01:48.803776  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:48.765604  6606 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.806926  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:48.808811  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.809525  1553 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 8.
W20260709 13:01:48.811530  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.814601  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:48.778208  6770 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.778056  6591 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:48.819808  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.821099  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.821861  1555 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:48.823207  1555 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:48.779103  6766 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:48.827608  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:48.726962  6723 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:48.797751  6602 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.000s
I20260709 13:01:48.829209  1552 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:48.719736  6769 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:48.917765  1555 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:48.958396  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:48.950497  2346 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:48.817656  6606 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:48.972592  1602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.045s	sys 0.028s
I20260709 13:01:48.763711  6759 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:48.978178  1602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.046s	sys 0.028s
I20260709 13:01:48.833180  6713 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:48.792651  6648 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.797645  6542 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.174s
I20260709 13:01:48.830569  6770 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:49.023402  6648 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:48.997227  6085 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:48.960117  6769 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.830972  6326 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.164s
I20260709 13:01:48.975186  6442 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:49.058260  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 389579us
W20260709 13:01:49.007706  6794 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.201s	user 0.001s	sys 0.001s
W20260709 13:01:49.006731  2350 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.973s	user 0.047s	sys 0.040s
W20260709 13:01:49.221272  2350 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.188s	user 0.047s	sys 0.041s
W20260709 13:01:49.008956  6791 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
I20260709 13:01:49.011950  6766 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:48.991348  6721 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:48.979705  6725 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:49.016966  6793 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:48.982720  6613 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.035058  6759 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:48.976704  6731 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.046002  2349 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 7.
I20260709 13:01:48.825296  6597 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.156s
W20260709 13:01:49.246526  6808 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 13:01:49.082849  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 485709us
I20260709 13:01:49.055814  6770 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:49.057521  6743 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:49.248339  6743 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:49.250833  6743 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:49.064937  6744 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.014s	sys 0.073s
W20260709 13:01:49.251497  6744 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.014s	sys 0.073s
I20260709 13:01:49.114467  6442 raft_consensus.cc:697] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:01:49.126519  6797 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:01:49.065824  6796 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:49.127876  6754 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.607s	user 0.015s	sys 0.000s
W20260709 13:01:49.130556  6788 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:01:49.136933  6697 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:01:49.139464  6619 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.624s	user 0.011s	sys 0.001s
I20260709 13:01:49.140208  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2645) took 1236 ms. Trace:
W20260709 13:01:49.141455  6753 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.617s	user 0.013s	sys 0.001s
W20260709 13:01:49.118186  6573 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.089s	user 0.014s	sys 0.001s
W20260709 13:01:49.142385  6756 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.636s	user 0.012s	sys 0.000s
I20260709 13:01:49.144662  6591 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.269654  6619 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.273317  6311 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:49.273442  6591 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:49.273914  6311 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:49.139468  6768 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.665s	user 0.012s	sys 0.000s
I20260709 13:01:49.274675  6768 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:49.063004  6739 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:49.151961  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:49.060626  6769 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.182933  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:49.184012  1628 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:49.186044  1631 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:01:49.279855  1631 pending_rounds.cc:85] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:01:49.280242  1631 pending_rounds.cc:107] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:01:49.280606  1631 raft_consensus.cc:2889] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:49.283106  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 131266us
I20260709 13:01:49.286556  1526 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:49.189946  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 756) took 1064 ms. Trace:
W20260709 13:01:49.193620  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.008s	sys 0.011s
W20260709 13:01:49.289014  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.008s	sys 0.011s
W20260709 13:01:49.200604  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.197430  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2762) took 1676 ms. Trace:
I20260709 13:01:49.156114  6407 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 13:01:49.195058  6645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.003s	sys 0.016s
W20260709 13:01:49.215230  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.222961  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 408461us
I20260709 13:01:49.230016  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 854) took 1230 ms. Trace:
I20260709 13:01:49.230647  1636 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:49.224512  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:48.989315  1709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.022s	sys 0.052s
I20260709 13:01:49.216490  6326 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:49.258160  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.258919  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:49.292678  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 681324us
I20260709 13:01:49.293138  2347 raft_consensus.cc:2410] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 13:01:49.228245  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 269843us
I20260709 13:01:49.231622  6567 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:49.232194  6574 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:01:49.265408  1634 rpcz_store.cc:276] 0709 13:01:47.903617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.139996 (+236379us) service_pool.cc:224] Handling call
0709 13:01:48.140307 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.759841 (+619534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.139791 (+379950us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b540017b454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:49.140114 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1363624320}
I20260709 13:01:49.274147  6619 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:49.276393  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1103) took 1199 ms. Trace:
I20260709 13:01:49.277021  1632 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:49.293629  1525 leader_election.cc:400] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13.
W20260709 13:01:49.270841  6618 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.520s	user 0.008s	sys 0.005s
I20260709 13:01:49.288898  1560 rpcz_store.cc:276] 0709 13:01:48.126039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.261615 (+135576us) service_pool.cc:224] Handling call
0709 13:01:48.262202 (+   587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.262577 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:48.641747 (+379170us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.189155 (+547408us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:49.189245 (+    90us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:49.189325 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.189431 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.189850 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27135,"prepare.queue_time_us":250976,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":1340,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":24536,"thread_start_us":720107,"threads_started":2,"wal-append.queue_time_us":553614}
I20260709 13:01:49.347496  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:49.347862  1708 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:49.348445  1703 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:49.264025  6796 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:49.295990  1709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.022s	sys 0.053s
I20260709 13:01:49.321403  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:49.296937  2350 rpcz_store.cc:276] 0709 13:01:47.999629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.999902 (+   273us) service_pool.cc:224] Handling call
0709 13:01:48.018029 (+ 18127us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:48.032701 (+ 14672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.032740 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:48.033411 (+   671us) log.cc:844] Serialized 52 byte log entry
0709 13:01:49.221860 (+1188449us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:49.229326 (+  7466us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.229610 (+   284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.229708 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.229757 (+    49us) raft_consensus.cc:1685] finished
0709 13:01:49.229771 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.229947 (+   176us) inbound_call.cc:177] Queueing success response
0709 13:01:49.251522 (+ 21575us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b18004468c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
Metrics: {"mutex_wait_us":14210,"prepare.queue_time_us":9440,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":15306,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":791,"spinlock_wait_cycles":41439488,"thread_start_us":1187959,"threads_started":1,"wal-append.queue_time_us":1108722}
I20260709 13:01:49.318311  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:49.323190  1675 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:49.252252  6613 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.278234  6810 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:49.294149  6645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.003s	sys 0.017s
I20260709 13:01:49.329746  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:49.330684  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.333250  1705 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:49.291260  6573 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:49.343613  1525 leader_election.cc:403] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:01:49.291625  6697 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:49.294471  1554 rpcz_store.cc:276] 0709 13:01:47.520867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.524214 (+  3347us) service_pool.cc:224] Handling call
0709 13:01:48.187876 (+663662us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b5800002798. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:01:48.188125 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.188341 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:01:48.773393 (+585052us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.196943 (+423550us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:49.197023 (+    80us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:49.197079 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.197156 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.197341 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7638,"prepare.queue_time_us":324861,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":160,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":6425,"spinlock_wait_cycles":36901248,"thread_start_us":998517,"threads_started":2}
W20260709 13:01:49.319425  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 95029.5us
I20260709 13:01:49.344139  2353 rpcz_store.cc:276] 0709 13:01:48.076804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.077356 (+   552us) service_pool.cc:224] Handling call
0709 13:01:48.077978 (+   622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.078383 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.078409 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:48.079419 (+  1010us) log.cc:844] Serialized 564 byte log entry
0709 13:01:48.688481 (+609062us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:48.688528 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.688602 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.276112 (+587510us) raft_consensus.cc:1685] finished
0709 13:01:49.276140 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.276310 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1792,"thread_start_us":608713,"threads_started":1,"wal-append.queue_time_us":721643}
W20260709 13:01:49.330530  6797 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:01:49.356132  1557 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 8: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:49.356967  6759 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:49.357247  6825 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:49.360351  6818 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:01:49.361135  6816 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:01:49.323019  6754 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:49.276463  6753 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.324667  6768 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:49.279296  6756 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:49.401075  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 78107.7us
W20260709 13:01:49.402247  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 108778us
I20260709 13:01:49.548346  1525 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:49.548585  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:49.475059  6200 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.368937  6574 consensus_queue.cc:1243] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:49.405284  6743 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:49.294783  6739 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.563156  6739 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:49.478868  6825 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.409384  2318 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:49.565588  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 156311us
I20260709 13:01:49.412078  6769 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:49.331393  6591 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
W20260709 13:01:49.501962  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 98988us
I20260709 13:01:49.327338  6326 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.349507  6606 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:49.477653  6815 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:01:49.480042  6781 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.285s	user 0.006s	sys 0.000s
I20260709 13:01:49.348863  6821 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.162s
I20260709 13:01:49.473780  6618 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:49.549731  1525 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:49.404567  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:49.416319  6763 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13.
I20260709 13:01:49.645167  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:49.647073  6763 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:49.647511  6763 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:49.647914  6763 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:49.512027  6754 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:49.473657  6824 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.480783  6573 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.650089  6754 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:49.650367  6573 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.366037  6810 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.640265  6618 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.653944  6618 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:49.480317  6759 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:49.645100  6739 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:49.558934  6820 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
W20260709 13:01:49.559666  6827 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.269s	user 0.000s	sys 0.000s
I20260709 13:01:49.403599  6442 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.573980  6727 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:49.474287  6679 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:01:49.618283  1707 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:01:49.580229  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:49.618913  2345 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:49.641500  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 236997us
I20260709 13:01:49.650393  6777 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:49.641649  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:49.564740  6200 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:49.645344  6826 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:01:49.567015  6798 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:01:49.668843  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2743) took 1507 ms. Trace:
W20260709 13:01:49.706638  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 65091.4us
I20260709 13:01:49.556586  6645 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:49.612717  6828 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:01:49.607640  6712 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:49.501869  6619 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.648906  6824 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.683079  6736 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:49.700814  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:49.641966  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.660689  6770 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.661365  6840 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:49.701889  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 152220us
I20260709 13:01:49.682868  6617 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:49.704079  1709 rpcz_store.cc:276] 0709 13:01:48.161402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.161611 (+   209us) service_pool.cc:224] Handling call
0709 13:01:48.162123 (+   512us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:48.184674 (+ 22551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.184718 (+    44us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:48.194979 (+ 10261us) log.cc:844] Serialized 78 byte log entry
0709 13:01:49.352345 (+1157366us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:49.402220 (+ 49875us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.472127 (+ 69907us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.660465 (+188338us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.668094 (+  7629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.668531 (+   437us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.668575 (+    44us) raft_consensus.cc:1685] finished
0709 13:01:49.668588 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.668777 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":190976,"prepare.queue_time_us":49696,"prepare.run_wall_time_us":3750,"raft.queue_time_us":131542,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":51423,"spinlock_wait_cycles":210832128,"thread_start_us":1210787,"threads_started":2,"wal-append.queue_time_us":854656}
I20260709 13:01:49.711609  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:49.737963  2353 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:49.660002  6573 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:49.592373  6571 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.107s
I20260709 13:01:49.611805  6756 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:49.721370  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:49.696916  6727 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:49.685070  6793 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:01:49.703275  6777 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:49.773999  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 129788us
I20260709 13:01:49.713465  6712 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.777525  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2172) took 1114 ms. Trace:
I20260709 13:01:49.725834  6739 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:49.763320  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:49.766399  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.714535  6341 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:49.713275  6769 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:49.727754  6368 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.903s	user 0.014s	sys 0.000s
I20260709 13:01:49.785332  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:49.823814  1633 raft_consensus.cc:2410] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:49.792624  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:49.786185  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:49.768210  6574 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:49.790721  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:49.838840  1631 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:49.726908  6619 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:49.779146  6613 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:49.802681  6341 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.803383  6768 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:49.805183  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:49.737105  6736 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:49.807817  6843 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:49.811591  6571 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.768782  6727 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:49.746261  6840 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:49.819967  6834 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 13:01:49.820499  6769 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:49.772531  6573 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:01:49.781821  6712 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:49.822836  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.722288  6597 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:01:49.794405  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:50.067269  1630 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:49.777994  1559 rpcz_store.cc:276] 0709 13:01:48.663529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.663695 (+   166us) service_pool.cc:224] Handling call
0709 13:01:48.664113 (+   418us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:48.664659 (+   546us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:48.664703 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:48.689458 (+ 24755us) log.cc:844] Serialized 52 byte log entry
0709 13:01:49.213056 (+523598us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:49.213123 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.213245 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.777230 (+563985us) raft_consensus.cc:1685] finished
0709 13:01:49.777259 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.777440 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4285,"raft.queue_time_us":112,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":12900,"thread_start_us":485756,"threads_started":1,"wal-append.queue_time_us":572567}
I20260709 13:01:49.741291  6770 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:49.866567  1627 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:49.806592  6841 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:01:49.800829  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:49.813978  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:50.074404  1629 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:50.055588  1634 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:49.860283  6743 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:49.860024  6723 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:50.005509  6753 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.007938  6744 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.006150  6613 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.837953  6368 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:50.079298  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 278575us
I20260709 13:01:50.087643  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:50.021183  6574 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:50.047493  6611 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:49.822662  6777 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:49.829339  6759 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:50.051631  6810 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:50.088495  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 379578us
W20260709 13:01:50.093178  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 196818us
I20260709 13:01:50.017248  6341 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.097312  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:50.081825  6845 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.048272  6571 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.081671  6768 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:50.099767  6753 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:50.084774  6824 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.084424  6825 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.041754  6843 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.087406  6723 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.085825  6743 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:50.228659  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.186525  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:50.206365  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:50.113317  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:50.283668  2350 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:50.163457  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:50.140203  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:50.181206  2349 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 861) took 1416 ms (client timeout 1887 ms). Trace:
W20260709 13:01:50.285542  2349 rpcz_store.cc:269] 0709 13:01:48.759961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.760107 (+   146us) service_pool.cc:224] Handling call
0709 13:01:50.175873 (+1415766us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540061fb54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:50.176090 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597872896}
I20260709 13:01:50.186744  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2128) took 1792 ms. Trace:
I20260709 13:01:50.107957  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:50.289736  2353 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:50.098192  6574 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.283866  1635 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:50.156570  6200 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.286819  1701 rpcz_store.cc:276] 0709 13:01:48.394131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.394442 (+   311us) service_pool.cc:224] Handling call
0709 13:01:48.394905 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.582045 (+187140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.186444 (+1604399us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54001e9954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:50.186669 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390935424}
W20260709 13:01:50.294390  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.100122  6606 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:50.214094  6311 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.940s	user 0.018s	sys 0.002s
W20260709 13:01:50.177086  6837 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260709 13:01:50.326741  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1297) took 1073 ms. Trace:
I20260709 13:01:50.327589  6843 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.327224  2344 rpcz_store.cc:276] 0709 13:01:49.253122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.254710 (+  1588us) service_pool.cc:224] Handling call
0709 13:01:49.255016 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:49.255363 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:49.255390 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:49.263757 (+  8367us) log.cc:844] Serialized 26 byte log entry
0709 13:01:49.505616 (+241859us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:49.505663 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.505743 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.172462 (+666719us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b48001e12d8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:01:50.326363 (+153901us) raft_consensus.cc:1685] finished
0709 13:01:50.326390 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.326653 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1774,"raft.run_cpu_time_us":3117,"raft.run_wall_time_us":913119,"spinlock_wait_cycles":5491968,"thread_start_us":233393,"threads_started":1,"wal-append.queue_time_us":499051}
W20260709 13:01:50.289687  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 181812us
W20260709 13:01:50.302326  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 205127us
W20260709 13:01:50.182148  6618 consensus_meta.cc:287] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.528s	user 0.011s	sys 0.007s
I20260709 13:01:50.294482  6200 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:50.335122  6200 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:50.335865  6200 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:50.337757  6618 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.307632  6753 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.338413  6341 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:50.278165  6326 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.338873  6753 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.317209  6573 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.317531  6723 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.320827  6743 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.323575  6571 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.119727  6821 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.119623  6845 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.120260  6777 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:50.171404  6849 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.239380  6857 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:50.315637  6311 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.171950  6770 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:50.096675  6744 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.206359  6619 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:50.344869  6753 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.207311  6736 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:50.172255  6772 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:50.477630  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:50.478806  2351 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:50.306201  6824 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:50.096442  6613 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:50.307883  6759 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:50.173177  6848 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.203s
I20260709 13:01:50.313443  6766 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:50.375841  6763 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.727s	user 0.014s	sys 0.002s
I20260709 13:01:50.400398  1710 raft_consensus.cc:2368] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 8. Current term is 9.
I20260709 13:01:50.431773  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:50.379509  6840 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.462575  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:50.467520  6843 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.379066  6743 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.504701  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2876) took 2020 ms. Trace:
I20260709 13:01:50.505198  1636 rpcz_store.cc:276] 0709 13:01:48.484272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.486052 (+  1780us) service_pool.cc:224] Handling call
0709 13:01:48.486312 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:49.297042 (+810730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.504424 (+1207382us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b54003cd7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:50.504634 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1711228160}
I20260709 13:01:50.480851  6753 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.370321  6341 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:01:50.368911  6573 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.482662  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:50.484404  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:50.437795  6368 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.471155  6862 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.462369  6858 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.471419  6856 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.491954  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:50.533995  2353 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:50.468518  6841 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:50.356091  6769 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:50.471766  6768 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.436787  6571 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:50.498577  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:50.499542  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:50.499989  6787 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.985s	user 0.009s	sys 0.000s
I20260709 13:01:50.500713  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:50.478953  6723 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.480000  6810 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:50.468799  2348 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:01:50.508190  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 344800us
I20260709 13:01:50.398000  6821 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.514197  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:50.518960  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:50.519595  2347 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:50.435087  6849 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.535602  1676 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:50.512670  6743 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.520460  6753 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.509615  6843 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.520255  6606 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:01:50.545979  1553 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:50.546747  1557 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:50.549765  2350 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:50.550108  2352 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:01:50.550457  2349 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:50.564289  2351 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:50.552374  6849 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.561039  6368 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.555331  6573 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.547051  6723 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.561309  6606 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:01:50.586217  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.442672  6857 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.489113  6759 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.585165  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:50.545928  1554 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:50.566537  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:50.613672  2347 raft_consensus.cc:2410] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:01:50.484091  6772 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.495184  6766 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.612594  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:50.535926  6855 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.554450  6311 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.542802  6571 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.495057  6763 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.582427  6721 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:50.636646  6845 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.637954  6613 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:50.639963  6766 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.622910  1676 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:50.577885  6848 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:50.547379  6810 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.556767  6743 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.609226  6540 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:01:50.643213  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:50.565011  6867 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:50.584534  6606 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:50.582607  6723 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.654897  1553 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:50.542558  6863 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:50.559127  6753 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.655887  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:50.656683  2318 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:50.670526  2347 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:01:50.621300  6866 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.565904  6841 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.578806  6368 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.643818  6571 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:50.531939  6862 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.541889  6768 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.645650  6848 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.566125  6856 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.667058  6573 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:50.680084  6613 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:50.667412  6618 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.717362  2347 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:50.561774  6858 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.683857  6723 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:50.683226  6326 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:50.642760  6855 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:50.716152  6712 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.631s	user 0.012s	sys 0.001s
I20260709 13:01:50.740710  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:50.768343  1561 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:01:50.714484  6861 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.269s	user 0.000s	sys 0.002s
I20260709 13:01:50.759675  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:50.771694  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 289195us
I20260709 13:01:50.771874  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:50.772877  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:50.738853  6613 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.709233  6848 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:50.738134  6744 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.777352  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:50.677899  6867 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.685181  6863 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.699052  6866 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:50.774219  6821 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.764834  6864 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.770433  6326 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:50.780153  6540 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:50.792724  6847 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:50.833140  6868 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:50.833833  6847 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.833990  6868 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.834380  6847 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:50.834496  6868 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:50.783604  6613 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:50.838060  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:50.839370  1557 raft_consensus.cc:2410] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:01:50.766429  6723 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:50.847497  2346 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 1104) took 2764 ms (client timeout 2065 ms). Trace:
I20260709 13:01:50.742298  6606 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.850944  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:50.848026  2346 rpcz_store.cc:269] 0709 13:01:48.077514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.077692 (+   178us) service_pool.cc:224] Handling call
0709 13:01:50.841903 (+2764211us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b5400686d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:50.842052 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1921584128}
W20260709 13:01:50.854719  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.866110  1552 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:50.797087  6843 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.775674  6712 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:50.873066  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:50.972304  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.977576  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:50.981737  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:50.984479  6859 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.254s	user 0.000s	sys 0.002s
W20260709 13:01:50.985997  6727 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.929s	user 0.009s	sys 0.005s
W20260709 13:01:50.986361  6881 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:01:50.986522  6727 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:50.988070  6879 log.cc:927] Time spent T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:01:50.971887  6841 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.938386  6772 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.990924  6768 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.991878  6862 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.992390  6723 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.850674  6872 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.870122  6866 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.927618  6743 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.994822  6743 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:50.990749  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:50.938351  6763 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.982120  6606 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:50.976351  6810 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.968550  6857 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:50.847360  6540 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:50.990620  6858 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.990700  6855 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:51.005646  6759 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:51.003727  6727 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:51.024207  6863 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.863839  6821 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:51.000893  6883 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:50.836072  6864 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.029979  6856 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:50.979712  6326 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.043118  6752 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:51.039754  6867 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:51.047514  6873 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.259s	user 0.003s	sys 0.000s
I20260709 13:01:50.993366  6872 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:51.035859  6777 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.632s	user 0.014s	sys 0.000s
I20260709 13:01:51.053889  6733 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:51.050680  6864 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.048048  6712 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:51.061324  6789 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 2.526s	user 0.000s	sys 0.006s
I20260709 13:01:51.060918  6743 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:51.036832  6821 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:51.074509  6833 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.443s	user 0.004s	sys 0.000s
I20260709 13:01:51.082489  6821 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.053668  6847 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.091812  6868 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.132195  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.134997  1552 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:51.137697  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.138919  1552 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:51.140399  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.141789  1552 raft_consensus.cc:2410] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:01:51.144225  1554 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:01:51.145087  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.191493  1561 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:51.156684  2346 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:01:51.159454  6200 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.822s	user 0.017s	sys 0.002s
W20260709 13:01:51.159901  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 386918us
I20260709 13:01:51.048898  6540 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.168263  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.169356  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:51.170142  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:51.172298  1630 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:51.187467  6311 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.196777  6766 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.063844  6733 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:51.214727  6872 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.214866  1525 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:51.219648  6847 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:51.220573  6872 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:51.225322  6887 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:01:51.227346  6817 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.828s	user 0.006s	sys 0.002s
I20260709 13:01:51.243824  6733 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.249923  6311 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.230687  6766 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.257812  1627 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:51.210983  6200 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.258891  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:51.232679  6824 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.002s	sys 0.004s
I20260709 13:01:51.281713  1630 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:51.290053  6824 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.002s	sys 0.004s
I20260709 13:01:51.282080  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.282917  6796 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.284852  6733 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:51.284972  6200 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.300254  6311 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.301090  6766 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:51.318393  1630 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:51.330081  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2125) took 2997 ms. Trace:
I20260709 13:01:51.331321  6868 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:51.338737  1627 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:01:51.330859  1704 rpcz_store.cc:276] 0709 13:01:48.332060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.332224 (+   164us) service_pool.cc:224] Handling call
0709 13:01:48.801119 (+468895us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.801950 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.329272 (+2527322us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54000cb2d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:51.329643 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949959936}
I20260709 13:01:51.347321  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1105) took 3268 ms. Trace:
I20260709 13:01:51.348603  2345 rpcz_store.cc:276] 0709 13:01:48.078285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.177510 (+ 99225us) service_pool.cc:224] Handling call
0709 13:01:48.229891 (+ 52381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:49.699940 (+1470049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.333035 (+1633095us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400478954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:51.347074 (+ 14039us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3193848192}
I20260709 13:01:51.325973  6766 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:51.353957  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.356429  1630 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:51.357359  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.361941  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:51.359366  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.363014  2347 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:51.363773  1635 raft_consensus.cc:2410] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:01:51.367087  1630 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:51.373732  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.376324  6574 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:51.377849  1634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2652) took 1563 ms (client timeout 1734 ms). Trace:
W20260709 13:01:51.382570  1634 rpcz_store.cc:269] 0709 13:01:49.785657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.785880 (+   223us) service_pool.cc:224] Handling call
0709 13:01:51.349032 (+1563152us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b540010dbd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:51.349320 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":520424192}
I20260709 13:01:51.387530  6697 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:51.388211  6567 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:51.389055  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.379515  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:51.388154  6697 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:51.398773  6326 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.406492  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.393148  6697 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.414646  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:51.416734  6801 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:51.417207  6801 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:51.417654  6801 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:51.418486  1636 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
W20260709 13:01:51.426460  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2157) took 3506 ms (client timeout 2973 ms). Trace:
W20260709 13:01:51.427147  1556 rpcz_store.cc:269] 0709 13:01:47.920050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.920582 (+   532us) service_pool.cc:224] Handling call
0709 13:01:51.425384 (+3504802us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b5400311d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:51.426178 (+   794us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1861953408}
I20260709 13:01:51.431260  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 854) took 2478 ms. Trace:
I20260709 13:01:51.434722  1634 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:51.435529  2346 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:01:51.437067  2345 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:51.437225  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.438202  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:51.438678  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2878) took 2953 ms. Trace:
I20260709 13:01:51.439431  2351 raft_consensus.cc:2368] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12.
I20260709 13:01:51.389106  6567 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.442150  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.400583  6326 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.431978  1706 rpcz_store.cc:276] 0709 13:01:48.952516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.975224 (+ 22708us) service_pool.cc:224] Handling call
0709 13:01:50.187010 (+1211786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.187534 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.187574 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:50.188205 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 13:01:50.356342 (+168137us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:50.356414 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.356541 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.985697 (+629156us) raft_consensus.cc:1685] finished
0709 13:01:50.985738 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.430448 (+444710us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54001e9950. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:51.430974 (+   526us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":21526,"spinlock_wait_cycles":2750612736,"thread_start_us":167603,"threads_started":1,"wal-append.queue_time_us":348184}
I20260709 13:01:51.439157  1632 rpcz_store.cc:276] 0709 13:01:48.484860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.486497 (+  1637us) service_pool.cc:224] Handling call
0709 13:01:48.486759 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:49.343602 (+856843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.438056 (+2094454us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54002dbed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:51.438429 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816854400}
I20260709 13:01:51.451356  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.450201  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.455888  6844 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:51.456387  6844 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:51.457978  6844 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.460160  6844 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.408277  6824 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.462282  6745 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.464068  6326 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:51.464167  6745 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.466601  6697 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:51.446509  6766 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.462294  6824 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.476940  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:51.477057  6843 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:51.478194  1634 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:51.478298  6843 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.481295  6824 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.499903  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.503963  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.504228  6754 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.504877  1629 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:51.505548  6754 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.509418  6825 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.510797  6825 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.514416  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.515584  2351 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:51.516395  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 866) took 1086 ms. Trace:
I20260709 13:01:51.516846  2350 rpcz_store.cc:276] 0709 13:01:50.430483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.430670 (+   187us) service_pool.cc:224] Handling call
0709 13:01:50.467704 (+ 37034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.571549 (+103845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.515888 (+944339us) spinlock_profiling.cc:243] Waited 81.8 ms on lock 0x7b5400550354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:51.516192 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188207488}
I20260709 13:01:51.518554  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.519542  1702 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:51.522522  6368 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.528596  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.530224  6761 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:51.531037  6761 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.533782  6761 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.538668  2349 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:51.540935  6731 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:51.502297  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.547956  1631 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:01:51.552937  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 852) took 2634 ms (client timeout 2927 ms). Trace:
W20260709 13:01:51.556800  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.559829  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:51.560783  1703 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:51.556318  1705 rpcz_store.cc:269] 0709 13:01:48.918156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.958004 (+ 39848us) service_pool.cc:224] Handling call
0709 13:01:51.552497 (+2594493us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54002e95d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:51.552697 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":855277440}
I20260709 13:01:51.571352  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.572189  1708 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:51.572605  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.573553  1702 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:51.574548  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.574767  1459 catalog_manager.cc:5697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 7 to 10, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 10 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:51.579463  6843 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.582429  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.553265  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.502946  6766 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:51.582557  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:51.545939  6745 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:51.594373  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.600442  1701 raft_consensus.cc:2368] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:01:51.601923  2346 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:01:51.586525  6843 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:51.590858  6848 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.650s	user 0.008s	sys 0.017s
I20260709 13:01:51.608664  2347 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:01:51.606130  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.611992  6567 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:51.613049  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:51.617941  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.620173  1556 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:01:51.620536  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.621131  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.621424  1560 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:51.621817  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.622612  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:49.781870   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:51.625417  6843 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.627964  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.605883  6824 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.622112  6645 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:51.632845  1561 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:01:51.631578  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:51.623694  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.635918  1703 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:51.637027  1706 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:51.637209  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.638200  1552 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:51.638372  1559 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:51.638895  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.639432  6659 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.640103  6659 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:51.641649  6899 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:01:51.642541  6659 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:51.645813  6745 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:51.645859  6738 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.646703  6745 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.646977  6738 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:01:51.648279  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 66031.9us
I20260709 13:01:51.649211  1527 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:51.650636  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.651213  6853 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.651819  1701 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:01:51.652249  6853 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.654294  6738 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:51.653285  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.655484  1701 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:51.655894  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.656502  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.657539  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.658035  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.658421  2352 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:51.659219  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.661562  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.660295  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.617065  6848 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.641153  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:51.674398  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.675439  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.680162  1629 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:51.676946  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.683065  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:51.683790  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.683864  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.663852  6745 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:51.678175  1708 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:51.685427  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.612632  6801 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.686271  2350 raft_consensus.cc:2410] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:51.682031  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.689280  1704 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:51.689384  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.688284  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.691915  1554 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:01:51.673115  6725 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:51.738471  6725 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:51.739998  6725 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.694283  1527 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:51.688115  1628 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:01:51.701030  6804 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 13:01:51.702124  6754 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.704779  1636 raft_consensus.cc:2410] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:01:51.705433  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.692996  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:51.706439  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 65532.5us
I20260709 13:01:51.710079  6326 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:51.710932  6745 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:51.714171  2352 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:51.715093  1702 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:51.715106  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.665591  6824 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.717962  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.717902  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:51.718519  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.718833  6368 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:51.719313  1558 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:51.721957  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.735363  6848 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:51.737058  6853 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.737692  6825 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:51.791139  1701 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:01:51.770407  1635 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
W20260709 13:01:51.792546  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 118760us
I20260709 13:01:51.778528  1628 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:01:51.779472  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 857) took 2200 ms (client timeout 1838 ms). Trace:
I20260709 13:01:51.780454  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:51.781286  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.716347  6801 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:51.761387  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 135130us
I20260709 13:01:51.789812  1560 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:01:51.794576  1559 raft_consensus.cc:2410] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:01:51.795120  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.797049  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.818935  1552 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:01:51.809445  1710 rpcz_store.cc:269] 0709 13:01:49.579079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.579959 (+   880us) service_pool.cc:224] Handling call
0709 13:01:51.694525 (+2114566us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54001e8a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:51.779235 (+ 84710us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1594908416}
I20260709 13:01:51.823537  1703 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:01:51.802801  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.811350  1554 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:51.835887  1631 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:01:51.829089  1556 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:51.839875  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.841727  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:51.836571  1631 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:01:51.790727  6368 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.817708  6754 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:51.846426  1703 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:51.852180  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.819993  6745 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:51.854795  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.856627  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.857342  1710 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:51.858383  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:51.859817  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.860585  1710 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:51.862582  6571 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.956s	user 0.020s	sys 0.008s
I20260709 13:01:51.862366  6745 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:51.867063  6870 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.093s	user 0.000s	sys 0.007s
I20260709 13:01:51.869122  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 865) took 1441 ms. Trace:
I20260709 13:01:51.869592  2344 rpcz_store.cc:276] 0709 13:01:50.427574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.427804 (+   230us) service_pool.cc:224] Handling call
0709 13:01:50.428293 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.428974 (+   681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.429025 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:50.429769 (+   744us) log.cc:844] Serialized 26 byte log entry
0709 13:01:50.602501 (+172732us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:50.602549 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.602623 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.865915 (+1263292us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:51.868516 (+  2601us) raft_consensus.cc:1685] finished
0709 13:01:51.868538 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.868916 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1282,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":28809,"spinlock_wait_cycles":46536448,"thread_start_us":118652,"threads_started":1,"wal-append.queue_time_us":250441}
I20260709 13:01:51.871517  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 777) took 1141 ms. Trace:
I20260709 13:01:51.846201  6326 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.871959  1553 rpcz_store.cc:276] 0709 13:01:50.730529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.730798 (+   269us) service_pool.cc:224] Handling call
0709 13:01:50.736501 (+  5703us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:51.001820 (+265319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.001854 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:51.002696 (+   842us) log.cc:844] Serialized 26 byte log entry
0709 13:01:51.050086 (+ 47390us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b4800002958. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:01:51.869822 (+819736us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b54004847a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27644090 00007fda276516b8 00007fda276da92b 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c
0709 13:01:51.869929 (+   107us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:51.869984 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.870990 (+  1006us) spinlock_profiling.cc:243] Waited 47.6 ms on lock 0x7b54003048d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:51.871034 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.871082 (+    48us) raft_consensus.cc:1685] finished
0709 13:01:51.871098 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.871297 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":171402,"raft.run_cpu_time_us":2251,"raft.run_wall_time_us":92345,"spinlock_wait_cycles":160811520,"thread_start_us":303061,"threads_started":2,"wal-append.queue_time_us":123941}
I20260709 13:01:51.873960  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:51.875129  6613 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.019s	user 0.010s	sys 0.015s
I20260709 13:01:51.848483  6754 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.875995  6754 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:51.875841  6613 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:51.879113  6892 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.665s	user 0.001s	sys 0.001s
I20260709 13:01:51.879500  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.880810  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.881779  1557 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:51.884356  6868 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:51.885660  6868 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:51.887060  6868 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.890944  6630 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.891975  1675 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:51.891911  6630 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.895427  6540 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:51.893743  6542 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:51.896871  6542 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:51.897500  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 117220us
I20260709 13:01:51.899274  6085 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.899816  6542 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:51.900070  6085 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:51.900707  6542 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:51.875399  1558 raft_consensus.cc:2410] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:01:51.903100  6326 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:51.905565  6085 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:51.903371  6630 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:51.897107  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.863157  6571 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:51.911945  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.861433  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.918924  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:51.920279  1708 raft_consensus.cc:2368] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12.
I20260709 13:01:51.848580  6368 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:51.916268  1710 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:51.844755  1631 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 12.10
I20260709 13:01:51.931533  1631 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Aborting config change with OpId 12.10: Aborted: Op aborted by new leader
W20260709 13:01:51.935436  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43811.1us
I20260709 13:01:51.944296  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.947419  1627 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:51.947779  1632 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:01:51.950434  1705 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:01:51.950958  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.952116  1633 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:01:51.952510  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.952919  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.952523  1633 pending_rounds.cc:85] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 8
I20260709 13:01:51.957754  1633 pending_rounds.cc:107] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:01:51.958098  1633 raft_consensus.cc:2889] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:51.959646  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.957743  1560 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:01:51.961808  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.963165  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.932417  1559 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:51.949636  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.966253  1631 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:51.950649  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.970072  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:51.969743  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:51.977036  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.979122  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:51.996999  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:52.003378  1634 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.002s
I20260709 13:01:51.990775  1633 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:52.003810  1634 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:52.000324  1710 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:52.006428  1555 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:52.006083  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:51.922494  6542 consensus_queue.cc:1243] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:52.022369  6618 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:01:52.023480  6736 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:52.024071  6736 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:52.024634  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.025770  1705 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:52.025341  6736 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:52.026665  6733 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.006s
I20260709 13:01:52.027024  6736 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:52.027352  1705 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:52.027424  1709 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:52.029927  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.030361  6645 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:52.030861  1706 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:01:52.031107  6645 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.033689  6756 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 13:01:52.035007  6567 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:52.035465  6567 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:52.037104  6567 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:52.038022  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 71091.3us
I20260709 13:01:52.038861  6567 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.037575  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:52.041198  2348 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:52.042354  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:52.043121  1559 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:52.038980  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:52.053876  1633 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:52.054304  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 48042.2us
I20260709 13:01:52.061349  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.059571  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:52.066072  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.071034  1633 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:52.077870  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:52.078884  2348 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:52.082515  1558 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:52.084090  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.084916  1558 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:52.089442  1553 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:52.020617  6844 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:52.093422  1704 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:52.096091  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.100966  1710 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:52.100679  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:52.102557  1705 raft_consensus.cc:2368] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12.
I20260709 13:01:52.096671  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.105829  1633 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:52.111789  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.113158  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.113926  1558 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:01:52.116686  1555 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:52.110896  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:52.118366  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.122435  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.124575  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.125309  1702 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:52.146035  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:52.128443  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.130345  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.134752  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:52.151180  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.102555  6613 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:52.154330  2348 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 10: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:52.111433  6756 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:52.155233  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.149835  1636 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:52.157436  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.160204  1705 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:52.121833  6733 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:52.162190  1627 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:52.160785  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:52.166179  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 127292us
I20260709 13:01:52.167568  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:52.107652  6844 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:52.169792  6824 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:52.175329  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.176594  1710 raft_consensus.cc:2410] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:01:52.177436  1558 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:52.181054  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:52.187664  1636 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:52.189393  1553 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:52.150549  6853 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:01:52.192418  1702 raft_consensus.cc:2368] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12.
I20260709 13:01:52.195420  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.196792  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.198369  1627 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:52.179633  6645 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:52.204391  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.181802  6697 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:52.182215  6571 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:52.186113  6824 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:52.155473  6754 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.230506  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:52.228303  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 221745us
I20260709 13:01:52.237628  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.196921  6747 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:52.246224  1703 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:52.251802  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:52.255210  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 87774.2us
I20260709 13:01:52.235949  6697 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:52.257627  1560 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:52.258759  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.271037  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.276021  1708 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:52.282533  2318 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:52.284709  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:52.285415  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 403563us
I20260709 13:01:52.286790  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:52.287004  6799 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:52.247553  6645 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:52.285764  1553 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:52.291648  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:52.295301  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.295601  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 65230.9us
I20260709 13:01:52.299440  1554 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:01:52.304953  1633 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:52.314291  1636 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:52.326115  1558 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:52.301772  6825 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.327278  1525 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.267817  6747 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.335062  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.300421  6571 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:01:52.347160  1633 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:52.355544  6611 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:01:52.338860  6567 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:52.372457  6368 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.325760  6853 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:01:52.365667  1602 leader_election.cc:400] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12.
I20260709 13:01:52.380192  1602 leader_election.cc:403] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:01:52.369994  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:52.373070  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 86420us
I20260709 13:01:52.366559  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.317920  6874 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.067s
W20260709 13:01:52.378545  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 297872us
I20260709 13:01:52.341609  6825 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:52.379956  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.345146  6756 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:52.364084  6844 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:52.382125  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 79717.7us
I20260709 13:01:52.357857  6645 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:52.379652  6697 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.394716  6874 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:52.395566  6874 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:52.397595  6868 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:01:52.400296  6799 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.401789  6540 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:52.402204  6618 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:52.401109  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:52.407488  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.386031  6326 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:52.418985  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.414583  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.387012  6567 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:52.389878  6645 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.425058  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.422467  2317 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:52.426503  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.427410  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:52.427605  1554 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:01:52.427943  1553 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:52.425633  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66792.1us
I20260709 13:01:52.438612  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.440420  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.424085  6730 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:52.426630  6645 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.397082  6874 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.446341  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.443323  1557 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:52.425822  6919 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.444521  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.415315  6824 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:01:52.441560  1631 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:52.445809  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.447835  1633 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:52.433931  6611 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.435846  6613 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:52.454715  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:52.450359  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:52.455379  2351 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.455725  2349 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:52.460242  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.461639  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:52.425141  6326 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.463244  1554 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:01:52.463236  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:52.458361  1675 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.442654  6730 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.442391  6910 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:52.467700  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:52.469204  2344 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.454609  6853 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:52.466485  1602 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:52.471841  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 70896.9us
I20260709 13:01:52.485213  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:52.493891  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 82754.5us
W20260709 13:01:52.494351  6918 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:01:52.495760  2317 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:52.497623  6611 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.501827  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.463567  6843 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:52.470268  6910 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.505805  6874 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.506120  6910 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:52.506376  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.508054  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:52.509801  6630 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:52.471923  6730 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.512845  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.515949  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.518186  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:52.510964  1675 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:52.523473  6368 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:52.486658  6754 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:52.533015  1559 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 9.
I20260709 13:01:52.484683  6659 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:52.546154  2349 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:52.548404  1633 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:52.550498  1559 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:52.559396  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.560258  1630 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:01:52.457850  6919 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:01:52.568642  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 66922.7us
I20260709 13:01:52.474500  6567 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.530241  6611 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:52.546015  6659 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.530218  6618 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:01:52.549621  6630 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:52.552704  6843 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:52.553833  6745 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:52.605935  1553 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:52.608098  6928 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:52.610309  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 163151us
I20260709 13:01:52.611030  1675 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.611583  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.615507  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:52.605556  6659 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.614288  6618 consensus_queue.cc:1243] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:52.617854  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.608142  6645 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:52.613945  6611 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:52.627450  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.629328  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:52.630199  1633 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:01:52.630245  6921 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:52.631044  6921 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.631430  6921 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:52.627750  6736 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.600s	user 0.014s	sys 0.003s
I20260709 13:01:52.640580  6736 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:52.641062  6736 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.641453  6736 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.641556  6630 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:01:52.641830  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 163325us
I20260709 13:01:52.642218  6630 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.642699  6630 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:52.643677  6924 raft_consensus.cc:3055] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:52.644114  6924 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:52.616603  6843 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:52.645460  6843 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.645680  6919 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:52.646340  6571 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:52.618557  6745 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:52.648448  6745 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:52.650309  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.650718  6326 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:52.651206  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.621747  6928 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.651913  6928 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.657462  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.660331  6738 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:52.660885  6738 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:52.660975  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:52.662369  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.662113  6738 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.663301  1627 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:52.663496  2351 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:52.664213  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:52.664778  1627 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:52.650807  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 155125us
W20260709 13:01:52.671130  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 121270us
I20260709 13:01:52.677439  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.678361  1676 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:01:52.687894  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 76394.4us
I20260709 13:01:52.689317  2344 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:52.629606  6756 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:52.696259  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.697499  1630 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:52.663424  6738 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.701236  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:52.706956  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.709870  1632 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:52.710209  1602 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:52.710851  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:52.708873  6801 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:01:52.713707  6733 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:52.717826  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:52.719029  6930 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:52.719592  6930 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.719987  6930 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.722815  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:52.724041  6326 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:52.724632  6326 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:52.725281  1527 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:52.725806  6326 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.727208  6730 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:52.731600  1674 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:52.732533  1553 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:01:52.733296  1631 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:52.734061  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:52.705888  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 240537us
W20260709 13:01:52.745208  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46104us
I20260709 13:01:52.695261  6756 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:52.703065  6919 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:52.755767  1627 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:52.753718  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.770115  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:52.706688  6799 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12.
I20260709 13:01:52.767119  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.755759  6730 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:52.756547  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:52.774250  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.776852  1553 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:52.779819  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:52.779889  1627 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:52.756862  6756 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:52.784036  2349 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 9: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:52.767437  6919 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.645131  6924 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.752231  6936 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:52.769552  6733 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:52.818459  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 118500us
I20260709 13:01:52.772461  6929 raft_consensus.cc:2764] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:01:52.823661  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:52.808511  6754 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:52.821851  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:52.825692  1553 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:52.821749  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 51748.3us
I20260709 13:01:52.805429  6919 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:52.835655  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 85465.5us
I20260709 13:01:52.831553  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:52.807902  6085 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:01:52.836483  1556 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:01:52.837177  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.840116  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:52.840786  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.805610  6824 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:52.844074  6909 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:52.821224  6801 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:52.845598  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.845758  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:52.846025  6924 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:52.846788  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:52.821096  6799 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:52.844906  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.852303  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:52.853250  6894 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:52.884732  6747 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:52.895015  6894 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.895175  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:52.895658  6894 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:52.896051  1558 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.849248  6085 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:52.827095  6730 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:52.897722  6085 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:52.898140  6730 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:52.899423  6730 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.862632  6645 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:52.863680  6368 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.833328  6540 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:52.868510  6801 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.873888  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.875201  1708 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
W20260709 13:01:52.915409  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 45401.4us
I20260709 13:01:52.875976  1702 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
W20260709 13:01:52.877311  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.878254  1703 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:52.874682  1633 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
W20260709 13:01:52.879503  2317 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.547s (SENT) [suppressed 13 similar messages]
W20260709 13:01:52.903848  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 68553.7us
W20260709 13:01:52.909591  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 72009.8us
I20260709 13:01:52.883445  6659 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:52.923779  6540 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.926414  6540 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:52.914657  6747 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:52.926964  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:52.927466  6747 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:52.927976  6747 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:52.928192  1555 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:52.929714  6733 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:52.933244  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.936414  1675 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:52.941886  1553 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:52.942752  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.943253  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:52.920444  6801 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.947973  6910 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:52.949545  6934 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:52.950089  6934 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.950505  6934 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.889236  6799 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.954640  2348 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:01:52.953804  6824 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:01:52.956164  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 255111us
I20260709 13:01:52.974498  1554 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:52.976019  1676 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:52.967171  6799 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.870808  6933 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.344s
I20260709 13:01:52.959781  6540 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:52.970513  6754 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.959326  6874 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:52.993079  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:53.007576  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:53.006692  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:53.010656  1602 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:53.008328  2351 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:01:52.994838  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 248617us
I20260709 13:01:52.967048  6931 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:52.995620  1553 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:53.022197  1553 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:52.972733  6824 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.000532  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:53.000595  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 269069us
I20260709 13:01:52.920955  6368 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.023788  1555 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.000310  6804 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:52.914992  6909 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.003713  6874 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.967507  6745 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:53.085791  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:53.088963  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 65106.1us
I20260709 13:01:53.088804  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 864) took 1522 ms. Trace:
W20260709 13:01:53.089920  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:53.090174  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 89491.4us
I20260709 13:01:53.064250  6931 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:53.065675  6824 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:53.090026  1709 rpcz_store.cc:276] 0709 13:01:51.560842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.581684 (+ 20842us) service_pool.cc:224] Handling call
0709 13:01:53.082796 (+1501112us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b5400336b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:53.083025 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1020850560}
W20260709 13:01:53.097684  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.098281  2349 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:52.999811  6736 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.073846  6611 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.110234  6951 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:53.098858  6931 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.110329  6611 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.112838  6611 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.005015  6754 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.113575  6940 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:53.114961  6921 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.115593  6754 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.109009  1602 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:53.085554  6874 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:53.102942  6949 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:53.093545  6927 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:53.108234  6925 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:53.101943  6929 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.209978  6921 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.213680  6925 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.213250  1555 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:53.213136  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2677) took 1531 ms. Trace:
W20260709 13:01:53.222065  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 199962us
I20260709 13:01:53.227648  1555 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:01:53.092654  6613 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
W20260709 13:01:53.231799  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 352310us
I20260709 13:01:53.200664  6611 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:53.186578  6940 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:53.203603  6927 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:53.220800  6925 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:53.244298  2349 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
W20260709 13:01:53.237908  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 231340us
W20260709 13:01:53.250130  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 141218us
I20260709 13:01:53.251106  1602 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:53.220196  1629 rpcz_store.cc:276] 0709 13:01:51.681199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.681377 (+   178us) service_pool.cc:224] Handling call
0709 13:01:51.685166 (+  3789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:51.688818 (+  3652us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:52.126005 (+437187us) log.cc:844] Serialized 14 byte log entry
0709 13:01:53.212519 (+1086514us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b18003affc0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287
0709 13:01:53.212663 (+   144us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:53.212725 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:53.212789 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.212879 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.213030 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":586133,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":32196,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":7596,"spinlock_wait_cycles":37465600,"thread_start_us":614688,"threads_started":3,"wal-append.queue_time_us":190741}
I20260709 13:01:53.244951  2349 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 9
I20260709 13:01:53.277753  2349 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 12.10
I20260709 13:01:53.278172  2349 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Aborting config change with OpId 12.10: Aborted: Op aborted by new leader
I20260709 13:01:53.278826  2349 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:53.256287  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:53.230099  6910 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.293421  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:53.298576  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:53.299345  2347 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:53.301991  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:53.250432  6927 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:53.304533  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:53.253239  6804 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:53.216200  6943 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:01:53.286676  6949 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:53.304471  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 214694us
I20260709 13:01:53.310330  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:53.331888  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 70687us
W20260709 13:01:53.339107  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 88626.1us
W20260709 13:01:53.352186  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:53.357015  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 52088.8us
W20260709 13:01:53.330334  6928 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.671s	user 0.010s	sys 0.003s
I20260709 13:01:53.228936  6929 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.312213  6949 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:53.366106  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.319974  6611 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:53.380770  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:53.381973  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:53.384114  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 137541us
W20260709 13:01:53.395530  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 85298.8us
I20260709 13:01:53.397065  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:53.371183  1526 connection.cc:256] RPC call timeout handler was delayed by -1.1059250459999816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
W20260709 13:01:53.402719  1526 connection.cc:256] RPC call timeout handler was delayed by -1.0759322539999516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
I20260709 13:01:53.397998  1708 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:01:53.431365  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.445894  1602 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:53.380982  6611 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:53.444244  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:53.449738  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:53.385610  6949 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.399283  6368 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:53.400373  6940 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.374253  6928 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.454524  6645 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:01:53.488459  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.473948  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:53.474154  6630 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.831s	user 0.013s	sys 0.000s
I20260709 13:01:53.476374  6951 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:53.472759  2352 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 13:01:53.484279  6919 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.000s
W20260709 13:01:53.488965  6801 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.543s	user 0.012s	sys 0.000s
I20260709 13:01:53.390352  6571 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:01:53.460944  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:53.479800  6843 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.834s	user 0.008s	sys 0.004s
I20260709 13:01:53.538044  1636 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:01:53.460144  1556 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.595s	user 0.007s	sys 0.005s
I20260709 13:01:53.429615  6745 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:53.486438  6946 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:01:53.509670  6969 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.119s
W20260709 13:01:53.558167  6738 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.848s	user 0.009s	sys 0.004s
I20260709 13:01:53.558755  6801 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.561100  6801 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:53.536980  1558 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.001s
I20260709 13:01:53.570564  1558 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:53.444813  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:53.546603  1556 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:53.530058  6951 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.461015  6611 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:53.556382  6747 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.000s
W20260709 13:01:53.562502  6799 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.000s
I20260709 13:01:53.585266  6799 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.587603  6799 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:53.587738  6756 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.770s	user 0.006s	sys 0.005s
I20260709 13:01:53.588320  6756 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.588884  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:53.589746  1706 raft_consensus.cc:2368] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:01:53.590162  6909 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.590622  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:53.591403  1706 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:53.564584  1708 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:53.604193  6924 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.699s	user 0.006s	sys 0.003s
I20260709 13:01:53.592512  6927 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.608072  2347 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:53.609428  6927 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.611359  6871 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:53.612068  6871 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:53.611444  6874 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.532146  6947 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:53.613922  6947 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.616955  6947 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:53.581909  6085 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.684s	user 0.014s	sys 0.000s
I20260709 13:01:53.641805  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:53.531489  6736 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:53.642244  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:53.538968  6930 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.818s	user 0.019s	sys 0.000s
I20260709 13:01:53.584272  6747 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.652875  1628 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:53.578598  6951 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.592486  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:53.593698  2350 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.594578  2353 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:53.661504  1709 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.530851  6630 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.563463  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:53.665617  1704 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:01:53.487998  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 57228.8us
I20260709 13:01:53.596691  6731 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:53.597265  6928 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.598558  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:53.599434  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2799) took 1559 ms. Trace:
I20260709 13:01:53.670677  1560 rpcz_store.cc:276] 0709 13:01:52.039647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.144928 (+105281us) service_pool.cc:224] Handling call
0709 13:01:53.599183 (+1454255us) spinlock_profiling.cc:243] Waited 88.9 ms on lock 0x7b54003309d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:53.599344 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204559232}
W20260709 13:01:53.601572  6904 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.368s	user 0.006s	sys 0.000s
I20260709 13:01:53.605674  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:53.564271  6934 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.613s	user 0.013s	sys 0.000s
W20260709 13:01:53.619494  6931 consensus_meta.cc:287] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 13:01:53.486071  6645 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.522929  6804 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.622119  6929 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:53.569300  6843 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.631436  1554 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:53.548121  6919 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.692550  6919 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.644189  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:53.646962  1558 raft_consensus.cc:1217] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 5.5->[7.6-15.10]   Dedup: 12.8->[14.9-15.10]
I20260709 13:01:53.693382  1559 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:53.696575  1558 pending_rounds.cc:85] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 8
I20260709 13:01:53.697005  1558 pending_rounds.cc:107] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:01:53.664697  6659 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:53.693889  1631 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:01:53.681638  6931 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.695132  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:53.699084  6630 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:53.531750  6910 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:53.699760  6910 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:53.589085  6894 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.693s	user 0.012s	sys 0.000s
I20260709 13:01:53.653859  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:53.665959  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 126797us
I20260709 13:01:53.704553  1635 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:53.663434  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:53.667433  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:53.705829  1634 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.639421  1675 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:01:53.674374  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2161) took 1514 ms (client timeout 1871 ms). Trace:
W20260709 13:01:53.708309  1705 rpcz_store.cc:269] 0709 13:01:52.159694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.598267 (+1438573us) service_pool.cc:224] Handling call
0709 13:01:53.674275 (+ 76008us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.639638  6825 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.680047  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:53.667423  6731 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:53.683947  6747 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:53.690246  6843 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:53.697326  1558 raft_consensus.cc:2889] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:53.699859  6931 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:53.639564  6738 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:53.651467  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:53.749245  1555 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.750109  6731 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.751307  6731 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:53.700727  1629 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:53.752830  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:53.753631  1629 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:53.754535  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:53.755124  1629 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:53.755918  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:53.757016  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:53.757076  6853 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:53.757736  6853 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:53.759850  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 889) took 1314 ms. Trace:
I20260709 13:01:53.760277  2348 rpcz_store.cc:276] 0709 13:01:52.445806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.445982 (+   176us) service_pool.cc:224] Handling call
0709 13:01:52.457635 (+ 11653us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:52.967365 (+509730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.759559 (+792194us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b54004f4a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:53.759780 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1142262528}
I20260709 13:01:53.766263  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 2908) took 1117 ms. Trace:
I20260709 13:01:53.700294  6910 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.660584  6085 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.777940  6085 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.779126  6925 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.780594  1560 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
W20260709 13:01:53.713739  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 74407.3us
I20260709 13:01:53.677855  6934 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.782712  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:53.652745  6540 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:01:53.783631  6542 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:53.784343  6542 consensus_queue.cc:1243] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:53.784369  6934 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.766781  1636 rpcz_store.cc:276] 0709 13:01:52.648837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.650936 (+  2099us) service_pool.cc:224] Handling call
0709 13:01:53.766035 (+1115099us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b540017a7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:53.766191 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2037213952}
I20260709 13:01:53.718343  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:53.720364  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:53.653151  6932 consensus_queue.cc:1059] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:01:53.741655  6967 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:53.733857  6645 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:53.661309  6930 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.704882  6799 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:53.746457  6924 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:53.746766  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:53.613066  6871 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.716483  6894 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:53.709626  1628 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:53.767719  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2920) took 1054 ms. Trace:
I20260709 13:01:53.749266  6756 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:53.789446  1561 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:53.789870  1629 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:53.788439  1555 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:53.804425  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:53.802886  1635 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:53.809430  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:53.798024  1633 rpcz_store.cc:276] 0709 13:01:52.713575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.933687 (+220112us) service_pool.cc:224] Handling call
0709 13:01:52.934128 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:52.971127 (+ 36999us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:53.229829 (+258702us) log.cc:844] Serialized 14 byte log entry
0709 13:01:53.545422 (+315593us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:53.545993 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:01:53.546143 (+   150us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:53.546398 (+   255us) log.cc:844] Serialized 14 byte log entry
0709 13:01:53.546564 (+   166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.546642 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.767475 (+220833us) raft_consensus.cc:1685] finished
0709 13:01:53.767505 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.767656 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22486,"prepare.queue_time_us":353766,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":91,"raft.queue_time_us":304422,"raft.run_cpu_time_us":552,"raft.run_wall_time_us":94409,"thread_start_us":492923,"threads_started":2,"wal-append.queue_time_us":424066}
I20260709 13:01:53.804593  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:53.812690  1629 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.807813  6931 raft_consensus.cc:3055] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:53.714521  6947 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:53.816550  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.788851  6659 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:53.818872  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:53.795356  6799 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.822276  1556 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:53.812996  6967 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:53.875191  1636 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:53.701767  6567 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.109s
I20260709 13:01:53.814049  6645 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:53.785166  6540 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:53.806715  6924 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:53.826252  1635 raft_consensus.cc:2410] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:01:53.828917  1552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2800) took 1789 ms (client timeout 1988 ms). Trace:
I20260709 13:01:53.818125  6871 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:53.873117  1555 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:53.874864  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:53.821643  6326 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:53.824198  6874 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:53.880690  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 134043us
I20260709 13:01:53.829841  6931 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:53.881735  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2916) took 1221 ms. Trace:
I20260709 13:01:53.892513  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:53.893698  2350 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:53.828338  6925 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:53.899988  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.015s	sys 0.021s
W20260709 13:01:53.891638  1552 rpcz_store.cc:269] 0709 13:01:52.039940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.589952 (+550012us) service_pool.cc:224] Handling call
0709 13:01:53.817999 (+1228047us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b54001ce9a8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4b7924 00007fda2c6a0227 00007fda2c69fab8 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6
0709 13:01:53.828822 (+ 10823us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11031808}
I20260709 13:01:53.900300  1634 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:53.868690  6611 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:53.994467  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.017s	sys 0.034s
I20260709 13:01:53.828119  6927 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.902848  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:53.904249  1636 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:01:53.874519  6659 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:53.908309  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.996004  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:53.969545  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:53.996600  1555 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:01:53.968566  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:53.965219  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.877652  6799 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:53.882195  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 491223us
W20260709 13:01:53.994860  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 678871us
W20260709 13:01:53.911860  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.998411  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.002723  1561 raft_consensus.cc:2368] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12.
W20260709 13:01:54.004634  6922 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.314s	user 0.013s	sys 0.032s
I20260709 13:01:54.004796  1636 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:01:54.005165  6922 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.494s	user 0.014s	sys 0.032s
I20260709 13:01:54.006412  6894 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:54.007795  6930 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:54.007661  6659 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.009014  6659 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:54.008634  6967 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.009608  6874 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:53.882422  6645 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:54.019999  1709 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:53.992122  1632 rpcz_store.cc:276] 0709 13:01:52.672651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.769018 (+ 96367us) service_pool.cc:224] Handling call
0709 13:01:52.783546 (+ 14528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.083444 (+299898us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:53.309493 (+226049us) log.cc:844] Serialized 14 byte log entry
0709 13:01:53.435985 (+126492us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:53.456637 (+ 20652us) log.cc:844] Serialized 26 byte log entry
0709 13:01:53.456891 (+   254us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:53.457165 (+   274us) log.cc:844] Serialized 14 byte log entry
0709 13:01:53.881210 (+424045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.881339 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.881406 (+    67us) raft_consensus.cc:1685] finished
0709 13:01:53.881430 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.881645 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6324,"prepare.queue_time_us":340992,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":102,"raft.queue_time_us":263595,"raft.run_cpu_time_us":506,"raft.run_wall_time_us":49271,"thread_start_us":961996,"threads_started":4,"wal-append.queue_time_us":138526}
I20260709 13:01:54.050971  1553 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:54.074026  1628 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:54.071393  1636 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:54.038290  6824 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.852s	user 0.011s	sys 0.003s
I20260709 13:01:54.076138  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:54.078680  1552 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:54.063359  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:54.047479  1555 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
W20260709 13:01:54.081118  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.044293  6645 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:54.086982  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 118519us
I20260709 13:01:53.881978  6947 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:53.909456  6540 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:54.066349  6951 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.062305  6844 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:54.101145  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:54.068856  6611 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:54.111050  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 789) took 1753 ms. Trace:
I20260709 13:01:54.109050  1561 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:54.114336  1559 rpcz_store.cc:276] 0709 13:01:52.358140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.552675 (+194535us) service_pool.cc:224] Handling call
0709 13:01:52.586108 (+ 33433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.696122 (+1110014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.110708 (+414586us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400313654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:54.110977 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2545990400}
W20260709 13:01:54.138702  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 140371us
I20260709 13:01:54.054697  6925 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:53.938064  6326 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.049978  6927 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.150537  1635 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.154313  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:54.147957  6930 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:54.157056  6951 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:54.157691  6930 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.116147  6874 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:54.149750  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 122140us
I20260709 13:01:54.161293  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:54.113579  6844 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:54.149518  6949 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.623s	user 0.005s	sys 0.009s
I20260709 13:01:54.166688  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:54.095832  6738 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:01:54.157203  6982 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:54.091926  6824 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.150098  6567 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:54.165053  6951 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:54.179872  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:54.202908  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 89863.3us
I20260709 13:01:54.203795  6951 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.205217  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:01:54.201516  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 400017us
I20260709 13:01:54.211407  2351 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:54.147503  6542 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:01:54.216923  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:01:54.225348  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45583.8us
I20260709 13:01:54.170490  6736 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:01:54.206187  6951 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.146407  6924 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.228776  6611 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:54.228921  6956 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:54.229668  6979 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:54.229877  6824 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:54.220033  6970 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.286710  1708 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:54.231321  6915 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.752s	user 0.017s	sys 0.020s
I20260709 13:01:54.233946  6606 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.234618  6927 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:54.214493  6804 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:54.235517  6978 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.346s	user 0.001s	sys 0.000s
I20260709 13:01:54.153846  6925 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.241024  1629 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:54.201035  6982 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:54.132138  6931 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.249019  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.295042  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:54.295433  1526 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:54.235538  6930 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:54.325002  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 108196us
I20260709 13:01:54.206836  6567 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.248615  6611 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:54.178054  6949 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.344295  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:54.251832  6756 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:54.324328  6910 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.001s
I20260709 13:01:54.296011  6956 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:54.364368  1709 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.294785  6970 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:54.381042  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.389839  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:54.236832  6844 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.396392  1708 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:54.367046  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.227989  6540 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.358301  6611 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:54.333978  6606 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.362876  6927 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:54.351774  6940 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:01:54.414170  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:54.393028  6804 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:54.433771  6804 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.434864  6804 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.442566  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:54.442943  6799 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.444072  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:54.420385  1709 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:54.424300  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:54.424949  6606 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:54.426401  6989 log.cc:927] Time spent T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:01:54.409796  6844 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.386536  6756 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.427721  6982 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.428602  6927 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.430822  6970 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:54.408716  6731 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:54.471354  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47210us
I20260709 13:01:54.472102  1552 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:01:54.477176  6934 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:01:54.426774  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.478965  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1323) took 1169 ms. Trace:
I20260709 13:01:54.386914  6910 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.480090  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:54.474500  6947 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:54.502242  2345 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:01:54.502276  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.472455  1635 raft_consensus.cc:2410] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:54.498220  2344 rpcz_store.cc:276] 0709 13:01:53.309651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.312462 (+  2811us) service_pool.cc:224] Handling call
0709 13:01:53.350663 (+ 38201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.350942 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:53.350968 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:53.364007 (+ 13039us) log.cc:844] Serialized 26 byte log entry
0709 13:01:53.659852 (+295845us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:53.659904 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.660005 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:54.234475 (+574470us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:54.476122 (+241647us) raft_consensus.cc:1685] finished
0709 13:01:54.476145 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.478882 (+  2737us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23628544,"thread_start_us":228920,"threads_started":1,"wal-append.queue_time_us":420353}
I20260709 13:01:54.505869  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:54.506384  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:54.506934  1708 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.507778  1709 raft_consensus.cc:2368] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12.
I20260709 13:01:54.488860  6326 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:54.491003  6922 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:54.519186  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 45300.3us
I20260709 13:01:54.497517  6934 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:54.503782  6799 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:54.512905  6982 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.529234  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:54.529810  1709 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:54.529618  2345 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:54.532595  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:54.533313  1705 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:54.517879  6738 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:54.535681  6542 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:54.552681  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:54.540053  6611 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:54.544368  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.545785  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2917) took 1836 ms. Trace:
I20260709 13:01:54.547223  6804 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.549968  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1320) took 1274 ms. Trace:
I20260709 13:01:54.561167  2350 rpcz_store.cc:276] 0709 13:01:53.275749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.275922 (+   173us) service_pool.cc:224] Handling call
0709 13:01:54.549726 (+1273804us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54009902d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:54.549862 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659748224}
I20260709 13:01:54.563337  2345 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:54.524799  6934 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:54.556557  2344 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:54.558486  1631 rpcz_store.cc:276] 0709 13:01:52.709118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.769175 (+ 60057us) service_pool.cc:224] Handling call
0709 13:01:52.783850 (+ 14675us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.698196 (+914346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.545258 (+847062us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b540010d6d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:54.545690 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2092992128}
I20260709 13:01:54.550189  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.557583  6799 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.567494  2318 leader_election.cc:400] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12.
I20260709 13:01:54.567819  2318 leader_election.cc:403] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:01:54.568523  1561 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
W20260709 13:01:54.564407  6871 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.662s	user 0.010s	sys 0.005s
I20260709 13:01:54.572062  6799 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.573632  6910 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:54.573508  6824 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:54.574853  6799 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:54.569391  6930 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12.
W20260709 13:01:54.570190  6988 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:01:54.566695  6970 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:01:54.562029  6956 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.525408  6645 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.579811  1705 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:01:54.557662  6933 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:01:54.583868  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 334933us
I20260709 13:01:54.575133  6927 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:54.591163  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:54.592051  6756 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:54.592667  6804 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:54.592778  6756 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:54.599502  2344 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:01:54.562268  6542 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:54.603958  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.541199  6951 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.608371  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1315) took 1786 ms. Trace:
I20260709 13:01:54.570071  6925 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:54.607401  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 57307us
I20260709 13:01:54.558208  6738 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.612330  2349 rpcz_store.cc:276] 0709 13:01:52.822163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.822525 (+   362us) service_pool.cc:224] Handling call
0709 13:01:52.847533 (+ 25008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.321930 (+474397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.608069 (+1286139us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b540081d8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:54.608295 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576052992}
I20260709 13:01:54.616915  1708 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:01:54.605953  6947 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:54.623617  6731 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:54.624107  6567 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:54.565161  6326 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:01:54.623610  6947 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.626613  6738 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:54.627444  6947 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:54.625731  6949 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:54.593968  6756 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:54.629909  2349 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:54.630255  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:54.632102  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.607026  6799 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:54.636085  6970 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:54.637395  6940 raft_consensus.cc:2764] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:54.639612  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.648576  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:54.650957  1601 leader_election.cc:400] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:01:54.651466  1601 leader_election.cc:403] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:54.627431  6611 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:54.628182  6933 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:54.667775  6933 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.653283  6992 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:54.669916  6992 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.670437  6992 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:54.630997  6542 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.675064  6995 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:54.675905  6995 consensus_queue.cc:1243] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:01:54.676321  6931 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.678763  6645 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:54.658280  6799 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:54.657586  6949 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
W20260709 13:01:54.681967  6853 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.924s	user 0.015s	sys 0.000s
I20260709 13:01:54.636722  6970 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:54.682839  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1319) took 1414 ms. Trace:
I20260709 13:01:54.682565  6853 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.683315  2351 rpcz_store.cc:276] 0709 13:01:53.268580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.268788 (+   208us) service_pool.cc:224] Handling call
0709 13:01:54.682585 (+1413797us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b540002b2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:54.682765 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2101753216}
I20260709 13:01:54.683364  6970 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.691447  6951 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:54.695566  1459 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:54.702280  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:54.703547  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:54.703964  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.705693  1552 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:54.707053  1634 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:54.709069  1631 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:54.711330  1633 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:54.718796  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:54.719853  1634 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:54.724332  6756 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.694550  6618 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:54.728440  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:54.703516  6571 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:01:54.734409  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2919) took 2021 ms. Trace:
I20260709 13:01:54.736755  6611 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:54.759217  1627 rpcz_store.cc:276] 0709 13:01:52.713239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.788356 (+ 75117us) service_pool.cc:224] Handling call
0709 13:01:52.919296 (+130940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.957064 (+ 37768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.957122 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:52.957200 (+    78us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:52.957252 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.957376 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.734196 (+1776820us) spinlock_profiling.cc:243] Waited 96.6 ms on lock 0x7b5400138dd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:54.734336 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48062,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":6574,"spinlock_wait_cycles":222089600}
I20260709 13:01:54.764955  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:54.766571  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:54.708473  6738 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:01:54.737674  6951 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:54.738467  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:54.771392  1628 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:54.740888  6853 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:54.741039  6567 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:54.760782  1555 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:01:54.772441  1633 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:54.775346  1634 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:54.775760  1634 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:01:54.775817  2317 connection.cc:256] RPC call timeout handler was delayed by -1.1646687150000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2678s for a response
I20260709 13:01:54.779490  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1322) took 1486 ms. Trace:
I20260709 13:01:54.780016  2347 rpcz_store.cc:276] 0709 13:01:53.300288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.305865 (+  5577us) service_pool.cc:224] Handling call
0709 13:01:53.326811 (+ 20946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.608502 (+281691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.779208 (+1170706us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b5400540e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:54.779422 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646363520}
I20260709 13:01:54.777946  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.771152  6611 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:54.782822  6924 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:54.784513  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:54.773465  1555 pending_rounds.cc:85] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 7
I20260709 13:01:54.786954  1555 pending_rounds.cc:107] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:01:54.787247  1555 raft_consensus.cc:2889] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:54.782673  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 944) took 1079 ms. Trace:
I20260709 13:01:54.789062  1636 rpcz_store.cc:276] 0709 13:01:53.703284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.006687 (+303403us) service_pool.cc:224] Handling call
0709 13:01:54.006963 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.077876 (+ 70913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.782404 (+704528us) spinlock_profiling.cc:243] Waited 64.1 ms on lock 0x7b54001b6f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:54.782606 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147525888}
I20260709 13:01:54.758098  6618 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:54.783452  6611 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.798806  2317 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:54.799160  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 70859.4us
I20260709 13:01:54.799342  1630 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:54.810307  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:54.811467  1627 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.773382  6567 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:54.786307  6874 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:54.796448  1634 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.814601  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:54.775189  6951 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.818216  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2760) took 5199 ms. Trace:
I20260709 13:01:54.820613  2317 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.820538  6970 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:54.820791  6924 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:54.806064  6571 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:54.828135  1634 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:54.835022  1627 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:01:54.835675  1628 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:01:54.840977  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:54.842109  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:54.845357  6966 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.270s	user 0.004s	sys 0.007s
I20260709 13:01:54.822402  1707 rpcz_store.cc:276] 0709 13:01:49.609418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.736921 (+127503us) service_pool.cc:224] Handling call
0709 13:01:51.553690 (+1816769us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:51.554554 (+   864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.554589 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:51.555310 (+   721us) log.cc:844] Serialized 26 byte log entry
0709 13:01:51.832059 (+276749us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:51.832107 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.839531 (+  7424us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.600865 (+1761334us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:53.782132 (+181267us) raft_consensus.cc:1685] finished
0709 13:01:53.782156 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.808781 (+1026625us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54002e95d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:54.808935 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1154,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":153,"spinlock_wait_cycles":4099427328,"thread_start_us":276150,"threads_started":1,"wal-append.queue_time_us":600239,"wal-append.run_cpu_time_us":8638,"wal-append.run_wall_time_us":2631041}
I20260709 13:01:54.850044  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2833) took 1049 ms. Trace:
I20260709 13:01:54.850733  1552 rpcz_store.cc:276] 0709 13:01:53.800552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.494147 (+693595us) service_pool.cc:224] Handling call
0709 13:01:54.494645 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.706171 (+211526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.849657 (+143486us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b5400207954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:54.849945 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484747648}
I20260709 13:01:54.809612  6992 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:54.854031  6618 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:54.855387  6618 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.855664  6992 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:54.857678  6951 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:54.862346  7001 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:01:54.862551  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2828) took 1215 ms. Trace:
W20260709 13:01:54.862973  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 567601us
I20260709 13:01:54.864133  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:54.864244  6567 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.865638  6567 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.865687  6871 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:54.866518  6871 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:54.866896  6871 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.867234  6871 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.869918  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2820) took 2130 ms. Trace:
I20260709 13:01:54.870340  1553 rpcz_store.cc:276] 0709 13:01:52.739105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.351427 (+612322us) service_pool.cc:224] Handling call
0709 13:01:53.351758 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.051598 (+699840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.869650 (+818052us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b5400230d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:54.869847 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1607452928}
W20260709 13:01:54.871022  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 135794us
I20260709 13:01:54.874996  6921 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:54.861299  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.881727  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:54.882861  1636 raft_consensus.cc:1217] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[7.4-13.7]   Dedup: 7.4->[9.5-13.7]
I20260709 13:01:54.863037  1555 rpcz_store.cc:276] 0709 13:01:53.647616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.095070 (+447454us) service_pool.cc:224] Handling call
0709 13:01:54.095416 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.799719 (+704303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.862255 (+ 62536us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b5400311d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:54.862485 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1519912192}
I20260709 13:01:54.886376  1636 pending_rounds.cc:85] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 4
I20260709 13:01:54.886749  1636 pending_rounds.cc:107] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:01:54.887051  1636 raft_consensus.cc:2889] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:54.887702  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44788.4us
I20260709 13:01:54.890269  2318 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.891274  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:54.893275  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:54.836962  6969 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:54.897215  2347 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:01:54.834129  6571 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:54.895176  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:54.903172  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:54.904414  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:54.894165  6951 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.903079  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:54.909567  2318 leader_election.cc:400] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12.
I20260709 13:01:54.911947  1634 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:01:54.915449  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:54.930761  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.836774  6948 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:01:54.921988  2350 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:54.952220  1628 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:54.952783  1627 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:54.946787  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 786) took 2629 ms. Trace:
I20260709 13:01:54.893329  6853 raft_consensus.cc:2764] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:01:54.957851  1555 pending_rounds.cc:85] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 7
I20260709 13:01:54.956832  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:54.959152  2351 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:54.953835  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 46626.8us
I20260709 13:01:54.958359  1555 pending_rounds.cc:107] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:01:54.959174  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:54.961059  1555 raft_consensus.cc:2889] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:54.955972  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.899603  6606 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEARNER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:54.898159  6969 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:54.899622  6824 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:01:54.969740  1631 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:01:54.971412  1631 pending_rounds.cc:85] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 8
I20260709 13:01:54.971767  1631 pending_rounds.cc:107] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:01:54.972510  1631 raft_consensus.cc:2889] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:54.972796  1675 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:54.946275  6825 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:54.975267  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:54.913267  6927 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:54.977108  1628 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:54.981729  1560 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:54.946525  6936 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:54.946496  6992 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.954731  6697 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.991315  1559 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:54.984900  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:54.953540  6804 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.964897  6981 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.999358  1628 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:54.899367  6645 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:01:54.906538  6951 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:54.969070  6630 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:54.976663  6969 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:54.914355  6934 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.007650  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:55.013423  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 98745.6us
I20260709 13:01:55.013092  1559 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:01:55.013927  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:54.982914  6995 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.967129  6326 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:01:54.967502  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:01:55.058156  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.019855  6868 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:55.022403  1459 catalog_manager.cc:5697] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 6 to 8, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:55.023975  1634 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:54.957320  1557 rpcz_store.cc:276] 0709 13:01:52.284720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.550894 (+266174us) service_pool.cc:224] Handling call
0709 13:01:52.566857 (+ 15963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.585161 (+ 18304us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:52.619262 (+ 34101us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.764720 (+2145458us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b18002c62c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:01:54.913891 (+149171us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.913953 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:54.914007 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.914123 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.914255 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1651,"prepare.queue_time_us":34072,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":4132,"raft.queue_time_us":5937,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":9767,"spinlock_wait_cycles":47893248,"thread_start_us":147155,"threads_started":1}
W20260709 13:01:55.060386  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44962.2us
W20260709 13:01:55.061249  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 93861.2us
I20260709 13:01:55.061666  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:55.025180  6932 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:55.035934  2351 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:55.038471  6940 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:54.991616  6933 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:55.044452  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:55.050880  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2822) took 2182 ms. Trace:
W20260709 13:01:55.054064  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 62873.6us
I20260709 13:01:55.005952  6736 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:55.082530  1629 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:01:55.080163  1556 rpcz_store.cc:276] 0709 13:01:52.868262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.580083 (+711821us) service_pool.cc:224] Handling call
0709 13:01:53.581347 (+  1264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.877904 (+296557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.050478 (+1172574us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400207e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:55.050794 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553391872}
I20260709 13:01:55.085811  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.086735  1556 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:55.088300  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:54.996466  6967 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:01:55.080924  1525 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:55.009891  6697 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.094102  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 945) took 1390 ms. Trace:
I20260709 13:01:55.061023  6868 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:55.004495  6992 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.098953  1632 rpcz_store.cc:276] 0709 13:01:53.703586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.073848 (+370262us) service_pool.cc:224] Handling call
0709 13:01:54.090169 (+ 16321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.090527 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:54.137443 (+ 46916us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.963440 (+825997us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b1800235ea0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:01:55.093722 (+130282us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.093789 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:55.093837 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.093905 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.094042 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6857,"prepare.queue_time_us":34845,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":19874,"spinlock_wait_cycles":15950080,"thread_start_us":111045,"threads_started":1,"wal-append.queue_time_us":937135}
I20260709 13:01:55.103164  1525 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:55.103897  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:55.080519  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 59682.5us
I20260709 13:01:55.105692  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:55.105021  1557 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:55.040031  6981 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.110216  1601 leader_election.cc:400] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12.
I20260709 13:01:55.110677  1601 leader_election.cc:403] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:01:55.014549  6804 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.050052  6630 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.091935  6894 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:55.109794  6844 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:55.116559  1560 raft_consensus.cc:2410] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:01:55.115003  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 119452us
I20260709 13:01:55.076855  7015 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:55.119910  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.097025  6949 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:55.126343  1557 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:01:55.121683  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:55.105070  6921 raft_consensus.cc:2764] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:55.127965  1525 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:55.096973  6927 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.138882  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:55.130417  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:55.141090  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.142004  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:55.144853  1557 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:55.064915  6571 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.079591  6933 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.149986  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.128546  6874 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12.
I20260709 13:01:55.162113  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:55.125059  6844 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:55.063511  6995 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.174336  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:55.144284  6645 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:55.147416  6927 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:55.180434  6999 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.362s	user 0.004s	sys 0.000s
I20260709 13:01:55.182394  6927 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:55.182662  6630 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:55.183547  6630 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:01:55.179101  7021 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:01:55.185778  6697 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:55.186986  1459 catalog_manager.cc:5697] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 5 to 9, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 9 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:55.176446  6940 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:55.188735  6940 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:55.176868  6844 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:55.187002  6951 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:55.194231  7010 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:01:55.194015  6940 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:55.195430  7007 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:01:55.196229  1459 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:55.198603  6571 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:55.199050  6618 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:55.199827  6618 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:55.201764  6992 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:55.204234  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 801) took 1104 ms. Trace:
I20260709 13:01:55.204676  1561 rpcz_store.cc:276] 0709 13:01:54.099630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.585270 (+485640us) service_pool.cc:224] Handling call
0709 13:01:54.616606 (+ 31336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.869387 (+252781us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:54.869421 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.870037 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 13:01:54.984001 (+113964us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:54.984065 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.985139 (+  1074us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.204022 (+218883us) raft_consensus.cc:1685] finished
0709 13:01:55.204043 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.204173 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":241111,"raft.queue_time_us":242470,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"thread_start_us":113529,"threads_started":1,"wal-append.queue_time_us":179149}
I20260709 13:01:55.206318  6934 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:55.129555  7017 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:55.207140  1561 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:55.208459  1561 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:55.209950  7004 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:01:55.210762  2317 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:55.211074  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2821) took 2471 ms. Trace:
I20260709 13:01:55.211432  1558 rpcz_store.cc:276] 0709 13:01:52.739404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.580418 (+841014us) service_pool.cc:224] Handling call
0709 13:01:53.580968 (+   550us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 13:01:53.747259 (+166291us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:53.882890 (+135631us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.172267 (+289377us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.720205 (+547938us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b18002de740. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:54.859675 (+139470us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b18002de740. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:01:55.000134 (+140459us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.082152 (+ 82018us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:55.082996 (+   844us) log.cc:844] Serialized 52 byte log entry
0709 13:01:55.083221 (+   225us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:55.083515 (+   294us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.083719 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.095699 (+ 11980us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.210857 (+115158us) raft_consensus.cc:1685] finished
0709 13:01:55.210882 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.211016 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":194989,"prepare.queue_time_us":521469,"prepare.run_cpu_time_us":164,"prepare.run_wall_time_us":166,"raft.queue_time_us":804871,"raft.run_cpu_time_us":630,"raft.run_wall_time_us":36126,"spinlock_wait_cycles":22056704,"thread_start_us":586313,"threads_started":4,"wal-append.queue_time_us":318128}
W20260709 13:01:55.214237  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.204294  1559 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:55.216827  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:55.217355  1561 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:55.217615  1633 raft_consensus.cc:2376] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11. Re-sending same reply.
I20260709 13:01:55.217885  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.218760  1635 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:55.209134  6933 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:55.221217  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.227533  1634 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:55.228507  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.214453  1459 catalog_manager.cc:5697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 6 to 9, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:55.229276  1634 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:55.154808  6956 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:55.226473  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:01:55.231875  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 57686.5us
I20260709 13:01:55.222636  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:55.235129  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.235893  1559 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:01:55.236734  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:55.238432  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:55.239442  1633 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:55.233386  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.241712  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:55.246595  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 104727us
I20260709 13:01:55.256965  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.272204  1635 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:01:55.213402  6936 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:55.274482  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:55.274463  1526 leader_election.cc:400] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12.
I20260709 13:01:55.274365  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:55.275566  1526 leader_election.cc:403] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:55.276087  2351 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:01:55.275475  2353 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:55.278232  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.278582  1633 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:01:55.279636  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:55.281289  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:55.226400  6970 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:55.276103  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:55.282770  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:55.283222  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.230711  6799 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:55.293604  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.295044  1560 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:55.226168  6645 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:55.241671  1459 catalog_manager.cc:5697] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 3 to 7, pending config changed. New cstate: current_term: 7 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:01:55.299612  1633 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:55.231233  7017 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:55.225960  6853 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:55.285821  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.316882  1632 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:55.286586  1561 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:01:55.280534  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 199459us
I20260709 13:01:55.249120  6925 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.244383  6940 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:55.280580  1559 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:55.302347  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.321949  1628 raft_consensus.cc:2368] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14.
I20260709 13:01:55.284855  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:55.322296  1526 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:55.304024  1631 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:55.305289  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:55.324164  1560 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:55.312687  2347 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:55.313437  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:55.281364  6936 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.243650  6948 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:01:55.174576  6868 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.329344  1558 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:55.327996  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 262991us
I20260709 13:01:55.320676  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:55.330974  1633 raft_consensus.cc:2368] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12.
I20260709 13:01:55.297959  6979 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12.
I20260709 13:01:55.349892  1525 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:55.350876  7026 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:55.351701  6979 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:55.352393  6979 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:55.335453  6697 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:55.353850  6697 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:55.354413  6697 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:55.354048  6645 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.300411  6611 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.358093  6981 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:55.334733  6868 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:55.342350  1455 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 273) took 28363 ms (client timeout 15000 ms). Trace:
I20260709 13:01:55.364445  6910 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:55.343626  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.313632  6736 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:55.349965  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.353152  7026 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:01:55.351027  7017 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.341986  6804 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:55.336385  6948 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:55.359671  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:55.360250  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:55.332821  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.364269  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:55.365159  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:55.379603  1632 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:01:55.365558  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:55.382645  1561 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:55.383258  1631 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:55.343449  6825 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:55.385366  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:55.396263  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.405162  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:55.408349  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 75810.4us
I20260709 13:01:55.408463  1558 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:01:55.392050  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 59053.5us
I20260709 13:01:55.397976  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:55.411958  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:55.386610  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.413774  1628 raft_consensus.cc:2410] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:55.404579  6853 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:55.418849  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:55.418939  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:55.421248  1629 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:55.428133  1560 raft_consensus.cc:2410] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:01:55.386557  6921 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:55.428718  6921 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:01:55.424463  6540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.004s	sys 0.008s
I20260709 13:01:55.429971  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:55.430202  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:55.432623  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 293817us
I20260709 13:01:55.433827  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:55.434629  1561 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:55.379340  6948 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:55.438186  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.438865  1629 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:55.439726  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.440564  1629 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:55.436597  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:55.365259  6910 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:01:55.431510  6871 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.004s
I20260709 13:01:55.444626  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.468957  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.469719  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:55.470187  1559 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:55.471359  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:55.472398  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 89466.5us
I20260709 13:01:55.478613  1526 leader_election.cc:400] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:01:55.478972  1526 leader_election.cc:403] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:55.429215  6921 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:55.332767  6925 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.481910  7038 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:55.363123  6844 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.486882  6981 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:55.426455  6853 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:55.487504  6995 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:55.489861  6932 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:01:55.490127  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 71367.1us
W20260709 13:01:55.490427  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 51523.3us
I20260709 13:01:55.529013  6630 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:55.492870  6994 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:01:55.531504  6994 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:55.431841  6804 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:55.534149  6804 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:55.534824  6804 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:55.541677  6659 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:55.543614  6659 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:01:55.549847  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.557077  7026 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:01:55.558595  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 91994.2us
I20260709 13:01:55.506289  1628 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
W20260709 13:01:55.433652  6540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.005s	sys 0.008s
I20260709 13:01:55.509701  1552 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:01:55.505009  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:55.565001  1629 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:55.516132  1527 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:55.519255  6825 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:01:55.381873  6894 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 13:01:55.567204  6825 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:55.568221  6825 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:55.568421  6894 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:55.569104  6894 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:55.520327  6981 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:55.580180  6645 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:55.581336  6825 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:55.581199  6981 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.581796  6825 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:55.582757  6981 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:55.583415  6611 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:55.582814  6825 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:55.582419  6697 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.587505  2352 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:55.589398  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:55.407044  7018 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
W20260709 13:01:55.362593  1455 rpcz_store.cc:269] 0709 13:01:26.978554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.978947 (+   393us) service_pool.cc:224] Handling call
0709 13:01:30.173738 (+3194791us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400179b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27644090 00007fda276e2f3a 00007fda2d4991cc 00007fda2d59737e 00007fda2ba582a4 00007fda2ba58232 00007fda2ba58161 00007fda2ba580dc 00007fda2ba57392 00007fda25a1548c 00007fda25a148c6
0709 13:01:55.341963 (+25168225us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:37.639906 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:37.670227 (+ 30321us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:37.670276 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:01:37.670298 (+    22us) tablet.cc:662] Decoding operations
0709 13:01:37.672274 (+  1976us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:37.672343 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:37.672378 (+    35us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:01:38.041698 (+369320us) tablet.cc:701] Row locks acquired
0709 13:01:38.047463 (+  5765us) write_op.cc:260] PREPARE: finished
0709 13:01:38.056603 (+  9140us) write_op.cc:270] Start()
0709 13:01:38.077374 (+ 20771us) write_op.cc:276] Timestamp: P: 1783602098053822 usec, L: 0
0709 13:01:38.082444 (+  5070us) op_driver.cc:348] REPLICATION: starting
0709 13:01:38.154397 (+ 71953us) log.cc:844] Serialized 8237 byte log entry
0709 13:01:39.540553 (+1386156us) op_driver.cc:464] REPLICATION: finished
0709 13:01:40.243392 (+702839us) write_op.cc:301] APPLY: starting
0709 13:01:40.250970 (+  7578us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:41.104866 (+853896us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:41.141970 (+ 37104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:41.899740 (+757770us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:41.900228 (+   488us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:01:41.900333 (+   105us) write_op.cc:312] APPLY: finished
0709 13:01:41.901023 (+   690us) log.cc:844] Serialized 180 byte log entry
0709 13:01:42.015294 (+114271us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:42.149851 (+134557us) write_op.cc:454] Released schema lock
0709 13:01:42.149946 (+    95us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:42.150653 (+   707us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:42.150684 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":223579,"reported_tablets":47,"spinlock_wait_cycles":3583400704,"child_traces":[["op",{"apply.queue_time_us":619903,"apply.run_cpu_time_us":108311,"apply.run_wall_time_us":1931604,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87727,"num_ops":20,"prepare.queue_time_us":594478,"prepare.run_cpu_time_us":54033,"prepare.run_wall_time_us":795911,"raft.queue_time_us":365702,"raft.run_cpu_time_us":28968,"raft.run_wall_time_us":188444,"replication_time_us":1438420,"thread_start_us":1069428,"threads_started":5,"wal-append.queue_time_us":406128,"wal-append.run_cpu_time_us":45955,"wal-append.run_wall_time_us":2998504}]]}
I20260709 13:01:55.590334  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:55.594569  1708 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
W20260709 13:01:55.527598  7002 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.817s	user 0.003s	sys 0.004s
I20260709 13:01:55.597821  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:55.598264  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:55.353369  6979 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:55.606072  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 127584us
I20260709 13:01:55.611202  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:55.612116  1708 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
W20260709 13:01:55.643070  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54766.6us
I20260709 13:01:55.639791  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:55.655560  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:55.644220  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:55.658749  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 235263us
I20260709 13:01:55.659899  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:55.666682  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:55.667694  1710 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:55.668467  6969 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.591s	user 0.011s	sys 0.000s
I20260709 13:01:55.513060  6871 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.682842  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.608990  6932 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:55.696278  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:55.700628  1707 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:55.520787  6736 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.708420  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:55.704260  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:55.729178  1561 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:55.737628  1557 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:55.669126  6969 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.704282  7032 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:55.749944  7032 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:55.750697  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:55.751319  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:55.751995  6933 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:55.755383  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:55.749275  7024 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:55.757115  7024 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:55.758332  7024 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:55.751078  7032 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.768685  7025 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:01:55.770865  7041 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:01:55.770704  7025 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:01:55.727710  7036 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:01:55.782799  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:55.785279  2349 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:55.780236  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:55.802119  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.678504  6995 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:01:55.810556  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.817356  1561 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:55.818432  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:55.824993  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:55.825614  1706 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:55.826609  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:55.828189  1707 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:01:55.828586  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:55.804446  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 288465us
I20260709 13:01:55.835673  1561 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 13:01:55.833029  7034 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:01:55.836629  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:55.840046  1703 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:55.675249  6910 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.608402  6948 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.832891  6931 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:55.833351  6567 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:55.834165  6824 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:55.836826  6933 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:55.857034  7043 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:55.857792  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:55.860991  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 201212us
W20260709 13:01:55.891224  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.870193  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:55.873994  1561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13debc393e8a4121883dda403cb4badf"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:01:55.885162  1710 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:55.895550  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:55.898910  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:55.908895  1703 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:55.912555  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:55.913574  1561 raft_consensus.cc:606] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:55.880728  6567 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:55.880235  6931 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:55.880506  6824 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:55.915885  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64049.8us
I20260709 13:01:55.891673  7043 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:55.911602  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 45703.9us
W20260709 13:01:55.920198  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.920397  1527 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:55.854144  6932 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.927851  1561 raft_consensus.cc:614] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:55.927636  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:55.904057  7044 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.891546  6925 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:55.910535  6540 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:55.914425  7016 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:55.933055  7044 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.934093  7016 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:55.936093  7044 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:55.937347  6540 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.939898  6540 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:55.917413  6567 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:55.915073  6910 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:55.941148  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:55.888358  6756 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:01:55.918601  6824 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:55.944142  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:55.948632  1709 raft_consensus.cc:2410] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:01:55.936162  1527 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:55.894594  6933 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:55.926667  6927 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.943485  1553 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:55.926390  7043 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:55.944409  7039 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:01:55.956019  2318 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:55.961519  6969 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:55.963826  6933 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:55.965579  7031 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:01:55.965915  6844 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:55.967003  7040 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:01:55.972853  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2841) took 1013 ms. Trace:
I20260709 13:01:55.973693  6931 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.974613  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:55.975021  6931 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.975659  1701 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:55.976696  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:55.941636  7047 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:55.959844  6927 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:55.978173  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.931582  6871 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:55.979669  6804 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:55.981118  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:55.982141  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 802) took 1009 ms. Trace:
I20260709 13:01:55.982554  1552 rpcz_store.cc:276] 0709 13:01:54.973046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.993876 (+ 20830us) service_pool.cc:224] Handling call
0709 13:01:55.981948 (+988072us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400312254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:55.982073 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890594176}
I20260709 13:01:55.987130  6804 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:55.982487  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:55.980069  6927 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:55.990345  6936 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.648s	user 0.012s	sys 0.000s
I20260709 13:01:55.952611  6853 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:55.993503  6936 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:56.004853  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.973387  1557 rpcz_store.cc:276] 0709 13:01:54.959934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.161788 (+201854us) service_pool.cc:224] Handling call
0709 13:01:55.972602 (+810814us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54001c0054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:55.972764 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297606528}
I20260709 13:01:55.991401  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:56.035197  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:56.036098  2353 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
W20260709 13:01:56.037041  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 81107.3us
I20260709 13:01:56.042130  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:56.043140  1557 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3897029c4b5f443296641c39af287e98"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:01:56.043772  1557 raft_consensus.cc:606] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:56.055588  2349 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:56.059547  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.062742  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:56.063753  1553 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
W20260709 13:01:56.064246  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51055.8us
I20260709 13:01:56.064846  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:56.063064  1561 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:56.072178  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:56.072897  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:56.074405  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:56.024590  6697 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:56.081774  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:56.072956  6970 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:56.078492  6940 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.746s	user 0.010s	sys 0.003s
I20260709 13:01:56.095288  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:56.086058  6981 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.503s	user 0.014s	sys 0.000s
I20260709 13:01:56.071996  7016 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.085623  6927 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:56.109640  1561 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:01:56.110440  1704 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:56.113143  2317 leader_election.cc:400] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14.
I20260709 13:01:56.105160  1709 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:01:56.119755  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:56.121833  1701 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:56.094017  6970 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.081396  6924 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:01:56.081418  6326 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:01:56.138083  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:56.140295  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59493.9us
I20260709 13:01:56.146127  1675 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:56.147958  6931 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:56.148561  6931 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.149060  6931 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:56.134632  6970 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:56.151875  6326 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:56.154599  7053 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:56.155200  7053 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:56.155627  7053 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:56.155771  6825 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:56.156323  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:56.157147  2351 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:01:56.161957  6868 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.780s	user 0.014s	sys 0.000s
I20260709 13:01:56.163940  6738 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:56.166252  1675 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:56.167894  6981 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:56.170137  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:56.171142  1707 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:56.172423  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:56.173959  1710 raft_consensus.cc:2368] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:01:56.176445  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:56.176894  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.177373  1701 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:56.179342  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:56.185745  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:56.188037  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:56.189541  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:56.208705  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 90629.6us
I20260709 13:01:56.211790  2317 leader_election.cc:400] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12.
I20260709 13:01:56.213644  2317 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:56.198907  6921 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.593s	user 0.013s	sys 0.000s
I20260709 13:01:56.193159  6853 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:56.218958  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:56.220022  1710 raft_consensus.cc:2410] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:56.125171  7016 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:56.205016  6645 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:01:56.222822  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:56.256636  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:56.179013  6571 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:01:56.289923  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:56.330281  1561 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:01:56.334858  7057 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:01:56.287339  7071 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:56.339257  7060 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
W20260709 13:01:56.341955  7059 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:01:56.344364  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:56.223728  6853 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.327991  6924 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:56.336848  6824 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.218936  6921 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.367853  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:56.373688  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 83880.9us
I20260709 13:01:56.374012  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2933) took 1610 ms. Trace:
I20260709 13:01:56.382696  2317 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:56.385074  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:56.378252  2353 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEARNER]: Advancing to term 14
I20260709 13:01:56.355269  7071 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:56.382432  1630 rpcz_store.cc:276] 0709 13:01:54.763721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.819058 (+ 55337us) service_pool.cc:224] Handling call
0709 13:01:54.819447 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.819892 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:55.027032 (+207140us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b1800351460. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda2b0d23c7 00007fda276e49a7 00007fda276f40bf
0709 13:01:55.027211 (+   179us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.122831 (+ 95620us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:55.147783 (+ 24952us) log.cc:844] Serialized 26 byte log entry
0709 13:01:55.147974 (+   191us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:55.148246 (+   272us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.150682 (+  2436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.150780 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.369035 (+218255us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b1800351460. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:56.340934 (+971899us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b1800351460. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:01:56.373713 (+ 32779us) raft_consensus.cc:1685] finished
0709 13:01:56.373750 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.373939 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34142,"prepare.queue_time_us":175869,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":10355,"raft.queue_time_us":28866,"raft.run_cpu_time_us":513,"raft.run_wall_time_us":32212,"spinlock_wait_cycles":201510400,"thread_start_us":142184,"threads_started":2,"wal-append.queue_time_us":151913}
I20260709 13:01:56.414155  2350 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 13:01:56.421465  1556 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:01:56.343760  6970 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:56.439879  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.388551  6756 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:56.451524  6921 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:56.454651  7064 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:56.455230  7062 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:56.391084  7071 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:56.362541  6567 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.458459  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:56.457052  7057 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:01:56.471262  1555 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:56.472429  1555 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:56.461892  6922 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:56.433602  6931 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:56.483078  6853 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:56.484882  2352 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1332) took 1567 ms (client timeout 1592 ms). Trace:
I20260709 13:01:56.466712  7071 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:56.488725  2352 rpcz_store.cc:269] 0709 13:01:54.917437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.958328 (+ 40891us) service_pool.cc:224] Handling call
0709 13:01:56.484635 (+1526307us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b540048ab54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:56.484799 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":841484544}
I20260709 13:01:56.494801  6894 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.496361  6894 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:56.502771  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:56.503958  1709 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:56.504307  6824 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:56.505009  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:56.505843  7016 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:56.506271  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:56.507803  1709 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:01:56.510891  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:56.512056  1710 raft_consensus.cc:2410] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:56.484063  6922 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:56.518975  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.522253  7069 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:01:56.522922  6970 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:56.529829  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2705) took 1305 ms. Trace:
W20260709 13:01:56.530270  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 1.11225e+06us
I20260709 13:01:56.532238  1602 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:56.530347  1634 rpcz_store.cc:276] 0709 13:01:55.224075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.249025 (+ 24950us) service_pool.cc:224] Handling call
0709 13:01:55.250065 (+  1040us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:55.250417 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.250451 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:55.251162 (+   711us) log.cc:844] Serialized 26 byte log entry
0709 13:01:55.428594 (+177432us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:55.428986 (+   392us) log.cc:844] Serialized 14 byte log entry
0709 13:01:56.529408 (+1100422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.529519 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.529563 (+    44us) raft_consensus.cc:1685] finished
0709 13:01:56.529577 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.529756 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1100091,"prepare.queue_time_us":248,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":1101409,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":108542,"threads_started":1,"wal-append.queue_time_us":355899}
I20260709 13:01:56.530856  7027 raft_consensus.cc:2764] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:01:56.526340  6910 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.579s	user 0.005s	sys 0.010s
I20260709 13:01:56.546844  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:56.547554  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:56.549012  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:56.549732  1627 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:56.550315  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:56.550968  1628 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:56.552495  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:56.558495  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:56.562801  6567 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:56.564097  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:56.565562  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:56.565750  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:56.566345  1627 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:56.566352  1629 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:56.572609  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2216) took 1212 ms. Trace:
I20260709 13:01:56.573324  1556 rpcz_store.cc:276] 0709 13:01:55.359759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.493876 (+134117us) service_pool.cc:224] Handling call
0709 13:01:56.572384 (+1078508us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b54002521d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:56.572528 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1659206400}
I20260709 13:01:56.583433  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:56.614936  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.627848  1628 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:56.542308  6874 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.669822  1630 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:56.673027  1628 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:56.542146  6799 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:56.582573  6951 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:56.697306  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:56.698590  1627 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:56.582846  6611 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:56.554083  6956 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.135s
I20260709 13:01:56.663182  6934 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:56.662590  7027 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:56.716822  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:56.682423  6949 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.706243  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:56.662990  6659 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:56.722675  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 144905us
I20260709 13:01:56.724326  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:56.696214  6951 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:56.731029  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 189210us
I20260709 13:01:56.734347  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:56.734969  2352 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:56.735330  1629 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:56.730202  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:56.732460  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:56.742169  1628 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:01:56.726876  7053 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.005s
W20260709 13:01:56.762858  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.762568  1525 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:56.781759  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:56.791626  2353 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:01:56.792266  2353 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 9
I20260709 13:01:56.793171  2353 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:01:56.794402  2353 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:56.690927  6874 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.731667  6825 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:56.849900  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:56.736953  7016 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:56.874881  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 134497us
I20260709 13:01:56.739491  6951 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.716329  7015 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:01:56.893796  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:56.781745  6853 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:56.914120  2352 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:01:56.782694  7043 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.757s	user 0.012s	sys 0.004s
I20260709 13:01:56.916500  1632 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:56.835042  7053 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.904395  6994 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:56.920234  6994 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:56.701083  6799 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.921355  1634 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:56.919369  2349 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:56.821517  7054 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:56.821871  6871 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:56.733316  6949 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.783344  6567 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.179s
I20260709 13:01:56.732841  6934 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.734692  6611 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.732617  6659 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:56.891450  7086 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.926975  7083 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.927323  6921 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:56.943584  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 51384.7us
I20260709 13:01:56.913915  6853 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.951423  1634 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:56.936179  6956 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:57.007085  6956 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.007718  6956 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:56.933399  6571 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.965922  6736 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.009835  6571 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.010349  6736 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.012279  6571 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:56.935240  6871 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:56.967581  7077 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.398s	user 0.000s	sys 0.001s
I20260709 13:01:56.964668  6921 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.014678  6871 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.014993  6921 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:56.964841  6853 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.993014  6949 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:56.972057  7062 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.022002  7062 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.023396  6956 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.025104  7062 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:56.978273  7081 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
W20260709 13:01:56.984874  7082 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:01:56.985970  7075 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.448s	user 0.000s	sys 0.001s
I20260709 13:01:56.985831  6934 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:56.986747  6922 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:56.938953  6931 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:56.988945  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 963) took 1641 ms. Trace:
I20260709 13:01:57.033294  1633 rpcz_store.cc:276] 0709 13:01:55.347367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.505443 (+158076us) service_pool.cc:224] Handling call
0709 13:01:55.591108 (+ 85665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.531471 (+940363us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:56.531506 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:56.537552 (+  6046us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.680631 (+143079us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:56.680706 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.680848 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.988597 (+307749us) raft_consensus.cc:1685] finished
0709 13:01:56.988656 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.988859 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":937683,"raft.queue_time_us":938774,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":2795,"thread_start_us":120013,"threads_started":1,"wal-append.queue_time_us":374044}
I20260709 13:01:57.035310  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:57.035545  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:57.036177  1627 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:01:56.989987  6659 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:56.936903  7054 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:57.039865  7054 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:56.930189  6932 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:01:57.036818  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:57.046734  1556 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:56.991343  6874 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:56.995661  6799 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:56.928679  7043 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.961776  7083 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:56.932785  7088 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:56.959446  7086 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.005860  7094 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:57.012610  6736 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:57.025408  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.974709  6611 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.031123  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2810) took 1151 ms. Trace:
I20260709 13:01:57.107380  1709 rpcz_store.cc:276] 0709 13:01:55.879951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.508549 (+628598us) service_pool.cc:224] Handling call
0709 13:01:56.508904 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.509148 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.509173 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:56.510236 (+  1063us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.540941 (+ 30705us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:56.541128 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.541239 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.030828 (+489589us) raft_consensus.cc:1685] finished
0709 13:01:57.030861 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.031042 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":13625,"threads_started":1,"wal-append.queue_time_us":205430}
I20260709 13:01:57.050729  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:57.052029  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:57.056761  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:57.058804  1552 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:01:57.089056  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 195357us
I20260709 13:01:57.103992  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:57.112368  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.105429  1557 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:57.106412  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2937) took 2336 ms. Trace:
I20260709 13:01:57.121174  1558 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:57.123333  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.125038  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:57.124017  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:57.126286  1557 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:57.128183  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 76234.2us
I20260709 13:01:57.058427  6844 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:57.135238  1553 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:57.136080  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:57.137526  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.144482  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:57.132726  6611 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.105098  6982 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:57.169179  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:57.171041  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:57.172686  1674 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.174187  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:01:57.127008  7093 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.130097  6874 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:01:57.166276  6934 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:57.188318  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:57.188442  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:57.139264  7016 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:57.213343  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:57.214524  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.214706  1552 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
W20260709 13:01:57.216866  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.201864  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:57.202728  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2163) took 1358 ms. Trace:
I20260709 13:01:57.219486  1704 rpcz_store.cc:276] 0709 13:01:55.824251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.828133 (+  3882us) service_pool.cc:224] Handling call
0709 13:01:55.837236 (+  9103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.116265 (+279029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.182801 (+1066536us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54001e9e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:57.182983 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":616921472}
W20260709 13:01:57.186168  6927 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.048s	user 0.011s	sys 0.015s
I20260709 13:01:57.224092  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.222323  1709 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:57.178408  7081 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:01:57.244132  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.182868  6932 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.243954  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:57.253846  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.245090  1702 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:01:57.189448  6931 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:57.210889  7099 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.166848  6844 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.246424  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:57.224393  6927 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.091s	user 0.012s	sys 0.016s
I20260709 13:01:57.171074  6736 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.254292  2352 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:57.219750  6934 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:57.262632  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 76835.9us
I20260709 13:01:57.180117  7088 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.219952  7016 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:57.263554  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.183472  6659 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.264250  1557 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:57.189625  6853 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.200393  7063 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:01:57.202195  6949 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:01:57.205355  6611 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:01:57.256273  6931 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:57.242975  6567 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:01:57.272078  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:57.261054  6927 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:57.249918  6874 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:01:57.238055  7062 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:01:57.265024  6934 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:57.267688  7016 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:57.203071  7093 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.274552  6956 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:57.265924  7099 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:57.310274  6927 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.308411  7102 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:01:57.310081  6853 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.310159  7109 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.316315  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2854) took 1229 ms. Trace:
I20260709 13:01:57.326930  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2853) took 1247 ms. Trace:
W20260709 13:01:57.320992  7071 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.805s	user 0.010s	sys 0.003s
I20260709 13:01:57.281265  7054 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.328858  6949 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.332245  1627 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:57.292785  6659 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.339378  1628 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:57.283272  6931 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.345607  1709 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:01:57.354719  1560 rpcz_store.cc:276] 0709 13:01:56.087363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.107041 (+ 19678us) service_pool.cc:224] Handling call
0709 13:01:56.127028 (+ 19987us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.127497 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.127535 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:57.039607 (+912072us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b1800452e60. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a
0709 13:01:57.040189 (+   582us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b2c00000658. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:01:57.040927 (+   738us) log.cc:844] Serialized 26 byte log entry
0709 13:01:57.181949 (+141022us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:57.185717 (+  3768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.185834 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.315936 (+130102us) raft_consensus.cc:1685] finished
0709 13:01:57.315970 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.316210 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":520,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":5263,"spinlock_wait_cycles":7866624,"thread_start_us":140472,"threads_started":1,"wal-append.queue_time_us":219324}
I20260709 13:01:57.354785  1559 rpcz_store.cc:276] 0709 13:01:56.079632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.085953 (+  6321us) service_pool.cc:224] Handling call
0709 13:01:56.086403 (+   450us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:56.102616 (+ 16213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.102647 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:57.031327 (+928680us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b2c00000658. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:01:57.060885 (+ 29558us) log.cc:844] Serialized 52 byte log entry
0709 13:01:57.262985 (+202100us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:57.263025 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.263130 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.326573 (+ 63443us) raft_consensus.cc:1685] finished
0709 13:01:57.326597 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.326845 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15694,"raft.queue_time_us":17757,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":19396,"spinlock_wait_cycles":24863104,"thread_start_us":99865,"threads_started":1,"wal-append.queue_time_us":201054}
I20260709 13:01:57.364123  7053 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.309707  7086 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.367671  7029 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.368538  7043 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.308411  7101 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.369738  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:57.399015  7091 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.399679  6868 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:57.287865  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.401685  7046 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:01:57.402091  6868 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:57.402496  7086 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:57.402688  6868 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:57.402298  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 114534us
I20260709 13:01:57.403358  6868 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.404222  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.404495  7064 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:57.404942  7064 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:57.399031  7101 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.406744  7086 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.384886  6921 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.407546  6921 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:57.370404  7099 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.411703  7095 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:57.412837  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.377321  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.378975  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:57.378281  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:57.113620  1636 rpcz_store.cc:276] 0709 13:01:54.770004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.828149 (+ 58145us) service_pool.cc:224] Handling call
0709 13:01:54.852807 (+ 24658us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:01:54.901811 (+ 49004us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:55.079948 (+178137us) log.cc:844] Serialized 14 byte log entry
0709 13:01:56.528474 (+1448526us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:56.529314 (+   840us) log.cc:844] Serialized 78 byte log entry
0709 13:01:56.569918 (+ 40604us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:56.926507 (+356589us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.087544 (+161037us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.103729 (+ 16185us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.104544 (+   815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.104635 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.104679 (+    44us) raft_consensus.cc:1685] finished
0709 13:01:57.104695 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.106343 (+  1648us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1323663,"prepare.queue_time_us":528623,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":141,"raft.queue_time_us":1418896,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":10949,"thread_start_us":825895,"threads_started":5,"wal-append.queue_time_us":440307,"wal-append.run_cpu_time_us":1469,"wal-append.run_wall_time_us":1011748}
I20260709 13:01:57.386066  1556 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:57.364293  7093 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.310493  7083 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.336978  6934 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.399650  7091 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.414685  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:57.444553  1557 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:57.416359  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.282150  6951 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.427929  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.430351  1702 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:57.375397  7071 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.431849  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 920) took 1275 ms. Trace:
I20260709 13:01:57.432425  1703 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:57.390256  6659 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:57.437986  7098 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:01:57.438426  7111 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:01:57.439131  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:57.440438  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 61225.4us
I20260709 13:01:57.309741  6844 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:57.446264  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 76602.8us
I20260709 13:01:57.458820  1527 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:57.418475  7099 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:57.449133  2349 rpcz_store.cc:276] 0709 13:01:56.156221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.157906 (+  1685us) service_pool.cc:224] Handling call
0709 13:01:56.158288 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.922911 (+764623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.431569 (+508658us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54004ed754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:57.431781 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749742848}
I20260709 13:01:57.454511  1636 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:01:57.454397  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.460126  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.465991  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.466512  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:57.460660  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.449143  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:57.461382  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:57.461630  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:57.469010  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 91796.5us
I20260709 13:01:57.469376  1558 raft_consensus.cc:2376] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17. Re-sending same reply.
I20260709 13:01:57.470669  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:57.462066  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:57.463667  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.382455  7054 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.465142  1703 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:01:57.461524  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.466583  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:57.473184  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:57.482062  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:57.472009  1556 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:57.478796  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.478108  1635 raft_consensus.cc:2368] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15.
I20260709 13:01:57.488106  1703 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:57.478592  2318 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:57.478592  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.402249  7046 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.486326  2347 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:57.456918  7083 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:01:57.491616  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.492815  1708 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:57.493669  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.497308  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:57.502404  1702 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:57.504189  1557 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:01:57.412474  7095 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.412634  7029 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.381347  7109 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.478711  6925 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.471259  7117 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:57.488148  6956 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.499281  7099 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:57.495080  7038 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:57.517604  1560 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:01:57.520239  1526 leader_election.cc:400] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:01:57.500113  7063 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:57.524083  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.528462  1526 leader_election.cc:403] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:01:57.525343  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:57.531734  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.533033  7118 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.533797  7118 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.537012  7118 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.540194  7095 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.542306  6659 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:57.542532  6951 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.543893  7119 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 13:01:57.544391  7119 raft_consensus.cc:1068] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:01:57.545535  7116 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:01:57.546945  7046 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.547843  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 905) took 1392 ms. Trace:
I20260709 13:01:57.548250  1710 rpcz_store.cc:276] 0709 13:01:56.155419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.019938 (+864519us) service_pool.cc:224] Handling call
0709 13:01:57.020269 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:57.020649 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.020682 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:57.021400 (+   718us) log.cc:844] Serialized 52 byte log entry
0709 13:01:57.356951 (+335551us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:57.357717 (+   766us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.358326 (+   609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.358422 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.547561 (+189139us) raft_consensus.cc:1685] finished
0709 13:01:57.547586 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.547762 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":184,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":5850,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":225,"thread_start_us":253296,"threads_started":1,"wal-append.queue_time_us":371980}
I20260709 13:01:57.550556  7071 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.518002  6921 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.551479  7117 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:57.552775  7117 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.555001  7062 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:01:57.557374  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 89010.2us
I20260709 13:01:57.529098  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:57.559031  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.560999  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.561972  1708 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:57.518402  7101 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.517436  6934 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.569115  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.571874  1710 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:01:57.569144  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 962) took 2222 ms. Trace:
I20260709 13:01:57.574791  1710 pending_rounds.cc:85] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 5
I20260709 13:01:57.575227  1710 pending_rounds.cc:107] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:01:57.575685  1710 raft_consensus.cc:2889] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:57.578862  7118 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.579680  7118 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:57.577872  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 96543.5us
I20260709 13:01:57.574854  1631 rpcz_store.cc:276] 0709 13:01:55.347110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.499599 (+152489us) service_pool.cc:224] Handling call
0709 13:01:55.500059 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.540286 (+1040227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.540318 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:56.540845 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.944454 (+403609us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b18000491e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:01:57.031287 (+ 86833us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:57.031333 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.032179 (+   846us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.558566 (+526387us) raft_consensus.cc:1685] finished
0709 13:01:57.558599 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.569058 (+ 10459us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1039763,"raft.queue_time_us":1056911,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":88441,"spinlock_wait_cycles":37902720,"thread_start_us":86624,"threads_started":1,"wal-append.queue_time_us":614066}
I20260709 13:01:57.564728  7099 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:57.568807  7063 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:57.520851  7120 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 13:01:57.613428  7118 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.614101  7091 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:57.615437  7120 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:57.619663  7120 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:57.621030  7120 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.622781  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:57.623466  7120 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:57.624825  6868 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.615916  7063 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.627197  6868 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:57.629776  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 98155.7us
I20260709 13:01:57.617348  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.623783  1710 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:01:57.616941  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.602316  7099 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:57.567104  7119 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:01:57.621323  7038 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.641566  1556 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:01:57.648233  1556 raft_consensus.cc:606] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:57.650524  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.654759  6934 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:57.655653  6934 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.685842  6925 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.688287  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.715360  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2856) took 1602 ms. Trace:
I20260709 13:01:57.715852  1561 rpcz_store.cc:276] 0709 13:01:56.112599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.130090 (+ 17491us) service_pool.cc:224] Handling call
0709 13:01:56.130633 (+   543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.330895 (+200262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.713417 (+1382522us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54003c0fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:57.715273 (+  1856us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456687872}
I20260709 13:01:57.708678  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1337) took 1697 ms. Trace:
I20260709 13:01:57.717829  2350 rpcz_store.cc:276] 0709 13:01:56.011017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.012328 (+  1311us) service_pool.cc:224] Handling call
0709 13:01:56.054744 (+ 42416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.418347 (+363603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.708381 (+1290034us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54006446d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:57.708605 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825040256}
I20260709 13:01:57.725142  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:57.639739  6932 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.746210  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:57.747593  6934 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.642576  7099 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:57.757361  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.756994  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.757686  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.757154  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:57.639534  6921 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:57.763932  6921 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.764299  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.764863  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.643024  7062 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:57.766991  1702 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:57.767251  6956 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.766871  7062 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.768080  6956 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:57.768685  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.769390  1702 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:01:57.768867  6956 consensus_queue.cc:1243] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:01:57.770884  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.771140  6921 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.771767  6956 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:57.772115  7046 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.772224  6956 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.772521  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.773000  6956 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.773527  6951 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:57.774220  6951 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:57.775424  7086 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:57.775561  6951 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.776651  6949 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:57.777129  6949 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:57.778146  6949 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.779621  6949 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:57.782037  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:57.783248  7046 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:57.783937  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:57.785682  6956 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.765687  7096 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:57.790391  7109 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.790650  6927 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:57.791329  7124 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:01:57.792075  7131 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:01:57.790446  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.792924  1709 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:57.787744  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:57.713655  7043 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.705018  7054 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.744451  7118 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:57.805117  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.806623  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.788910  6853 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:01:57.810384  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:57.686885  6925 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.744325  6931 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.814395  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.704397  7064 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.825846  7083 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:57.814306  7118 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.813803  7053 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.811729  7096 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:57.828732  6934 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:57.846587  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:57.852203  2353 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.849241  1703 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:57.806982  7109 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:01:57.810655  7043 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:57.842947  7096 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.873116  7110 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.879647  2353 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:01:57.876628  7055 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.742s	user 0.000s	sys 0.003s
I20260709 13:01:57.806754  7124 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.875789  7128 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:57.892970  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51317.3us
I20260709 13:01:57.878767  7096 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:57.895254  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:57.863822  6934 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:01:57.902952  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:57.908346  1702 raft_consensus.cc:2410] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:57.913086  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:57.914572  1710 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.919245  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:57.894073  6956 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:57.917199  7016 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.587s	user 0.016s	sys 0.000s
W20260709 13:01:57.928973  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.942715  7043 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.943361  7113 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.941874  7016 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.943922  7113 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.937767  6956 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:57.923000  7083 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:01:57.938103  6927 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.946631  7083 raft_consensus.cc:3055] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:57.946558  6927 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.947085  7083 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:57.948060  7083 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.948745  6927 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:57.945518  7016 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.939318  6931 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.940490  7064 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.941287  6925 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:01:57.950003  7134 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.073s	user 0.001s	sys 0.001s
I20260709 13:01:57.938468  7118 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.950944  7113 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:57.951277  7124 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.953887  6936 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:57.949299  7083 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:57.964332  7126 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.374s	user 0.001s	sys 0.000s
I20260709 13:01:57.908334  7142 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:57.969805  7142 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:57.970357  7142 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:57.982066  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:57.983197  1631 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:57.984686  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:57.997704  1631 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:01:58.016371  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2713) took 1466 ms (client timeout 1709 ms). Trace:
W20260709 13:01:58.016956  1635 rpcz_store.cc:269] 0709 13:01:56.550444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.558087 (+  7643us) service_pool.cc:224] Handling call
0709 13:01:58.016092 (+1458005us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b540010b8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:58.016294 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2112495488}
I20260709 13:01:58.018759  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.019440  1635 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:58.020020  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 972) took 1145 ms. Trace:
I20260709 13:01:58.020735  1627 rpcz_store.cc:276] 0709 13:01:56.874867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.384013 (+509146us) service_pool.cc:224] Handling call
0709 13:01:57.387327 (+  3314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:57.387815 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.387858 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:57.412499 (+ 24641us) log.cc:844] Serialized 26 byte log entry
0709 13:01:57.576639 (+164140us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:57.576706 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.576810 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.963503 (+386693us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:01:57.995710 (+ 32207us) raft_consensus.cc:1685] finished
0709 13:01:57.995744 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.019932 (+ 24188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1978,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":12324,"spinlock_wait_cycles":7717248,"thread_start_us":137043,"threads_started":1,"wal-append.queue_time_us":150521}
I20260709 13:01:58.023484  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.024330  1627 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:01:58.020183  7120 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:01:58.032797  7123 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:01:58.033270  7120 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:57.985037  7096 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:58.036620  7096 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:58.039812  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.039712  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.045351  1627 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:58.045071  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.047549  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:58.049158  1675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.870s (SENT) [suppressed 15 similar messages]
I20260709 13:01:58.038470  7120 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:58.026312  6931 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:01:58.057721  1635 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:01:58.061522  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.062721  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:58.055203  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 53197.5us
I20260709 13:01:58.057825  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.065279  2353 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:58.059613  1631 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:58.070597  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.072978  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.073308  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.063702  1635 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:01:58.073359  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 226862us
W20260709 13:01:58.060520  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 261898us
I20260709 13:01:58.075800  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:01:58.080883  2346 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:58.080674  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.083775  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.084821  1635 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:01:58.087592  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:58.088591  1635 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:58.089804  2346 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:58.093566  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:58.094985  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.096406  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:58.098582  1674 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:58.097038  6994 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.098904  7133 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:58.099614  7133 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.099553  6994 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.100220  7133 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:58.102382  6994 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:58.103192  7095 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:58.104692  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.105887  7038 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:58.106293  7038 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.106650  7038 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:58.108009  6995 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:58.108616  6995 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:58.104532  7099 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.112222  7099 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:58.112958  6995 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.084184  6931 consensus_queue.cc:1243] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:58.114622  7025 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:58.122527  6948 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.123828  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:58.117369  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.125455  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:58.004403  1710 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:01:58.129364  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:58.133265  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 214116us
I20260709 13:01:58.138685  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:58.146090  2353 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.141038  1602 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:58.147804  1710 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:01:58.148123  1602 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:58.148515  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:01:58.136613  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 87486.8us
I20260709 13:01:58.149302  1702 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:58.151865  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.152304  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:58.154677  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:58.156083  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.160728  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:58.170105  1602 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:58.136642  6931 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:58.124001  7038 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.173286  6951 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:58.187808  2353 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:01:58.164953  7099 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:58.162312  7083 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:58.174166  7027 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:58.192818  1602 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.176551  6934 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:58.144994  7142 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.186827  6931 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.205821  1635 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:58.174911  7096 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:01:58.207203  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.197184  6951 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:58.195003  7083 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:58.210500  1635 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:01:58.217972  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.222186  6969 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:58.222682  6969 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:58.223677  1674 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.223613  6969 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.224756  6969 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.224885  6995 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:58.225363  6995 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.225736  6995 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.228286  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:58.228922  1628 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.201867  6949 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:58.229790  1631 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:01:58.229974  6949 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:58.230983  7142 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:58.231348  6949 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.205937  6931 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:58.234109  7083 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:58.231894  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.234849  1631 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:01:58.235958  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:58.236768  1631 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:01:58.233223  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:58.239029  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.239392  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:58.239897  2349 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:58.241237  6994 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:01:58.208740  7045 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 2.378s	user 0.006s	sys 0.016s
I20260709 13:01:58.242560  7027 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.243191  7027 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.243695  7095 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:58.245551  6910 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.246718  7038 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:58.246269  6910 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.248190  6951 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.251410  6910 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:58.251693  6982 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:58.252409  6982 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.253931  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:58.241781  7096 consensus_queue.cc:1243] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:01:58.256942  1702 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:01:58.258864  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.260076  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:58.272061  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:58.272857  1635 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:58.245292  7027 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:58.274507  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.282771  1628 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:01:58.288077  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.288822  1631 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:58.198678  7133 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.270287  7096 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:58.293622  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:58.298218  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:58.311168  1635 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:01:58.312614  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.314049  1635 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:58.317929  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:58.320384  1628 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:58.322605  1628 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:01:58.324188  1628 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:58.327409  1709 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:01:58.329378  1628 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:58.330384  1628 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:58.330469  7142 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:58.331418  7142 consensus_queue.cc:1243] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:01:58.333633  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50279.5us
I20260709 13:01:58.339340  1631 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:58.341826  1631 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:58.320254  7096 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:58.344767  1704 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:01:58.347111  1631 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:58.339213  7159 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:58.349047  1631 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:58.366051  7154 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:01:58.366756  7133 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:58.368044  7096 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:01:58.369658  6982 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:58.347152  7157 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:58.377377  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:58.382880  6931 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.383328  7083 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.384467  6995 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.385319  6982 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.385643  2346 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 10.
I20260709 13:01:58.378015  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 923) took 1875 ms. Trace:
I20260709 13:01:58.387713  2347 rpcz_store.cc:276] 0709 13:01:56.502931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.510894 (+  7963us) service_pool.cc:224] Handling call
0709 13:01:56.519817 (+  8923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.498785 (+978968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.377718 (+878933us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b5400756554. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:58.377943 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2206397312}
W20260709 13:01:58.385322  7155 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.075s	user 0.003s	sys 0.000s
W20260709 13:01:58.391098  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 73298.7us
I20260709 13:01:58.397208  1708 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:58.397853  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.387161  7027 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:58.398653  7027 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.399142  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:58.401124  7027 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:58.402630  6630 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:58.400619  7165 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:58.402953  6949 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:58.409240  7083 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:58.391104  6982 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.416509  6982 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:58.418466  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:58.408046  7165 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:58.435289  7165 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:58.420362  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2175) took 2592 ms (client timeout 1965 ms). Trace:
W20260709 13:01:58.437601  1707 rpcz_store.cc:269] 0709 13:01:55.827628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.035233 (+207605us) service_pool.cc:224] Handling call
0709 13:01:58.420144 (+2384911us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400222c54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:58.420284 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294864128}
I20260709 13:01:58.420594  6949 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.426766  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.439534  1627 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:01:58.440606  6949 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:58.434067  1631 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:58.440543  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:58.445940  1627 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.431874  6995 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:58.467648  1631 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:01:58.468972  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:58.469702  1631 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:58.455976  6931 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:58.478199  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2225) took 1301 ms. Trace:
I20260709 13:01:58.478749  1553 rpcz_store.cc:276] 0709 13:01:57.177356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.177600 (+   244us) service_pool.cc:224] Handling call
0709 13:01:57.183822 (+  6222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.410636 (+1226814us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b18004051c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda276d2060 00007fda276d2bfb 00007fda276dd64b
0709 13:01:58.410817 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.410845 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.411492 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 13:01:58.450929 (+ 39437us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:58.450987 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.451107 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.474125 (+ 23018us) raft_consensus.cc:1685] finished
0709 13:01:58.474149 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.478118 (+  3969us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2122,"raft.queue_time_us":79687,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":1043,"spinlock_wait_cycles":8378240,"thread_start_us":124974,"threads_started":2,"wal-append.queue_time_us":60495}
I20260709 13:01:58.488088  1627 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:58.495884  7110 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:01:58.497825  6824 raft_consensus.cc:670] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } [suppressed 5 similar messages]
I20260709 13:01:58.531642  1631 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
W20260709 13:01:58.512679  7078 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.823s	user 0.005s	sys 0.000s
W20260709 13:01:58.515076  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.515465  7110 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:58.534029  1631 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: [11.6-14.8]
W20260709 13:01:58.539347  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.546437  6738 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.553020  6738 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.543957  7025 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:58.579478  6738 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:58.584420  6738 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.585251  6738 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.590122  6738 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:58.607293  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.608356  1630 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:58.622925  6931 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:58.624903  6931 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:58.628829  6931 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:58.597502  7025 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.594933  7102 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.610695  7110 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.639271  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.640146  1630 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:58.645975  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.646788  2353 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:58.647184  7038 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 13:01:58.647854  7038 raft_consensus.cc:1068] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:58.649046  2353 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:58.652333  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2815) took 2554 ms. Trace:
W20260709 13:01:58.649909  1634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 969) took 2103 ms (client timeout 1805 ms). Trace:
I20260709 13:01:58.650044  1636 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:58.653764  2349 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:58.647128  1627 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:58.656741  1634 rpcz_store.cc:269] 0709 13:01:56.546467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.546592 (+   125us) service_pool.cc:224] Handling call
0709 13:01:58.649679 (+2103087us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b540017b454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:58.649817 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":859111296}
I20260709 13:01:58.632436  6931 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.652781  1705 rpcz_store.cc:276] 0709 13:01:56.098247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.556572 (+458325us) service_pool.cc:224] Handling call
0709 13:01:56.665775 (+109203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.966103 (+1300328us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:57.966626 (+   523us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.179568 (+212942us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.179754 (+   186us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:58.179807 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.652081 (+472274us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54002e95d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:58.652137 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.652268 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":127195,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":104,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":4603,"spinlock_wait_cycles":2989709696,"thread_start_us":212426,"threads_started":1,"wal-append.queue_time_us":237519}
W20260709 13:01:58.638543  7015 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.239s	user 0.010s	sys 0.024s
I20260709 13:01:58.662084  6931 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:58.661000  7015 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.262s	user 0.010s	sys 0.025s
W20260709 13:01:58.669621  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.665967  7110 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:58.679960  7120 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:01:58.683262  1628 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.686520  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 930) took 1431 ms. Trace:
I20260709 13:01:58.698848  7158 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:58.692636  2348 rpcz_store.cc:276] 0709 13:01:57.255598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.255763 (+   165us) service_pool.cc:224] Handling call
0709 13:01:57.256200 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.421139 (+1164939us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b18003ce7a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda276d2060 00007fda276d2bfb 00007fda276dd64b
0709 13:01:58.421324 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.421355 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.421949 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 13:01:58.559688 (+137739us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:58.559723 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.559783 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.685974 (+126191us) raft_consensus.cc:1685] finished
0709 13:01:58.686005 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.686413 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9797,"raft.queue_time_us":23248,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":925,"spinlock_wait_cycles":6721792,"thread_start_us":154310,"threads_started":2,"wal-append.queue_time_us":214242}
I20260709 13:01:58.701186  7158 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:01:58.704548  7128 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:01:58.705803  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.713044  7128 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.715605  7128 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:58.711086  7029 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:58.697237  1628 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:58.708757  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.721614  7158 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:58.723145  1635 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:58.722151  7158 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.724532  6921 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:58.725317  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:58.725644  6921 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:58.728149  6921 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:58.729517  7047 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:58.730036  7047 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.721340  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.731550  6921 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.731914  7147 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.732604  7047 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:58.732489  7147 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:01:58.733618  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 653752us
I20260709 13:01:58.734817  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:58.734992  7147 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:58.736260  1526 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:58.735764  7147 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:01:58.737380  6936 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:58.738235  6936 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:58.730930  7158 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:58.739241  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2720) took 1547 ms. Trace:
I20260709 13:01:58.739986  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:58.743786  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:58.744990  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:58.674309  7165 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:01:58.746003  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:58.751936  1557 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:01:58.752164  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.721283  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2870) took 1343 ms. Trace:
I20260709 13:01:58.752918  2346 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:58.751477  1553 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
W20260709 13:01:58.756703  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.746793  7172 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:01:58.747695  7171 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:01:58.739679  1636 rpcz_store.cc:276] 0709 13:01:57.191607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.485336 (+293729us) service_pool.cc:224] Handling call
0709 13:01:58.649826 (+1164490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.659050 (+  9224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.738910 (+ 79860us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540017b450. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:58.739157 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2643651840}
I20260709 13:01:58.730454  1525 leader_election.cc:400] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15.
I20260709 13:01:58.751418  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:58.794695  1525 leader_election.cc:403] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:01:58.794963  2347 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:58.795351  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 65061.9us
I20260709 13:01:58.796728  1558 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:01:58.799286  1710 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:58.799998  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:58.800423  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:58.801373  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:58.753108  1560 rpcz_store.cc:276] 0709 13:01:57.378191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.397511 (+ 19320us) service_pool.cc:224] Handling call
0709 13:01:58.721073 (+1323562us) spinlock_profiling.cc:243] Waited 59 ms on lock 0x7b540026b1d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:58.721213 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135802240}
I20260709 13:01:58.760179  1635 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:58.764199  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.803660  2317 connection.cc:256] RPC call timeout handler was delayed by -1.2802969230000372s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
I20260709 13:01:58.804057  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:01:58.804500  2317 connection.cc:256] RPC call timeout handler was delayed by -1.0840969230000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
W20260709 13:01:58.767527  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46327.5us
I20260709 13:01:58.771476  1634 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:01:58.806993  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.808061  1634 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:01:58.808732  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:58.809418  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:58.810969  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.774186  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.775424  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.813562  2353 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:01:58.778301  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 968) took 2232 ms (client timeout 1587 ms). Trace:
W20260709 13:01:58.814782  1632 rpcz_store.cc:269] 0709 13:01:56.546028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.546240 (+   212us) service_pool.cc:224] Handling call
0709 13:01:58.777990 (+2231750us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54003cdcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:58.778193 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738223488}
I20260709 13:01:58.780575  1631 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: []
W20260709 13:01:58.788059  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 62845.7us
I20260709 13:01:58.802008  1710 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:58.817873  1710 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:58.811641  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.818878  1634 raft_consensus.cc:2410] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:58.805047  1635 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:01:58.823974  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:58.824970  1632 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:58.825402  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:58.807137  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:58.827702  1602 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.806085  1674 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.829214  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.831131  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:58.831887  2344 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:58.832790  1702 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:58.833401  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:58.834944  1629 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:58.822144  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.837859  1631 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:01:58.838480  1631 pending_rounds.cc:85] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:01:58.838831  1631 pending_rounds.cc:107] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:01:58.839160  1631 raft_consensus.cc:2889] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:58.846243  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.831945  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:58.847333  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:58.847821  1554 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:01:58.848732  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:58.850062  2350 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:01:58.852205  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46270us
I20260709 13:01:58.853266  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.855683  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.857990  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.858682  2351 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:58.859565  1709 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:58.689749  7038 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 13:01:58.860893  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:01:58.865176  1674 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.869164  1635 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:01:58.871030  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:58.872850  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:58.880671  1707 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:01:58.858492  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:01:58.874491  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:58.884384  2345 raft_consensus.cc:2368] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15.
I20260709 13:01:58.886289  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:58.793395  6542 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:58.912077  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:01:58.912559  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:58.823307  6936 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:58.732290  6921 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:58.917172  2344 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:01:58.924434  1704 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:01:58.823022  7151 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:58.919903  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.933553  2353 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:01:58.928581  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.930378  1557 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:01:58.938143  1558 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:01:58.938670  1558 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 9
I20260709 13:01:58.938932  1558 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:01:58.939219  1558 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:58.858106  6979 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.961468  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:58.962594  1560 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:01:58.944029  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:01:58.857859  6922 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.831135  6825 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 13:01:58.949925  2317 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:58.951275  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:58.954068  7175 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:01:58.956573  1706 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: [10.10-12.11]
I20260709 13:01:58.752483  7164 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.164s
I20260709 13:01:58.858180  7124 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.862566  7096 raft_consensus.cc:2764] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:01:58.969149  1561 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.970640  2317 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:58.954259  6930 raft_consensus.cc:3055] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:58.891619  7123 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:58.987287  2345 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:58.988781  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 102618us
I20260709 13:01:59.002950  7128 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:58.914553  7118 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:59.017105  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46584.5us
I20260709 13:01:58.915469  6981 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:58.928642  6936 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:59.073321  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:59.038815  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.025622  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:59.060972  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:01:59.009112  7171 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:01:58.933110  7151 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.070092  1561 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:01:59.109298  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:59.110811  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:58.956000  6645 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.107s
I20260709 13:01:58.914953  7120 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.148s
I20260709 13:01:59.076964  1560 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:01:59.081142  2317 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:59.001011  2347 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:01:59.122556  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:59.122537  1602 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:01:59.093541  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.115690  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:59.024283  7053 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:59.125982  7053 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.016664  7054 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:59.126515  7053 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:59.133307  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:59.007151  6922 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.124891  7086 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:59.118417  7128 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:59.014547  7066 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 13:01:59.133373  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:59.071653  7025 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:59.078424  6844 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:59.137282  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:59.091140  6936 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:59.022068  6930 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:01:59.065260  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:59.140827  1628 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:01:59.196141  1628 pending_rounds.cc:85] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 11
I20260709 13:01:59.196575  1628 pending_rounds.cc:107] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:01:59.196921  1628 raft_consensus.cc:2889] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:59.142947  1707 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:59.105223  7071 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:01:59.148626  1560 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:01:59.104918  7109 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:01:59.143544  7074 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:59.117626  7151 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:59.053421  6853 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:01:59.120164  7096 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:01:59.026598  7124 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.140239  7054 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:59.206097  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 68911.8us
I20260709 13:01:59.206949  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2967) took 1225 ms. Trace:
I20260709 13:01:59.148857  7147 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:59.128882  7111 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:01:59.141307  6645 consensus_queue.cc:1243] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:01:59.223912  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.222470  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:59.248129  1628 rpcz_store.cc:276] 0709 13:01:57.981083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.328309 (+347226us) service_pool.cc:224] Handling call
0709 13:01:58.329203 (+   894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.682833 (+353630us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b5400341fa0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda2765806f 00007fda276d0f60 00007fda276d27de 00007fda276dcb49 00007fda276dd416 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62
0709 13:01:59.206496 (+523663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.206866 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2776960}
I20260709 13:01:59.160219  7086 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:01:59.207887  6738 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:59.259049  1560 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:01:59.160036  6844 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:59.303591  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 242082us
W20260709 13:01:59.296125  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.307505  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:01:59.212585  7180 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:59.261699  6825 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:59.335888  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:59.221696  7038 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:01:59.345000  6853 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:59.275278  7093 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:59.357770  7093 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.213282  7074 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:59.359598  7074 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:59.360246  7093 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:01:59.220804  7103 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:01:59.364329  6844 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.365674  6844 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:59.366941  7109 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:59.367434  7109 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.367887  7109 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:59.346351  6825 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:59.370683  6825 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:59.352298  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:59.377076  1560 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:01:59.257663  7147 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:59.352663  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:59.345083  7083 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.350442  6645 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:59.289276  6922 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:59.255339  7054 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:59.336998  7180 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.355315  6894 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:01:59.355469  7176 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:01:59.242061  6921 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:59.391289  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:01:59.376443  6825 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:59.397847  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:59.392692  6645 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:59.376997  7038 raft_consensus.cc:1068] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:59.261946  7047 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.136s
I20260709 13:01:59.382535  7111 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:01:59.417668  1552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.008s	sys 0.038s
W20260709 13:01:59.401751  1701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.015s	sys 0.033s
W20260709 13:01:59.419482  1701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.015s	sys 0.033s
W20260709 13:01:59.419067  1552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.010s	sys 0.038s
I20260709 13:01:59.402513  6894 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.405635  6922 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:59.291046  6930 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.403048  7180 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:59.396745  7147 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:59.279441  7128 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.397528  7083 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:59.429523  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:01:59.430423  7102 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.793s	user 0.009s	sys 0.005s
I20260709 13:01:59.426599  7093 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:59.426250  6894 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:59.441109  6922 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:59.473688  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.410883  6921 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.463274  6645 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:59.476282  7102 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:59.505470  2348 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
W20260709 13:01:59.514024  1676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.005s	sys 0.043s
W20260709 13:01:59.514736  1676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.005s	sys 0.043s
I20260709 13:01:59.488415  2345 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:01:59.494462  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 913) took 1840 ms. Trace:
I20260709 13:01:59.498071  6645 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:59.531879  1708 rpcz_store.cc:276] 0709 13:01:57.654144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.654353 (+   209us) service_pool.cc:224] Handling call
0709 13:01:58.393767 (+739414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.403805 (+ 10038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.494190 (+1090385us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b5400250dd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:59.494380 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1532185344}
I20260709 13:01:59.422397  7111 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.532966  2345 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 9
I20260709 13:01:59.596156  2345 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.10
I20260709 13:01:59.596625  2345 raft_consensus.cc:2917] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Aborting config change with OpId 10.10: Aborted: Op aborted by new leader
I20260709 13:01:59.597102  2345 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:59.441346  7047 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:01:59.519135  7124 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:01:59.628320  2353 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEARNER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:59.639856  6756 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEARNER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:59.593462  7158 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.854s	user 0.014s	sys 0.000s
I20260709 13:01:59.641062  7158 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:59.650725  1703 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:01:59.652848  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:59.654227  7158 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:01:59.654651  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2954) took 2968 ms. Trace:
I20260709 13:01:59.655159  1633 rpcz_store.cc:276] 0709 13:01:56.686242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.052675 (+366433us) service_pool.cc:224] Handling call
0709 13:01:58.778206 (+1725531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.778632 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.778666 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.779238 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.258467 (+479229us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:59.258517 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.258602 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.525841 (+267239us) raft_consensus.cc:1685] finished
0709 13:01:59.525876 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.654377 (+128501us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54003cdcd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:59.654585 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33397,"raft.queue_time_us":843,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":38275,"spinlock_wait_cycles":3967272320,"thread_start_us":328428,"threads_started":1,"wal-append.queue_time_us":588359}
W20260709 13:01:59.660637  6931 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.991s	user 0.012s	sys 0.000s
I20260709 13:01:59.665006  6844 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:59.665560  6844 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:01:59.666469  6844 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.668040  6844 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:59.669827  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.638343  7096 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:01:59.672237  6936 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:59.673401  1558 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:59.471805  6922 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:59.607271  7071 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.677266  7071 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.679613  7071 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:59.680163  7071 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.680619  7071 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.682343  1558 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:01:59.696758  7095 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:01:59.697355  7095 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:59.698300  7095 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.699549  7095 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:59.685127  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:01:59.434900  7086 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.712728  1558 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:01:59.507448  6921 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:59.693593  7096 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:59.704474  7103 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.705051  7124 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:59.731711  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:59.738408  1708 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
W20260709 13:01:59.742076  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.744449  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:01:59.745381  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.746905  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:59.747663  1701 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:59.750597  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:01:59.754426  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:01:59.754794  1708 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:59.758374  1560 raft_consensus.cc:2410] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:01:59.771377  1555 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.015s	sys 0.010s
W20260709 13:01:59.787151  1555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.016s	sys 0.011s
I20260709 13:01:59.758283  2352 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:59.788347  1527 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.016s	sys 0.014s
I20260709 13:01:59.758448  2350 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:01:59.788730  1527 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.016s	sys 0.014s
W20260709 13:01:59.789314  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 584983us
W20260709 13:01:59.784770  2317 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.039s	sys 0.027s
W20260709 13:01:59.790374  1556 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.020s	sys 0.009s
W20260709 13:01:59.801633  1556 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.021s	sys 0.009s
I20260709 13:01:59.802587  1557 raft_consensus.cc:606] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:59.803036  1557 raft_consensus.cc:614] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:59.803755  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2880) took 1028 ms. Trace:
W20260709 13:01:59.790503  2317 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.039s	sys 0.027s
I20260709 13:01:59.737636  7124 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:59.805732  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 683252us
I20260709 13:01:59.806972  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:01:59.787990  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2876) took 1045 ms. Trace:
I20260709 13:01:59.808270  1555 rpcz_store.cc:276] 0709 13:01:58.742702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.742839 (+   137us) service_pool.cc:224] Handling call
0709 13:01:58.743232 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.743472 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:59.143015 (+399543us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.787610 (+644595us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.787672 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:59.787722 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.787789 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.787917 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":619235,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":1555,"thread_start_us":1043306,"threads_started":2}
I20260709 13:01:59.810396  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:01:59.811064  1555 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:59.770465  7110 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.091s	user 0.010s	sys 0.004s
W20260709 13:01:59.774900  6542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.004s	sys 0.024s
W20260709 13:01:59.869552  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 79392.2us
W20260709 13:01:59.871309  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:59.870707  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 78851.4us
W20260709 13:01:59.762214  6736 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.004s	sys 0.018s
W20260709 13:01:59.896767  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 89867.7us
W20260709 13:01:59.900717  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.901150  2317 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:59.748807  7096 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:59.902886  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 498152us
W20260709 13:01:59.793875  7123 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.016s	sys 0.050s
I20260709 13:01:59.785763  6853 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:01:59.897440  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.013s	sys 0.020s
W20260709 13:01:59.911675  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.014s	sys 0.020s
W20260709 13:01:59.912042  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 1.0469e+06us
W20260709 13:01:59.750687  7120 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.020s	sys 0.018s
W20260709 13:01:59.870748  7151 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.623s	user 0.007s	sys 0.007s
I20260709 13:01:59.868698  7117 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:59.900120  6736 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.008s	sys 0.030s
I20260709 13:01:59.983268  1525 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:01:59.870287  6936 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:59.747134  7103 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:59.870317  6542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.005s	sys 0.025s
I20260709 13:01:59.909888  1710 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:01:59.946215  7093 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:01:59.949795  6738 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.951326  7102 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:59.952667  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1350) took 1004 ms. Trace:
I20260709 13:01:59.958750  6921 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:59.960687  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 953) took 1252 ms. Trace:
W20260709 13:01:59.995025  7053 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.840s	user 0.009s	sys 0.004s
I20260709 13:01:59.962090  6853 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:01:59.899190  6931 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:01:59.963406  7151 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:01:59.965153  7180 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 13:01:59.966267  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41400 (request call id 22) took 1036 ms. Trace:
I20260709 13:02:00.038846  1557 rpcz_store.cc:276] 0709 13:01:58.929398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.930249 (+   851us) service_pool.cc:224] Handling call
0709 13:01:59.966007 (+1035758us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b5400208354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d4853 00007fda2c6a260a 00007fda25c287d4 00007fda25c28762 00007fda25c28691 00007fda25c2860c 00007fda25c278c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:59.966159 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1996550912}
I20260709 13:01:59.968443  6756 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:59.969141  6995 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.941578  6825 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:01:59.974301  7119 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:01:59.975848  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:59.978536  7074 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.618s	user 0.012s	sys 0.000s
I20260709 13:01:59.978605  7066 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.979636  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:01:59.979990  6894 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
I20260709 13:01:59.980376  6824 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:59.980808  7071 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:01:59.899636  7110 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.981501  7118 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.907s	user 0.009s	sys 0.004s
I20260709 13:01:59.907351  7083 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:01:59.927361  7123 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.017s	sys 0.062s
W20260709 13:01:59.983317  1554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.005s	sys 0.019s
I20260709 13:01:59.936591  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:00.102995  1554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.005s	sys 0.019s
I20260709 13:02:00.072181  6921 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.990105  2347 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.016s	sys 0.040s
I20260709 13:01:59.998720  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2835) took 1202 ms. Trace:
W20260709 13:02:00.104851  2347 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.016s	sys 0.040s
I20260709 13:02:00.000200  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:01:59.956866  7054 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.551s	user 0.006s	sys 0.001s
I20260709 13:02:00.010888  1555 raft_consensus.cc:2410] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:00.014372  1675 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:01:59.869012  6567 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.114s
I20260709 13:02:00.109004  6567 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:00.100785  7110 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.109397  6567 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:00.110450  6567 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.111305  7110 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.111732  6567 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.111867  7110 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:00.025990  1710 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.098134  7118 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:00.116679  7123 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:01:59.804234  1556 rpcz_store.cc:276] 0709 13:01:58.775722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.778797 (+  3075us) service_pool.cc:224] Handling call
0709 13:01:58.779133 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.779356 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:59.143584 (+364228us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.802343 (+658759us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.802405 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:59.802463 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.803532 (+  1069us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.803692 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53247,"prepare.queue_time_us":583893,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":119,"thread_start_us":918019,"threads_started":2}
I20260709 13:02:00.117722  7123 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.118372  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:00.119526  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:00.028705  7102 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.120260  7102 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.091949  7074 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.122061  2349 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:00.123191  7074 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.123391  7103 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.124311  7103 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.125062  7151 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.028999  2353 rpcz_store.cc:276] 0709 13:01:58.947964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.980294 (+ 32330us) service_pool.cc:224] Handling call
0709 13:01:59.628028 (+647734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.633382 (+  5354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.952381 (+318999us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54009902d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:59.952572 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1488473984}
I20260709 13:02:00.036535  7158 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 13:02:00.126443  7158 raft_consensus.cc:1068] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:00.074038  6995 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.131350  6995 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.094360  7066 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:00.133041  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 196651us
I20260709 13:02:00.024127  7093 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.013979  6542 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:01:59.944020  6981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.019s	sys 0.018s
W20260709 13:02:00.148641  6981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.020s	sys 0.018s
I20260709 13:02:00.149016  6981 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:00.150024  6981 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.151866  7102 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:00.047811  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.006s	sys 0.021s
W20260709 13:02:00.153448  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.011s	sys 0.021s
W20260709 13:02:00.047648  1559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.000s	sys 0.037s
I20260709 13:02:00.084796  1558 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:02:00.154441  7066 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.035183  7111 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.028905  7016 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:00.159767  7016 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:00.160535  7158 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:00.161170  7158 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.160893  7016 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.162446  7016 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:00.163674  7158 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.088461  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:00.165822  6936 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:00.166395  1557 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:00.090342  2346 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:02:00.072871  7109 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.705s	user 0.009s	sys 0.004s
W20260709 13:02:00.072702  6979 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.087s	user 0.007s	sys 0.008s
I20260709 13:02:00.094396  1709 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
W20260709 13:01:59.929646  7120 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.025s	sys 0.019s
I20260709 13:02:00.104190  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2883) took 1240 ms. Trace:
W20260709 13:01:59.989615  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.093454  6824 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:00.180343  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.107357  7096 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.107889  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:00.182296  1555 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:00.183374  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:00.034289  2348 rpcz_store.cc:276] 0709 13:01:58.708020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.716804 (+  8784us) service_pool.cc:224] Handling call
0709 13:01:58.717256 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.507985 (+790729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.960392 (+452407us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54004be1d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:01:59.960610 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812103424}
I20260709 13:02:00.136291  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:00.184692  1560 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:00.104934  1703 rpcz_store.cc:276] 0709 13:01:58.796715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.831666 (+ 34951us) service_pool.cc:224] Handling call
0709 13:01:59.998419 (+1166753us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54002e81d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:01:59.998634 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1845606656}
W20260709 13:02:00.146879  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.186870  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.134413  7083 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.096158  6894 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.147774  6542 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.166545  6936 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.090675  7119 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.156076  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2898) took 1032 ms. Trace:
I20260709 13:02:00.232654  1558 rpcz_store.cc:276] 0709 13:01:59.134587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.975546 (+840959us) service_pool.cc:224] Handling call
0709 13:02:00.155981 (+180435us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:00.156493  1559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.002s	sys 0.039s
W20260709 13:02:00.157824  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.134892  7147 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.652s	user 0.006s	sys 0.002s
I20260709 13:02:00.045197  7180 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.021658  7124 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.124935  7103 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.171732  1710 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:00.177209  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:00.038938  7053 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.030678  6738 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:00.158888  6995 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:00.196813  7165 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:00.197122  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:00.196969  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 182748us
I20260709 13:02:00.255776  1560 raft_consensus.cc:2410] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:00.256014  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:00.179289  1554 rpcz_store.cc:276] 0709 13:01:58.863718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.864013 (+   295us) service_pool.cc:224] Handling call
0709 13:01:58.920567 (+ 56554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.920788 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:59.403172 (+482384us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.103681 (+700509us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:00.103760 (+    79us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:00.103827 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.103924 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.104112 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8436,"prepare.queue_time_us":442673,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":1063,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":11618,"thread_start_us":1152306,"threads_started":2}
W20260709 13:02:00.198129  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51724.3us
I20260709 13:02:00.146500  6921 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.247035  6825 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:00.259073  6825 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.260836  6921 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.206017  7083 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.240309  7043 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:00.241884  7202 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.208328  7102 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:00.209003  6894 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.194332  7111 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:00.210616  6824 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.224931  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:00.112411  7110 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.237172  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2875) took 1494 ms. Trace:
I20260709 13:02:00.180167  6979 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15.
I20260709 13:02:00.244177  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2842) took 1445 ms. Trace:
I20260709 13:02:00.289122  1710 rpcz_store.cc:276] 0709 13:01:58.798978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.801290 (+  2312us) service_pool.cc:224] Handling call
0709 13:01:58.801627 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.909306 (+1107679us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b4c00306980. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda259a6a20 00007fda259b7a29 00007fda2597d644 00007fda259a4f2d 00007fda259a4b93 00007fda276d518b 00007fda276d5cb5 00007fda276d0f30 00007fda276d27de 00007fda276dcb49 00007fda276dd416
0709 13:02:00.243732 (+334426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.244090 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62445952}
W20260709 13:02:00.249174  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.013s	sys 0.022s
I20260709 13:02:00.145313  7054 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:00.246004  6932 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:02:00.263484  7083 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.252005  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:00.179908  7109 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.272899  1557 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:00.276100  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:00.280109  1559 rpcz_store.cc:276] 0709 13:01:58.742388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.753495 (+ 11107us) service_pool.cc:224] Handling call
0709 13:01:58.753902 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.754187 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:59.189968 (+435781us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.236778 (+1046810us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:00.236838 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:00.236886 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.236963 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.237107 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1414,"prepare.queue_time_us":608807,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":2323,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":1094,"thread_start_us":1361841,"threads_started":2}
W20260709 13:02:00.292732  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.013s	sys 0.022s
I20260709 13:02:00.256968  6995 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.241658  7188 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:02:00.293390  6824 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:00.326659  7119 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.241459  7147 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.262441  7202 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.293444  7195 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:02:00.294057  1555 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:00.331388  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2889) took 1383 ms. Trace:
I20260709 13:02:00.331790  1555 rpcz_store.cc:276] 0709 13:01:58.947538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.288738 (+1341200us) service_pool.cc:224] Handling call
0709 13:02:00.289125 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.331056 (+ 41931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.331315 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.274644  7124 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:00.295115  1558 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:00.334192  6930 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:00.334848  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2894) took 1385 ms. Trace:
I20260709 13:02:00.335290  1558 rpcz_store.cc:276] 0709 13:01:58.949066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.294575 (+1345509us) service_pool.cc:224] Handling call
0709 13:02:00.294933 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.334532 (+ 39599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.334778 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.332499  7202 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.298055  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2899) took 1149 ms. Trace:
I20260709 13:02:00.261449  6825 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.339296  1557 rpcz_store.cc:276] 0709 13:01:59.148975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.176810 (+1027835us) service_pool.cc:224] Handling call
0709 13:02:00.297849 (+121039us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b54002ff654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:00.297995 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58292608}
I20260709 13:02:00.333328  1555 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:00.339145  1561 raft_consensus.cc:2410] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:00.299243  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2890) took 1351 ms. Trace:
I20260709 13:02:00.342681  1554 rpcz_store.cc:276] 0709 13:01:58.947827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.289020 (+1341193us) service_pool.cc:224] Handling call
0709 13:02:00.298699 (+  9679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.298916 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.298943 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:00.298990 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:00.299023 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.299079 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.299185 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.300940  1560 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:00.302395  1559 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:00.344957  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 833) took 1319 ms. Trace:
I20260709 13:02:00.345259  1559 rpcz_store.cc:276] 0709 13:01:59.025270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.301296 (+1276026us) service_pool.cc:224] Handling call
0709 13:02:00.302202 (+   906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.344686 (+ 42484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.344899 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:00.347694  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.347237  1553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.009s	sys 0.028s
W20260709 13:02:00.349047  1553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.010s	sys 0.029s
I20260709 13:02:00.349857  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 832) took 1326 ms. Trace:
I20260709 13:02:00.350418  1560 rpcz_store.cc:276] 0709 13:01:59.024252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.300433 (+1276181us) service_pool.cc:224] Handling call
0709 13:02:00.300746 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.349541 (+ 48795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.349766 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.352360  1560 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:02:00.338304  7147 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.354159  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 834) took 1315 ms. Trace:
I20260709 13:02:00.368870  1555 rpcz_store.cc:276] 0709 13:01:59.025560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.332751 (+1307191us) service_pool.cc:224] Handling call
0709 13:02:00.333112 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.340506 (+  7394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.340713 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.370482  1555 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:00.353761  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2230) took 1187 ms. Trace:
W20260709 13:02:00.302778  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 862387us
I20260709 13:02:00.316056  7053 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:00.322517  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:00.371798  1560 rpcz_store.cc:276] 0709 13:01:59.166237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.351589 (+1185352us) service_pool.cc:224] Handling call
0709 13:02:00.352031 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.352880 (+   849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.353517 (+   637us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.372592  1628 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:00.357170  1554 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:00.374361  1560 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:02:00.375064  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 840) took 1172 ms. Trace:
I20260709 13:02:00.375299  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2234) took 1117 ms. Trace:
I20260709 13:02:00.357754  6736 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:00.324702  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:00.294047  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1346) took 1411 ms. Trace:
I20260709 13:02:00.330626  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:00.382725  1631 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:00.362478  7026 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.298341  1552 raft_consensus.cc:2368] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:00.274876  7096 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.255033  6571 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:02:00.293217  6922 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.021s	sys 0.011s
I20260709 13:02:00.265722  7111 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:00.367885  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 79779us
I20260709 13:02:00.395917  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:00.341363  1557 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:00.375941  1560 rpcz_store.cc:276] 0709 13:01:59.257945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.373573 (+1115628us) service_pool.cc:224] Handling call
0709 13:02:00.374032 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.374949 (+   917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.375223 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.375636  1554 rpcz_store.cc:276] 0709 13:01:59.202696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.356154 (+1153458us) service_pool.cc:224] Handling call
0709 13:02:00.356673 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.374666 (+ 17993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.374973 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.387799  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2233) took 1133 ms. Trace:
I20260709 13:02:00.398837  1555 rpcz_store.cc:276] 0709 13:01:59.254102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.369921 (+1115819us) service_pool.cc:224] Handling call
0709 13:02:00.370290 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.370914 (+   624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.387715 (+ 16801us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.262856  7193 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:00.401080  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:00.401949  1708 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:00.296757  7109 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.295143  7180 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.314157  6936 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.341768  7117 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:00.393818  1635 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 10.
I20260709 13:02:00.393899  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:00.402688  1560 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:00.390697  1710 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:02:00.428007  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:00.388113  7119 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.309949  7103 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.404914  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:00.314375  7196 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 13:02:00.405998  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:00.396211  1675 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:00.407327  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:00.314653  6931 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.379348  7158 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.298856  7054 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.412495  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:00.415505  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:00.421396  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:00.310422  6995 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:00.413941  7202 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:00.397503  6922 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.021s	sys 0.012s
I20260709 13:02:00.431949  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 835) took 1378 ms. Trace:
I20260709 13:02:00.432186  1560 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:00.429241  7117 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:00.391294  6571 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:00.403651  7197 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.381537  2346 rpcz_store.cc:276] 0709 13:01:58.882596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.928309 (+ 45713us) service_pool.cc:224] Handling call
0709 13:02:00.293839 (+1365530us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400540e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:00.293987 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2649904384}
I20260709 13:02:00.403898  7096 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.397532  7195 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.293126  7093 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.441638   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:02:00.447098  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51139.1us
I20260709 13:02:00.456048  1554 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:02:00.462205  1561 rpcz_store.cc:276] 0709 13:01:59.053422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.110538 (+ 57116us) service_pool.cc:224] Handling call
0709 13:02:00.431720 (+1321182us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54002521d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:00.431877 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2822415232}
I20260709 13:02:00.392158  7201 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.464298  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2240) took 1082 ms. Trace:
W20260709 13:02:00.481515  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.037s	sys 0.042s
W20260709 13:02:00.487032  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.037s	sys 0.042s
I20260709 13:02:00.486389  1552 rpcz_store.cc:276] 0709 13:01:59.381739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.652550 (+270811us) service_pool.cc:224] Handling call
0709 13:02:00.464008 (+811458us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400207954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:00.464207 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1482139520}
I20260709 13:02:00.459735  7188 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:00.489025  1703 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:00.397574  7147 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:02:00.467211  1631 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:00.493983  1552 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:02:00.494515  1552 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 9
I20260709 13:02:00.494797  1552 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.10
W20260709 13:02:00.467669  2345 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.031s	sys 0.043s
I20260709 13:02:00.495122  1552 raft_consensus.cc:2917] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Aborting config change with OpId 10.10: Aborted: Op aborted by new leader
W20260709 13:02:00.493364  2317 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.016s	sys 0.014s
I20260709 13:02:00.495625  1552 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.496809  1552 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 12.11
W20260709 13:02:00.444887  1600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.018s	sys 0.058s
I20260709 13:02:00.485519  1554 pending_rounds.cc:85] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 6
I20260709 13:02:00.397867  7026 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:00.495599  2345 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.031s	sys 0.043s
I20260709 13:02:00.408947  7120 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
W20260709 13:02:00.497187  2317 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.017s	sys 0.014s
I20260709 13:02:00.497406  1554 pending_rounds.cc:107] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 11.7
W20260709 13:02:00.498792  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.510126  1554 raft_consensus.cc:2889] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:00.498025  1600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.018s	sys 0.058s
W20260709 13:02:00.512954  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.513626  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.467811  1602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.000s	sys 0.028s
W20260709 13:02:00.515236  1602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.000s	sys 0.029s
I20260709 13:02:00.518003  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:00.446640  7119 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.473438  7110 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.461158  6931 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.528981  6931 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:00.546996  7111 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:00.550442  1459 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 188) took 29573 ms (client timeout 15000 ms). Trace:
I20260709 13:02:00.465374  7158 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.554256  6738 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.556726  7158 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.499768  1552 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.496412  7103 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.522426  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:00.497500  7201 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.564525  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:00.523654  7043 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:00.525600  6571 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.579069  6571 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:00.580494  7093 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.526501  7025 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:00.486789  7195 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.467785  7202 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.529055  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2897) took 1416 ms. Trace:
I20260709 13:02:00.588650  1557 rpcz_store.cc:276] 0709 13:01:59.112722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.340667 (+1227945us) service_pool.cc:224] Handling call
0709 13:02:00.341096 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.397046 (+ 55950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.528712 (+131666us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540023d5a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27651dc0 00007fda276dda02 00007fda276d9dd6 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:02:00.528973 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257703808}
I20260709 13:02:00.538923  7096 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.540076  7110 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.511929  1632 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:02:00.560765  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 659753us
I20260709 13:02:00.493396  7188 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.558823  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:00.501147  2318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.004s	sys 0.045s
I20260709 13:02:00.561764   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:02:00.572805  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.572811  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:00.594828  2318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.004s	sys 0.045s
I20260709 13:02:00.572057  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:00.584733  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:00.596484  2353 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:02:00.597178  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:00.587671  1661 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:43857 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:43857 timed out after 15.000s (SENT)
I20260709 13:02:00.583724  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:00.511660  6922 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.564500  6738 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.603237  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:00.583397  7043 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:00.482437  7197 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:00.608912  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.586702  7025 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:00.558550  7103 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.684870  7103 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.688627  7103 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.689720  7103 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.598053  1557 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.658243  1702 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:00.598569  7096 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:00.696673  7096 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.699184  7096 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.700297  7192 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.700906  7192 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.701874  6738 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:00.702893  6738 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:00.703301  7192 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:00.704464  7158 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:00.704950  7158 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:00.706131  7158 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.707973  7158 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.699803  7096 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:00.713241  7096 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:00.714481  7096 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.703943  7192 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:00.716961  7192 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:00.718546  7192 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.573727  6995 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.609306  6922 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.722644  6922 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.603312  7188 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:00.723857  7025 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.725436  7025 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:00.727669  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.607281  7111 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:00.728621  7111 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:00.729800  7111 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:00.673389  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:00.676085  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:00.742370  1706 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:00.744097  7103 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.751695  6995 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:00.676728  7128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.005s	sys 0.023s
W20260709 13:02:00.753922  7128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.006s	sys 0.023s
W20260709 13:02:00.677012  2344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.012s	sys 0.021s
W20260709 13:02:00.755472  2344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.012s	sys 0.021s
W20260709 13:02:00.677304  6981 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.006s	sys 0.010s
W20260709 13:02:00.757144  6981 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.007s	sys 0.010s
W20260709 13:02:00.677691  7086 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.010s	sys 0.021s
W20260709 13:02:00.758445  7086 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.010s	sys 0.021s
W20260709 13:02:00.551115  1459 rpcz_store.cc:269] 0709 13:01:30.977041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.977324 (+   283us) service_pool.cc:224] Handling call
0709 13:02:00.550255 (+29572931us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:55.869183 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:55.986010 (+116827us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:55.986088 (+    78us) write_op.cc:435] Acquired schema lock
0709 13:01:55.986114 (+    26us) tablet.cc:662] Decoding operations
0709 13:01:55.987670 (+  1556us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:55.987808 (+   138us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:55.987841 (+    33us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:01:55.989130 (+  1289us) tablet.cc:701] Row locks acquired
0709 13:01:55.989183 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:01:55.989405 (+   222us) write_op.cc:270] Start()
0709 13:01:56.064765 (+ 75360us) write_op.cc:276] Timestamp: P: 1783602115989327 usec, L: 0
0709 13:01:56.077451 (+ 12686us) op_driver.cc:348] REPLICATION: starting
0709 13:01:56.157837 (+ 80386us) log.cc:844] Serialized 5506 byte log entry
0709 13:01:56.558433 (+400596us) op_driver.cc:464] REPLICATION: finished
0709 13:01:56.851433 (+293000us) write_op.cc:301] APPLY: starting
0709 13:01:56.910096 (+ 58663us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:57.478763 (+568667us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:57.490288 (+ 11525us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:57.967702 (+477414us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:57.968103 (+   401us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 13:01:57.968174 (+    71us) write_op.cc:312] APPLY: finished
0709 13:01:57.968668 (+   494us) log.cc:844] Serialized 121 byte log entry
0709 13:01:57.968949 (+   281us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:57.969125 (+   176us) write_op.cc:454] Released schema lock
0709 13:01:57.969203 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:58.032063 (+ 62860us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:58.032103 (+    40us) write_op.cc:454] Released schema lock
0709 13:01:58.665268 (+633165us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b18002cfc20. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
Metrics: {"mutex_wait_us":9767949,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":153754,"apply.run_cpu_time_us":31464,"apply.run_wall_time_us":1274995,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58467,"num_ops":13,"prepare.queue_time_us":365684,"prepare.run_cpu_time_us":39536,"prepare.run_wall_time_us":447086,"raft.queue_time_us":82471,"raft.run_cpu_time_us":22199,"raft.run_wall_time_us":124359,"replication_time_us":476636,"spinlock_wait_cycles":9047296,"thread_start_us":536638,"threads_started":4,"wal-append.queue_time_us":919065,"wal-append.run_cpu_time_us":29684,"wal-append.run_wall_time_us":2248107}]]}
I20260709 13:02:00.746961  7103 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:02:00.800529  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 132233us
W20260709 13:02:00.773329  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.770562  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:00.808574  7102 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:00.810263  7053 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:00.811116  7188 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.812374  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:00.812024  1456 catalog_manager.cc:5697] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 4 to 9, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:00.813498  7053 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:00.815409  7053 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:00.815222  1456 catalog_manager.cc:5697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 3 to 11. New cstate: current_term: 11 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:00.820950  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:00.821894  6922 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:02:00.787436  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.844658  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.846174  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:00.846571  7119 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.848963  7026 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:00.850684  7202 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.852018  7201 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:00.853641  7195 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:00.651700  7083 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.598157  7110 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.858269  7083 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:00.860328  7110 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.860446  7083 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:00.861270  7216 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:00.868137  1557 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:00.858222  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:00.871951  1560 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.816920  1456 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 6 to 7, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:00.878619  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.766425  6853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.000s	sys 0.021s
W20260709 13:02:00.766966  7123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.003s	sys 0.020s
W20260709 13:02:00.902587  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.902864  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:00.923727  6930 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.004s
I20260709 13:02:00.924633  2346 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
W20260709 13:02:00.925010  7124 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.592s	user 0.013s	sys 0.000s
I20260709 13:02:00.926371  1556 raft_consensus.cc:2410] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 14 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 13:02:00.926855  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:00.927778  1557 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:02:00.927924  1456 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 16 to 17, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:00.877203  7192 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:00.935329  1456 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 5 to 8, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:00.908502  6853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.000s	sys 0.023s
W20260709 13:02:00.907938  7123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.003s	sys 0.021s
I20260709 13:02:00.951594  1557 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
W20260709 13:02:00.971668  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.974187  2347 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.011s	sys 0.041s
W20260709 13:02:00.939741  7016 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.772s	user 0.015s	sys 0.001s
W20260709 13:02:00.941850  6824 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.615s	user 0.013s	sys 0.000s
I20260709 13:02:00.984768  2318 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:00.989904  6567 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:00.992923  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 957) took 1246 ms. Trace:
I20260709 13:02:00.994589  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 843) took 1067 ms. Trace:
W20260709 13:02:00.994946  7117 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.001s
I20260709 13:02:00.995179  1456 catalog_manager.cc:5697] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:01.003679  2347 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.012s	sys 0.042s
I20260709 13:02:01.003252  1560 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:01.006585  7202 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.012312  1555 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:02:01.027981  1555 pending_rounds.cc:85] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 9
I20260709 13:02:01.028355  1555 pending_rounds.cc:107] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:02:01.028724  1555 raft_consensus.cc:2889] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:01.009001  1559 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.010s	sys 0.011s
W20260709 13:02:01.030593  1559 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.010s	sys 0.011s
I20260709 13:02:01.013854  1556 rpcz_store.cc:276] 0709 13:01:59.927257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.119283 (+192026us) service_pool.cc:224] Handling call
0709 13:02:00.994358 (+875075us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b5400230d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:00.994515 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1853518208}
W20260709 13:02:01.022940  1630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.011s	sys 0.036s
W20260709 13:02:01.023303  1636 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.000s	sys 0.047s
W20260709 13:02:01.034015  1636 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.000s	sys 0.047s
I20260709 13:02:01.033885  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.034444  2348 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:01.034797  2349 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:01.035692  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.036494  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.037060  2353 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:02:01.037277  2349 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:01.038260  2346 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:01.039021  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:01.035036  1630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.011s	sys 0.036s
I20260709 13:02:01.013700  2344 rpcz_store.cc:276] 0709 13:01:59.746471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.758415 (+ 11944us) service_pool.cc:224] Handling call
0709 13:01:59.758799 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.759280 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.759319 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.759899 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.756486 (+996587us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:00.756579 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.756762 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.992608 (+235846us) raft_consensus.cc:1685] finished
0709 13:02:00.992635 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.992821 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":218,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":5234,"spinlock_wait_cycles":41216,"thread_start_us":887891,"threads_started":1,"wal-append.queue_time_us":1203681}
I20260709 13:02:01.058219  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:01.012979  7165 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.747s	user 0.013s	sys 0.000s
W20260709 13:02:01.057128  1627 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.008s	sys 0.041s
I20260709 13:02:01.061527  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.063437  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:01.064741  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.038311  7216 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.065654  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:01.066658  1558 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.016s	sys 0.016s
I20260709 13:02:01.067973  2350 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:02:01.074743  1558 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.017s	sys 0.016s
W20260709 13:02:01.069649  1633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.036s	sys 0.053s
I20260709 13:02:01.070758  2344 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: [12.13-12.13]
W20260709 13:02:01.076519  1633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.036s	sys 0.053s
W20260709 13:02:01.069885  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 85207.7us
I20260709 13:02:01.070544  1456 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 9 to 13, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:01.020464  6542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.010s	sys 0.012s
W20260709 13:02:01.072337  1627 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.008s	sys 0.041s
W20260709 13:02:01.072362  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.003s	sys 0.019s
W20260709 13:02:01.080018  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.003s	sys 0.020s
I20260709 13:02:01.055915  6930 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:01.080338  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 595174us
W20260709 13:02:01.036705  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.007s	sys 0.015s
I20260709 13:02:01.079589  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:01.080734  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:01.086131  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.007s	sys 0.015s
I20260709 13:02:01.087284  2318 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:01.088495  2318 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:01.086318  6542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.012s	sys 0.013s
I20260709 13:02:01.065966  7165 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:01.093463  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.023581  7117 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:01.095045  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.068452  7096 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:01.113365  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.114223  2348 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:01.096573  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.115037  1705 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:01.074702  6922 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.097222  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:01.088482  7193 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.687s	user 0.009s	sys 0.005s
I20260709 13:02:01.098511  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.098208  7016 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.093838  7215 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:01.116863  1628 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:01.093289  7128 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:01.121351  1628 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:01.118629  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.122902  1628 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: [11.6-14.8]
I20260709 13:02:01.122896  1629 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:02:01.095220  7095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.003s	sys 0.031s
I20260709 13:02:01.098102  7053 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:02:01.076951  7196 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.002s	sys 0.022s
I20260709 13:02:01.103176  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:01.136261  1634 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:01.094066  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.107437  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.032506  7202 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.110589  7188 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:01.125878  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.119108  1632 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:02:01.094272  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.134034  1456 catalog_manager.cc:5697] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 6 to 11, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.140650  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:01.151608  2348 raft_consensus.cc:2410] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:02:01.114560  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:01.113636  6853 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.091876  6930 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.147245  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.159752  2353 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:02:01.152468  2344 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:01.116173  7124 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.083815  7216 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.162042  1707 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
W20260709 13:02:01.165302  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 55201.1us
W20260709 13:02:01.136090  7196 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.002s	sys 0.023s
I20260709 13:02:01.127067  7193 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:01.125136  6922 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:01.115659  6981 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:02:01.139249  7215 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:01.138800  7197 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:01.185465  2351 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:02:01.187593  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.188372  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:01.191713  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.189980  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.191733  1632 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.192971  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2848) took 1944 ms. Trace:
I20260709 13:02:01.193792  1707 rpcz_store.cc:276] 0709 13:01:59.248304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.248830 (+   526us) service_pool.cc:224] Handling call
0709 13:01:59.254917 (+  6087us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:01.167007 (+1912090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.192610 (+ 25603us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b540016bf54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:01.192890 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4371344640}
I20260709 13:02:01.156970  7096 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:01.128751  7095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.004s	sys 0.031s
I20260709 13:02:01.171473  7212 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.127599  7016 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:01.205852  2344 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:02:01.203418  2346 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
W20260709 13:02:01.201800  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 66834.8us
I20260709 13:02:01.184698  6981 consensus_queue.cc:1243] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:01.205770  1634 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:02:01.214609  1456 catalog_manager.cc:5697] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 9 to 10, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 10 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.230545  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.231544  1634 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:01.231002  1456 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.232957  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.216817  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.233448  1456 catalog_manager.cc:5697] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.237156  1456 catalog_manager.cc:5697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.233678  1555 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:01.243789  7216 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:01.168656  7124 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:01.248327  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.169220  6853 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:01.247812  7216 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.251931  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43696.6us
I20260709 13:02:01.213462  7212 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.246074  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.234781  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.259166  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:01.260309  1628 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:01.261754  1555 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:01.233719  1634 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:02:01.263765  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:01.210101  7093 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:02:01.266513  7212 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:01.270205  7220 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
W20260709 13:02:01.270977  7217 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
W20260709 13:02:01.271916  7225 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:02:01.272058  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:02:01.272876  7222 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:02:01.274745  6931 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:01.279156  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.306891  1676 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:01.212900  7096 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.310999  2346 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:01.285537  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:01.317219  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.318145  1633 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:02:01.319116  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.319594  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.319993  1633 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:01.320380  2352 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:01.321673  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.323304  1634 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:01.324505  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.325901  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:01.326838  2351 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:01.325392  1634 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:02:01.327430  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:01.287197  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:01.290043  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:01.292804  1552 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:02:01.331606  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.262979  7216 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.297415  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:01.297415  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:01.264921  7218 log.cc:927] Time spent T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.148s	user 0.001s	sys 0.001s
W20260709 13:02:01.264021  7221 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:02:01.305279  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:01.294734  7210 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:02:01.283429  1557 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:02:01.295086  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:01.293893  1553 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.011s	sys 0.011s
I20260709 13:02:01.332723  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:01.317132  7096 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:01.284135  7228 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:02:01.303905  6567 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:02:01.377175  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.396473  2344 raft_consensus.cc:2376] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11. Re-sending same reply.
W20260709 13:02:01.397717  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 110569us
I20260709 13:02:01.398387  1628 raft_consensus.cc:2368] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17.
I20260709 13:02:01.382853  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.398885  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:01.330608  1555 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
W20260709 13:02:01.317220  6931 raft_consensus.cc:1068] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:01.333225  1632 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:01.397765  1554 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:01.394232  1704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.008s	sys 0.014s
W20260709 13:02:01.402422  1704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.008s	sys 0.015s
W20260709 13:02:01.395713  1553 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.012s	sys 0.014s
I20260709 13:02:01.404296  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:01.381069  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.313077  6542 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:01.313280  7095 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:01.394027  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.013s	sys 0.010s
W20260709 13:02:01.424364  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.013s	sys 0.010s
W20260709 13:02:01.425812  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 822560us
I20260709 13:02:01.418167  6932 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:01.428666  6932 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:01.429119  6932 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:01.430655  7202 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:01.433656  7117 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:01.434401  7117 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.435207  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.435885  1629 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:02:01.418314  6542 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.437027  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.437661  7102 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:01.438067  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.434914  7117 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:01.442058  7117 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:01.445060  6567 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:01.445902  6567 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.371459  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:01.298923  7215 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.439370  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.445183  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1161) took 1769 ms. Trace:
W20260709 13:02:01.328192  7111 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.010s	sys 0.031s
I20260709 13:02:01.377974  7165 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.439015  1632 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:01.450027  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.453022  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:01.456080  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.453495  7165 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.456818  1629 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:01.457861  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.458475  1629 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:02:01.458978  7165 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:01.455703  1633 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:01.449787  2345 rpcz_store.cc:276] 0709 13:01:59.676230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.702953 (+ 26723us) service_pool.cc:224] Handling call
0709 13:01:59.703290 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.703653 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.703672 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.704039 (+   367us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.502274 (+798235us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:00.502313 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.502382 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.444766 (+942384us) raft_consensus.cc:1685] finished
0709 13:02:01.444804 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.445096 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":6780,"thread_start_us":774854,"threads_started":1,"wal-append.queue_time_us":1287569}
I20260709 13:02:01.461094  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:01.450412  7111 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.010s	sys 0.032s
I20260709 13:02:01.461915  1633 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:01.461995  7016 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:01.329491  6853 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:01.462189  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.462551  7111 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.463174  7111 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.464389  1629 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:02:01.448300  6567 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:01.437690  6542 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.466884  7216 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:01.467468  7216 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:01.467865  7216 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:01.454532  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:01.450878  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.470479  1628 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:01.473059  7096 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:01.474287  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2995) took 1156 ms. Trace:
I20260709 13:02:01.474678  1636 rpcz_store.cc:276] 0709 13:02:00.318277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.322596 (+  4319us) service_pool.cc:224] Handling call
0709 13:02:00.322938 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.323380 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.323408 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.323895 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 13:02:01.034715 (+710820us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:01.034785 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.034892 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.474060 (+439168us) raft_consensus.cc:1685] finished
0709 13:02:01.474093 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.474224 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6536,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":58,"thread_start_us":710290,"threads_started":1,"wal-append.queue_time_us":832796}
I20260709 13:02:01.404995  1634 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:01.477126  1636 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:01.477329  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:01.479027  7124 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:01.479946  7223 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:02:01.398087  6930 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:01.483589  6922 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:01.484889  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.485641  2351 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:01.486234  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:01.420267  7025 consensus_meta.cc:287] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.679s	user 0.015s	sys 0.000s
I20260709 13:02:01.504930  6922 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:01.506983  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:01.510108  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:01.511292  7124 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:01.511809  7124 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.512241  7124 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:01.473238  6571 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.894s	user 0.012s	sys 0.000s
I20260709 13:02:01.468878  7216 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.531625  7216 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:01.535277  1636 raft_consensus.cc:2955] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:01.545166  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:01.546886  7117 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:01.423986  7095 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.559974  7095 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:01.560959  1628 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:02:01.568830  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:01.570537  7117 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:01.571322  7117 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:01.572190  7117 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:01.489544  7111 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:01.419111  7219 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.256s	user 0.000s	sys 0.002s
I20260709 13:02:01.490087  7096 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:01.491264  1601 leader_election.cc:400] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:01.723176  1601 leader_election.cc:403] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:02:01.496719  7043 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.879s	user 0.010s	sys 0.003s
I20260709 13:02:01.729619  7043 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:01.732642  7043 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:01.499744  7192 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.542s	user 0.013s	sys 0.000s
I20260709 13:02:01.588482  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2862) took 1052 ms. Trace:
I20260709 13:02:01.733963  1709 rpcz_store.cc:276] 0709 13:02:00.536016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.555029 (+ 19013us) service_pool.cc:224] Handling call
0709 13:02:00.555438 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.586623 (+1031185us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:01.586656 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:01.586700 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:01.586752 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.588204 (+  1452us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b5400223654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:01.588249 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.588418 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81307,"raft.queue_time_us":81980,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":153,"spinlock_wait_cycles":2183445760}
I20260709 13:02:01.589850  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:01.736119  1702 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:01.591094  1708 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:02:01.629354  2347 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.013s	sys 0.035s
I20260709 13:02:01.637782  1554 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:02:01.584333  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 185568us
W20260709 13:02:01.742386  2347 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.014s	sys 0.035s
I20260709 13:02:01.605355  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:01.743742  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 138535us
I20260709 13:02:01.744558  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:01.632104  1558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.009s	sys 0.021s
I20260709 13:02:01.586417  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:01.745658  1558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.009s	sys 0.022s
I20260709 13:02:01.746150  1707 raft_consensus.cc:2410] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 14 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 13:02:01.746784  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 836) took 2681 ms. Trace:
W20260709 13:02:01.747586  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 256473us
I20260709 13:02:01.747670  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2997) took 1428 ms. Trace:
I20260709 13:02:01.747949  6853 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:01.588047  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:01.748879  6853 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:01.747233  1558 rpcz_store.cc:276] 0709 13:01:59.065904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.336649 (+1270745us) service_pool.cc:224] Handling call
0709 13:02:00.337742 (+  1093us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.338095 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:01.075288 (+737193us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.746276 (+670988us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.746437 (+   161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:01.746481 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:01.746558 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.746622 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.746726 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":810952,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":96,"raft.queue_time_us":7010,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":133,"thread_start_us":1407002,"threads_started":2}
I20260709 13:02:01.752013  1525 leader_election.cc:400] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17.
I20260709 13:02:01.752386  1525 leader_election.cc:403] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:01.753091  7016 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17.
W20260709 13:02:01.638085  1630 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.013s	sys 0.018s
I20260709 13:02:01.590330  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:01.753871  1630 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.014s	sys 0.019s
I20260709 13:02:01.754112  6922 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:01.754272  1706 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:01.755295  7236 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:01.755306  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 2998) took 1436 ms. Trace:
I20260709 13:02:01.756501  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:01.758670  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.753667  6824 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:01.748363  1627 rpcz_store.cc:276] 0709 13:02:00.318946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.324782 (+  5836us) service_pool.cc:224] Handling call
0709 13:02:00.325137 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.325556 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:01.080081 (+754525us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.405869 (+325788us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:01.430031 (+ 24162us) log.cc:844] Serialized 26 byte log entry
0709 13:02:01.430378 (+   347us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:01.430478 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.434074 (+  3596us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.747388 (+313314us) raft_consensus.cc:1685] finished
0709 13:02:01.747422 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.747589 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5584,"prepare.queue_time_us":869135,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":2272,"raft.queue_time_us":10991,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":18455,"thread_start_us":1040891,"threads_started":2,"wal-append.queue_time_us":647930}
I20260709 13:02:01.770776  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:01.771530  1705 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:01.779227  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:01.641376  1559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.006s	sys 0.043s
I20260709 13:02:01.759950  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1160) took 2365 ms. Trace:
W20260709 13:02:01.781059  1559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.007s	sys 0.043s
W20260709 13:02:01.773726  1631 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.015s	sys 0.043s
W20260709 13:02:01.782451  1631 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.016s	sys 0.044s
W20260709 13:02:01.766269  1635 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.029s	sys 0.029s
I20260709 13:02:01.756868  1630 rpcz_store.cc:276] 0709 13:02:00.319260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.326083 (+  6823us) service_pool.cc:224] Handling call
0709 13:02:00.326434 (+   351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:00.326824 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.326855 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:00.327432 (+   577us) log.cc:844] Serialized 52 byte log entry
0709 13:02:01.069563 (+742131us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:01.754364 (+684801us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.754789 (+   425us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.754971 (+   182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.755053 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.755097 (+    44us) raft_consensus.cc:1685] finished
0709 13:02:01.755110 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.755241 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":178846,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":95,"raft.queue_time_us":4986,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":205,"thread_start_us":1393691,"threads_started":2,"wal-append.queue_time_us":857395}
I20260709 13:02:01.577433  1602 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:01.585691  7158 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:01.723258  7165 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.771612  1627 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:01.793413  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:01.786294  2316 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.015s	sys 0.001s
I20260709 13:02:01.781216  2347 rpcz_store.cc:276] 0709 13:01:59.394498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.394719 (+   221us) service_pool.cc:224] Handling call
0709 13:01:59.395278 (+   559us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:59.395915 (+   637us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:00.105519 (+709604us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.742901 (+1637382us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.743203 (+   302us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:01.743920 (+   717us) log.cc:844] Serialized 52 byte log entry
0709 13:02:01.744082 (+   162us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:01.744306 (+   224us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.744486 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.744590 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.759628 (+ 15038us) raft_consensus.cc:1685] finished
0709 13:02:01.759662 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.759849 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6258,"prepare.queue_time_us":994729,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":20456,"raft.run_cpu_time_us":560,"raft.run_wall_time_us":29228,"thread_start_us":2259292,"threads_started":3,"wal-append.queue_time_us":955269}
I20260709 13:02:01.783537  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:01.789633  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:01.796057  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:01.797029  2316 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.015s	sys 0.002s
I20260709 13:02:01.797093  1628 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:01.785693  1635 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.029s	sys 0.029s
W20260709 13:02:01.799265  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.769454  6922 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:01.805007  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:01.777885  7192 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:01.800596  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:01.819028  1630 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:01.819859  7193 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:01.820345  7193 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.820708  7193 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:01.791435  7053 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.005s	sys 0.010s
W20260709 13:02:01.821784  7053 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.006s	sys 0.010s
I20260709 13:02:01.822196  7053 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:01.815830  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:01.776136  7212 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:02:01.823498  7053 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.824801  7212 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:01.821290  7231 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:01.827476  1674 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:01.828764  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2864) took 1265 ms. Trace:
I20260709 13:02:01.829171  1704 rpcz_store.cc:276] 0709 13:02:00.563240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.563405 (+   165us) service_pool.cc:224] Handling call
0709 13:02:00.588456 (+ 25051us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.588876 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.588906 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.590325 (+  1419us) log.cc:844] Serialized 26 byte log entry
0709 13:02:01.403037 (+812712us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:01.403124 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.403234 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.828526 (+425292us) raft_consensus.cc:1685] finished
0709 13:02:01.828551 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.828697 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":395,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":8535,"thread_start_us":670949,"threads_started":1,"wal-append.queue_time_us":1113115}
I20260709 13:02:01.602761  6542 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.592744  6932 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.633953  7124 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.799991  6567 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.819119  7197 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:02:01.816754  7158 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:01.837668  7192 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.805212  6931 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:01.839862  1702 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:01.778465  7236 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:02:01.791178  7188 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:02:01.841555  7228 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:01.799921  7165 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:01.851204  6571 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:01.852712  6931 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:01.866792  2318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.039s	sys 0.026s
W20260709 13:02:01.872159  2318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.039s	sys 0.026s
W20260709 13:02:01.868043  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.875713  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:01.876993  2347 raft_consensus.cc:2410] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:01.850380  6567 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.857831  7228 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:01.837740  7128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.004s	sys 0.013s
I20260709 13:02:01.871809  6931 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:01.874801  6571 raft_consensus.cc:697] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:01.881368  7228 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:01.902776  7192 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:01.898151  7165 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:01.886073  7158 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:01.910480  7124 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:01.936106  7095 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.939774  7095 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:01.882592  7128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.004s	sys 0.013s
W20260709 13:02:01.872735  7196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.015s	sys 0.006s
I20260709 13:02:01.944146  7193 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:01.973017  7193 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:01.791676  1556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.005s	sys 0.011s
I20260709 13:02:02.087327  7236 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:02.046638   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:02.047276  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.015s	sys 0.013s
I20260709 13:02:02.059640  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 838) took 2911 ms. Trace:
W20260709 13:02:02.055011  1527 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.000s	sys 0.004s
I20260709 13:02:02.091845  6932 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:02.055011  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.014s	sys 0.033s
I20260709 13:02:02.096313  7235 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:02.100900  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 331865us
W20260709 13:02:02.056022  1553 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.000s	sys 0.004s
W20260709 13:02:02.100376  7093 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.000s	sys 0.012s
I20260709 13:02:01.957618  7128 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:02.258764  7093 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.003s	sys 0.015s
I20260709 13:02:02.093863  7096 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:01.950915  7117 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:02.263967  7046 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:01.957963  7196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.015s	sys 0.006s
I20260709 13:02:01.968070  7228 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:02.184415  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:02.177927  1633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.009s	sys 0.016s
W20260709 13:02:02.177907  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 593545us
W20260709 13:02:02.190351  1629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.002s	sys 0.009s
I20260709 13:02:02.191562  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:02.192987  1634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.001s	sys 0.014s
W20260709 13:02:02.193812  1632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.008s	sys 0.004s
W20260709 13:02:02.268364  1634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.002s	sys 0.014s
W20260709 13:02:02.268508  1632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.009s	sys 0.004s
I20260709 13:02:02.194818  6571 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:02.212013  7053 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.274211  2346 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:02:02.274693  7053 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:02.256233  7236 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:02.274897  7128 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:02.275507  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:02.275876  7236 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:02.277956  7270 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:02.278008  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:02.215912  1636 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.678s	user 0.012s	sys 0.000s
I20260709 13:02:02.219873  2347 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:02.221614  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:02.223084  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:01.872753  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 631019us
W20260709 13:02:02.225507  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.227465  6567 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:02.185021  6542 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:02.240099  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 963) took 1147 ms. Trace:
W20260709 13:02:02.258272  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.014s	sys 0.048s
W20260709 13:02:02.258286  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.021s	sys 0.031s
W20260709 13:02:02.261246  1556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.017s	user 0.005s	sys 0.011s
I20260709 13:02:02.291450  7228 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:02.261979  7235 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:02.263073  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2886) took 3335 ms. Trace:
I20260709 13:02:02.263542  7096 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:02.294379  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.295795  7235 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:02.295892  7256 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:02.296355  7096 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:02.169189  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3004) took 1080 ms. Trace:
W20260709 13:02:02.260514  1527 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.005s	sys 0.015s
W20260709 13:02:02.297739  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 856438us
I20260709 13:02:02.297056  7256 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:02.298177  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:02.299268  1702 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:02.300505  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:02.301146  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:02.302232  1704 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:02.297066  1631 rpcz_store.cc:276] 0709 13:02:01.088350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.088512 (+   162us) service_pool.cc:224] Handling call
0709 13:02:01.088816 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.089077 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:01.783690 (+694613us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.168030 (+384340us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:02.168155 (+   125us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:02.168324 (+   169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.168509 (+   185us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.168843 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5285,"prepare.queue_time_us":195797,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":169,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":218,"spinlock_wait_cycles":81280,"thread_start_us":1055204,"threads_started":2,"wal-append.queue_time_us":383465}
W20260709 13:02:02.269605  1629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.002s	sys 0.009s
W20260709 13:02:02.270805  1633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.009s	sys 0.017s
I20260709 13:02:02.282635  7272 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:02.313694  7117 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:02.314656  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3016) took 1138 ms. Trace:
I20260709 13:02:02.315132  1633 rpcz_store.cc:276] 0709 13:02:01.175853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.463633 (+287780us) service_pool.cc:224] Handling call
0709 13:02:01.463959 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:01.464773 (+   814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.464803 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:01.465309 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.314034 (+848725us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:02.314157 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.314301 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.314364 (+    63us) raft_consensus.cc:1685] finished
0709 13:02:02.314379 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.314592 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":838475,"threads_started":1,"wal-append.queue_time_us":474285}
I20260709 13:02:02.316610  1558 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:02.318570  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:02.320238  1706 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:02.277779  7093 consensus_queue.cc:1243] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 13.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:02:02.287688  2349 rpcz_store.cc:276] 0709 13:02:01.092574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.093211 (+   637us) service_pool.cc:224] Handling call
0709 13:02:02.239431 (+1146220us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400540e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:02.239763 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2464597888}
I20260709 13:02:02.255853  1559 rpcz_store.cc:276] 0709 13:01:59.148053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.346087 (+1198034us) service_pool.cc:224] Handling call
0709 13:02:00.346427 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.432585 (+ 86158us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:01.062107 (+629522us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.781940 (+719833us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:01.782021 (+    81us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:01.782102 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.058648 (+276546us) spinlock_profiling.cc:243] Waited 85.9 ms on lock 0x7b5400312754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:02.058788 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.059249 (+   461us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":785470,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":3366,"raft.queue_time_us":1017,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":7159,"spinlock_wait_cycles":197470464,"thread_start_us":1318104,"threads_started":2,"wal-append.queue_time_us":757547}
I20260709 13:02:02.303325  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:02.311201  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:02.311864  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2749) took 1129 ms. Trace:
I20260709 13:02:02.325850  1629 rpcz_store.cc:276] 0709 13:02:01.182346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.466432 (+284086us) service_pool.cc:224] Handling call
0709 13:02:01.466795 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:01.467251 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.467285 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:01.467853 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.311076 (+843223us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:02.311204 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.311343 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.311437 (+    94us) raft_consensus.cc:1685] finished
0709 13:02:02.311461 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.311726 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":842411,"threads_started":1,"wal-append.queue_time_us":486419}
I20260709 13:02:02.327818  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:02.330466  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:02.329250  2347 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:02.312320  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:02.331835  1704 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:02.325912  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:02.333634  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:02.334571  2349 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:02.335043  7202 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:02.335743  7202 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:02.337158  1555 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:02.294031  1553 rpcz_store.cc:276] 0709 13:01:58.927297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.927532 (+   235us) service_pool.cc:224] Handling call
0709 13:01:58.927842 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.928079 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:59.457040 (+528961us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.541982 (+1084942us) spinlock_profiling.cc:243] Waited 50 ms on lock 0x7b180039d5a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda2b0d23c7 00007fda276e49a7 00007fda276f40bf 00007fda276f4038
0709 13:02:01.426498 (+884516us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.262120 (+835622us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:02.262268 (+   148us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:02.262443 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.262620 (+   177us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.262938 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39863,"prepare.queue_time_us":1281291,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":94,"raft.queue_time_us":17107,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":248,"spinlock_wait_cycles":115011840,"thread_start_us":3028962,"threads_started":4,"wal-append.queue_time_us":1535638}
I20260709 13:02:02.331828  1554 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:02.300527  7256 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:02.332516  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:02.339679  1705 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
W20260709 13:02:02.347496  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 73610.4us
I20260709 13:02:02.352689  7270 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 13:02:02.353448  7270 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:02.355070  7093 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.357154  7093 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:02.337399  7202 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:02.342236  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.310598  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3005) took 1220 ms. Trace:
I20260709 13:02:02.333120  7053 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:02.375239  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:02.376314  1554 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:02.376662  1555 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:02.377902  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:02.377907  7228 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:02.378433  7274 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:02.378742  7188 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:02.379109  7228 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:02.379968  7274 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:02.380525  7274 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:02.380925  7117 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:02.382292  7046 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:02.382712  7228 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:02.382738  7046 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:02.377812  7193 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:02.379129  1553 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:02.383855  7193 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:02.384421  7046 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:02.385792  7046 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:02.386709  7193 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:02.398689  1456 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.981s	user 0.004s	sys 0.012s
W20260709 13:02:02.407582  1456 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.144s	user 0.005s	sys 0.013s
W20260709 13:02:02.411967  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 584459us
W20260709 13:02:02.422878  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 145119us
I20260709 13:02:02.423295  7093 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:02.424199  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:02.425895  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:02.430088  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:02.433555  7274 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:02.435724  7274 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:02.443413  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:02.449599  2349 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
W20260709 13:02:02.450923  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 77702us
I20260709 13:02:02.462849  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:02.463397  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 50327.7us
I20260709 13:02:02.370615  1635 rpcz_store.cc:276] 0709 13:02:01.089879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.090057 (+   178us) service_pool.cc:224] Handling call
0709 13:02:01.110763 (+ 20706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.111045 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:01.798827 (+687782us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.246130 (+447303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:02.252385 (+  6255us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:02.257989 (+  5604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.267099 (+  9110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.285662 (+ 18563us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b1800392020. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1
0709 13:02:02.310267 (+ 24605us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1744,"prepare.queue_time_us":208913,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":2575,"raft.queue_time_us":377,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":2877,"spinlock_wait_cycles":4529536,"thread_start_us":1112097,"threads_started":2,"wal-append.queue_time_us":455378}
I20260709 13:02:02.483362  1553 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:02.486590  7096 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:02.487241  7096 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:02.487710  7096 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:02.486591  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:02.490540  1702 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:02.494966  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 70980.3us
I20260709 13:02:02.495617  7265 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:02.496138  7265 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.496459  7265 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:02.483780  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:02.501335  1558 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:02.516489  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:02.517521  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:02.510986  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:02.525007  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:02.544190  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 70881us
I20260709 13:02:02.566439  1706 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:02.582176  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:02.583056  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.559020  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:02.617578  1553 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:02.619452  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:02.622349  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:02.648919  1558 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:02.641634  1704 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:02:02.674227  1628 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.761s	user 0.012s	sys 0.010s
I20260709 13:02:02.642513  1554 pending_rounds.cc:85] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 6
W20260709 13:02:02.647919  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 207630us
I20260709 13:02:02.623536  2349 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:02:02.693212  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:02.693979  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 91187.6us
I20260709 13:02:02.642040  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:02.673923  7278 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:02:02.641582  1707 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
W20260709 13:02:02.697259  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:02.674474  6931 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.715s	user 0.015s	sys 0.003s
I20260709 13:02:02.648340  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:02.700398  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 199709us
I20260709 13:02:02.623492  7193 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.696535  1554 pending_rounds.cc:107] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:02:02.566205  7235 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.136s
I20260709 13:02:02.716104  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:02.712733  1554 raft_consensus.cc:2889] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:02.711701  1628 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:02.709220  7046 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:02.729549  7046 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.730111  7046 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:02.754307  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 230490us
I20260709 13:02:02.754428  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:02.755504  2351 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:02.756124  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:02.715871  7270 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:02.763473  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 849) took 1526 ms. Trace:
I20260709 13:02:02.777230  6932 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:02.763015  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:02.778957  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64712.8us
I20260709 13:02:02.777137  1556 rpcz_store.cc:276] 0709 13:02:01.236744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.236988 (+   244us) service_pool.cc:224] Handling call
0709 13:02:01.237428 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.289595 (+1052167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.289659 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.290705 (+  1046us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.444888 (+154183us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:02.444951 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.455535 (+ 10584us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.763173 (+307638us) raft_consensus.cc:1685] finished
0709 13:02:02.763198 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.763394 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":193407,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":117,"thread_start_us":1204684,"threads_started":2,"wal-append.queue_time_us":311981}
I20260709 13:02:02.788102  1709 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 11: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:02.790388  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 996) took 1170 ms. Trace:
I20260709 13:02:02.790881  1628 rpcz_store.cc:276] 0709 13:02:01.574872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.780580 (+205708us) service_pool.cc:224] Handling call
0709 13:02:02.738571 (+957991us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:02.778374  6932 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:02.800868  6932 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:02.787745  1706 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:02.718381  6931 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:02.831404  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:02.838627  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:02.843461  1630 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.846s	user 0.012s	sys 0.009s
W20260709 13:02:02.841917  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 274544us
W20260709 13:02:02.847976  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:02.830190  7236 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.009s
W20260709 13:02:02.874662  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 158311us
I20260709 13:02:02.874402  1630 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:02.886860  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:02.883426  1708 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:02.884132  1558 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:02.822461  7193 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:02.890690  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 62705.3us
I20260709 13:02:02.892985  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:02.893635  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:02.915891  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.919606  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 988) took 1407 ms. Trace:
I20260709 13:02:02.922928  1557 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:02:02.923310  1560 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:02.925058  7212 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:02:02.928707  7202 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:02.888669  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:02.929962  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:02.915894  1710 pending_rounds.cc:85] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 7
I20260709 13:02:02.942294  1710 pending_rounds.cc:107] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:02:02.943825  2351 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:02:02.944159  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 994) took 1365 ms. Trace:
I20260709 13:02:02.942549  1710 raft_consensus.cc:2889] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:02.944494  1630 rpcz_store.cc:276] 0709 13:02:01.574239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.800255 (+226016us) service_pool.cc:224] Handling call
0709 13:02:02.912847 (+1112592us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:02.947563  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 78341.1us
I20260709 13:02:02.860554  7236 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:02.950289  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:02.954265  1552 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:02.956103  2349 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:02.920027  1636 rpcz_store.cc:276] 0709 13:02:01.511831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.534302 (+ 22471us) service_pool.cc:224] Handling call
0709 13:02:01.534763 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.535083 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:02.325271 (+790188us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.510578 (+185307us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.714224 (+203646us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.918403 (+204179us) spinlock_profiling.cc:243] Waited 993 us on lock 0x7b180037d680. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda2b0d23c7 00007fda276e49a7 00007fda276f40bf 00007fda276f4038
0709 13:02:02.918618 (+   215us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.918964 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.919086 (+   122us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:02.919145 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:02.919310 (+   165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.919393 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.919536 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29042,"prepare.queue_time_us":45006,"prepare.run_cpu_time_us":262,"prepare.run_wall_time_us":20609,"raft.queue_time_us":217772,"raft.run_cpu_time_us":590,"raft.run_wall_time_us":42055,"spinlock_wait_cycles":2282880,"thread_start_us":247902,"threads_started":2,"wal-append.queue_time_us":152818}
I20260709 13:02:02.973680  1556 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:02.976385  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:02.889684  7046 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:02.989506  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.986550  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:02.993216  1553 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:02.954476  7256 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:02.998975  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 273098us
W20260709 13:02:03.001381  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 142316us
I20260709 13:02:03.002482  7256 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:03.022383  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 69967.8us
I20260709 13:02:03.026527  7256 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.035569  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:03.050299  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.047919  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:03.029425  7093 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:03.049464  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.054865  1558 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:02:03.067502  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 191514us
W20260709 13:02:03.070465  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 470896us
I20260709 13:02:03.054386  7093 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.076207  1553 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:03.078094  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 997) took 1502 ms. Trace:
I20260709 13:02:03.078562  1627 rpcz_store.cc:276] 0709 13:02:01.575150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.211340 (+636190us) service_pool.cc:224] Handling call
0709 13:02:02.212107 (+   767us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.212922 (+   815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.212959 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.252027 (+ 39068us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.405739 (+153712us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:02.405803 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.405910 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.077780 (+671870us) raft_consensus.cc:1685] finished
0709 13:02:03.077799 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.077922 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11192,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":203,"thread_start_us":149677,"threads_started":1,"wal-append.queue_time_us":806041}
I20260709 13:02:03.085357  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:03.072396  1525 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.563s (SENT) [suppressed 18 similar messages]
I20260709 13:02:03.065651  6931 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:03.090967  1553 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:03.093505  7289 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:03.098227  1558 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:02:03.099870  7288 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:02:03.101589  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 43569.7us
I20260709 13:02:03.102169  7046 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:03.102490  7096 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.614s	user 0.014s	sys 0.000s
I20260709 13:02:03.103045  1553 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:03.102919  7096 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.103899  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 999) took 1528 ms. Trace:
I20260709 13:02:03.114876  1631 rpcz_store.cc:276] 0709 13:02:01.575716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.303576 (+727860us) service_pool.cc:224] Handling call
0709 13:02:02.315786 (+ 12210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.354755 (+ 38969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.354795 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.355878 (+  1083us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.477383 (+121505us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:02.477441 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.477541 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.103692 (+626151us) raft_consensus.cc:1685] finished
0709 13:02:03.103716 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.103842 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38146,"raft.queue_time_us":40709,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":203,"thread_start_us":114065,"threads_started":1,"wal-append.queue_time_us":735132}
W20260709 13:02:03.126732  7279 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.472s	user 0.001s	sys 0.001s
I20260709 13:02:03.127413  7096 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:03.084259  7093 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:03.129688  7117 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:02:03.134936  7274 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.004s
I20260709 13:02:03.139349  7236 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:03.140483  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 52816.8us
I20260709 13:02:03.070077  7230 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:03.107945  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:03.108449  7282 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.419s	user 0.005s	sys 0.002s
I20260709 13:02:03.108927  7228 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:03.113659  7202 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:03.114099  7193 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:03.141450  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:03.142974  2349 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
W20260709 13:02:03.147697  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 78780.8us
I20260709 13:02:03.161388  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.162057  1631 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:03.174190  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.147327  7230 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.156611  7202 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:03.181031  2348 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1364) took 1991 ms (client timeout 2109 ms). Trace:
I20260709 13:02:03.181923  2344 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:02:03.182319  2348 rpcz_store.cc:269] 0709 13:02:01.186199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.189611 (+  3412us) service_pool.cc:224] Handling call
0709 13:02:03.167731 (+1978120us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b54007701d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:03.177034 (+  9303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4477344512}
I20260709 13:02:03.152714  7236 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:02:03.164412  7096 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:03.179694  7230 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.194983  1627 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:03.202242  1636 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:02:03.197526  1702 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.706s	user 0.013s	sys 0.002s
I20260709 13:02:03.203254  1702 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:03.184047  7202 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:03.181058  7291 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:03.218482  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 166622us
W20260709 13:02:03.216149  7272 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.900s	user 0.006s	sys 0.013s
I20260709 13:02:03.223697  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.227063  1630 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
W20260709 13:02:03.229007  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59166.6us
I20260709 13:02:03.215663  7291 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:03.242818  7270 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:03.243464  7270 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:03.245033  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:03.241855  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.242384  7235 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.242898  7291 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:03.245729  7286 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.247531  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.248013  7212 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:03.248675  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.249375  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.250104  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.250242  1675 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:03.253707  7267 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:03.254832  7265 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.758s	user 0.009s	sys 0.006s
I20260709 13:02:03.257570  7102 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:03.209769  7096 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:03.227914  7272 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:03.269028  6931 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:03.269749  7193 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:03.270553  7270 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:03.271296  7128 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.235857  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:03.236208  7298 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:02:03.284689  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 143467us
I20260709 13:02:03.290180  1631 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:03.287396  1634 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
W20260709 13:02:03.295074  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.297310  7267 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:03.297750  7286 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.249238  7256 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:03.307199  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:03.311352  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.288303  7212 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.324918  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.286772  7235 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:03.327020  7128 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.328140  7286 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:03.348029  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.329989  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.333179  1629 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:03.343616  7299 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:03.344125  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 110031us
I20260709 13:02:03.345229  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.349298  1631 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:03.301433  7265 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.351569  1630 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:03.365062  7193 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:03.322118  7102 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.365713  7193 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.366248  7193 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:03.367544  7102 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:03.370323  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.373525  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:03.374325  2344 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:03.379184  7301 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:03.374143  7102 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.382547  7102 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.385283  7102 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:03.385869  7102 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.386427  7102 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.387044  7193 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:03.388830  7102 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:03.401826  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.404764  1632 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:03.406252  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:03.407594  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 382063us
W20260709 13:02:03.420280  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51140.7us
I20260709 13:02:03.423987  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.424226  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:03.424808  2344 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:03.426934  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.389422  7102 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.429766  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.430321  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.431259  1559 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:03.426378  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.459679  1554 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:03.460044  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.463229  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.461297  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.465984  1702 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:03.467702  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.469574  1706 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:03.470053  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:03.471449  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 245188us
I20260709 13:02:03.462235  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.476305  1708 raft_consensus.cc:2368] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13.
I20260709 13:02:03.477599  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.470188  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.368382  7093 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:03.492532  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 67166.2us
W20260709 13:02:03.512770  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 85509us
I20260709 13:02:03.471015  1633 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:03.521773  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.522377  1635 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.524017  7286 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 13:02:03.524456  7286 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:03.525935  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.525578  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.527145  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.527504  1701 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:03.528316  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.525543  7102 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.530827  2349 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:03.534992  1635 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:03.539003  1629 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:03.532219  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.548827  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:03.549551  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 532079us
I20260709 13:02:03.550675  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.552861  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:03.553228  1629 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:03.550849  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.554680  1633 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:03.515127  7193 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:03.560940  1635 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.565346  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:03.560972  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:03.569425  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.572396  1701 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:03.575493  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.576210  1708 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:03.572932  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.580717  1633 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:03.581871  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.582573  1629 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:03.594576  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 297401us
I20260709 13:02:03.594039  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.606324  7212 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:03.572363  7301 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:03.618800  1628 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:02:03.633929  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 383765us
I20260709 13:02:03.591962  7256 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:03.641120  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.654172  1628 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:03.646632  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55372.6us
I20260709 13:02:03.591574  7096 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:03.671541  1629 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:03.677961  7196 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:03.680538  7102 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:03.680683  7196 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.681201  7196 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.681586  1629 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:02:03.673686  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:03.690193  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 141340us
I20260709 13:02:03.632465  7301 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:03.691174  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.692181  1675 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:03.693435  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 98247.3us
I20260709 13:02:03.650434  7267 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:03.698927  1631 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:03.702581  1628 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
W20260709 13:02:03.706331  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 150178us
I20260709 13:02:03.699721  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.710309  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.711606  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.712659  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.708221  1554 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:02:03.724118  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:03.689818  7299 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:03.727747  1706 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:03.693468  7301 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:03.729426  1560 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:03.731093  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:03.731319  1557 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:03.731680  1709 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.726559  1555 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:03.734949  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:03.735328  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:03.735903  1629 raft_consensus.cc:2376] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13. Re-sending same reply.
I20260709 13:02:03.736027  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.736011  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.737023  1628 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:03.597709  7308 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.125s
I20260709 13:02:03.696067  7265 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:03.736588  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:03.737514  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.701987  7235 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:03.738732  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.745337  1635 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:03.734771  7305 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.176s
I20260709 13:02:03.752139  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:03.751644  1558 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:03.753602  7304 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.753857  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.754714  1707 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:03.754302  7304 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.757076  7304 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:03.751915  1561 raft_consensus.cc:2368] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13.
I20260709 13:02:03.757977  7316 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:03.700557  7230 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:03.760003  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 176577us
I20260709 13:02:03.760423  7256 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:03.761554  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:03.748335  1559 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:03.802057  1629 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:03.803782  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.767014  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:03.769208  7307 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:02:03.770342  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.771576  7265 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:03.770388  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:03.772698  7304 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.778880  1554 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:02:03.785249  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:03.798784  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 106714us
W20260709 13:02:03.822750  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 95118.7us
I20260709 13:02:03.752691  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.745939  1631 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:03.812098  1558 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:03.814440  1555 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:03.812865  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:03.828171  1706 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:02:03.828392  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:03.834437  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.840726  1704 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:03.840972  1631 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:03.845162  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.836117  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.851265  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:03.851437  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:03.852669  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 82420.9us
I20260709 13:02:03.853149  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:03.825663  7235 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:03.828176  7305 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:03.856593  7313 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:02:03.861317  7309 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:03.863897  7202 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.003s
I20260709 13:02:03.868904  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:03.870973  1630 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
I20260709 13:02:03.878209  1630 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:02:03.879040  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:03.875263  1702 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:03.871753  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:03.881148  1555 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:03.881701  2344 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:03.879914  1559 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:03.883155  1709 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:03.884039  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.831046  7265 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.887339  7322 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:03.888475  7265 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:03.890311  1675 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:03.890990  7230 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:03.891500  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 51931us
I20260709 13:02:03.891877  7230 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:03.892411  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:03.892546  7230 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.893805  7303 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:03.894685  7303 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:03.895043  7303 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:03.895387  7303 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:03.869958  7316 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:03.885672  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 153035us
I20260709 13:02:03.893208  7230 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:03.898087  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:03.914752  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:03.959896  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 69660us
W20260709 13:02:03.964488  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 79289us
I20260709 13:02:03.877452  7309 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:03.853935  7304 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:03.877761  7128 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:03.971657  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.916142  7322 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:03.927172  7321 raft_consensus.cc:2764] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:02:03.884684  7202 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:03.984433  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 144142us
I20260709 13:02:03.985402  1601 leader_election.cc:400] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13.
I20260709 13:02:03.985857  1601 leader_election.cc:403] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:03.969854  7318 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:03.878105  7235 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:03.990451  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:03.916699  6931 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:02:03.914507  7305 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:04.001930  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:03.970610  7309 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:03.975565  7320 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:03.969853  7193 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:02:03.992863  7265 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13.
I20260709 13:02:04.016053  7235 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:04.018378  7315 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:02:04.019943  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:03.962167  7316 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:04.027555  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 62038.5us
I20260709 13:02:04.021145  7333 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:04.030354  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 105253us
I20260709 13:02:04.044613  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:04.032610  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:04.033635  7306 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:02:04.004521  7318 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:04.035463  7093 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:04.013065  7314 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.026757  7228 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
W20260709 13:02:04.043639  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44851.9us
W20260709 13:02:04.073994  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 181716us
W20260709 13:02:04.074266  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 63024us
I20260709 13:02:04.076990  1702 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:02:04.081666  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61824.9us
I20260709 13:02:04.083271  7235 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:04.085270  7256 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:04.086231  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:04.087359  7235 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:04.087785  7324 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:02:04.088960  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:04.090312  7235 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:04.030896  7196 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:04.090950  7318 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:04.076606  7265 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:04.092232  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:04.093093  7265 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:04.093587  1707 raft_consensus.cc:2368] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13.
W20260709 13:02:04.093662  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61170.2us
W20260709 13:02:04.096900  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47169.4us
I20260709 13:02:04.101359  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:04.096256  7291 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.794s	user 0.014s	sys 0.008s
W20260709 13:02:04.086259  7323 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:02:04.110327  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:04.110746  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:04.113521  1707 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:04.119109  7316 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:04.123610  7317 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:04.124217  7317 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:04.103935  7193 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:04.125851  7193 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:04.126924  1527 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:04.111765  7291 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:04.128326  1632 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:04.129386  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:04.130273  2349 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:04.131561  7291 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:04.132156  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:04.105645  7306 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:02:04.132979  2346 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 11.
I20260709 13:02:04.134384  7332 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:04.135299  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:04.095145  7322 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:04.136204  2348 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:04.126961  7193 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:04.138321  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:04.125401  7317 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:04.141245  7317 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:04.142664  7320 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:04.143289  7320 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:04.145203  7193 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.101415  7321 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:04.146010  7196 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:04.152733  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:04.153774  7093 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:04.153959  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:04.154367  7093 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:04.155726  7302 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.476s	user 0.003s	sys 0.000s
I20260709 13:02:04.155438  7093 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:04.158320  7331 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:02:04.155915  7335 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.165125  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:04.164115  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:04.166823  1701 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:04.166019  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2884) took 1557 ms. Trace:
I20260709 13:02:04.166293  7291 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:04.169734  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:04.170459  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 72169.3us
W20260709 13:02:04.170960  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 58639.2us
W20260709 13:02:04.172787  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 66603.7us
I20260709 13:02:04.173702  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:04.176170  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 422904us
I20260709 13:02:04.173571  1602 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:04.176973  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50156.5us
I20260709 13:02:04.177630  1602 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:04.168342  1710 rpcz_store.cc:276] 0709 13:02:02.608633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.684390 (+ 75757us) service_pool.cc:224] Handling call
0709 13:02:02.763935 (+ 79545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.009584 (+245649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.009632 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.029193 (+ 19561us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.330606 (+301413us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:03.330641 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.330704 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.154766 (+824062us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:04.160370 (+  5604us) raft_consensus.cc:1685] finished
0709 13:02:04.160409 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.165918 (+  5509us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31613056,"thread_start_us":260345,"threads_started":1,"wal-append.queue_time_us":507795}
I20260709 13:02:04.179852  7235 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:04.180413  7235 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.180787  7235 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:04.181962  7306 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:04.182364  7306 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:04.183370  7306 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:04.179996  7330 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:04.122781  7305 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:04.188783  1701 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:04.192104  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:04.193333  7096 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:04.189667  7291 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:04.113835  7202 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:04.205799  2346 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:04.204326  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:04.200628  6931 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:04.209312  7272 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:04.209510  7339 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:04.211414  6931 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:04.211879  6931 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:04.212080  7339 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:04.213048  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:04.213672  7305 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:04.198254  7321 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:04.186674  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:04.221297  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47668.5us
W20260709 13:02:04.206140  7096 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:04.224599  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:04.218583  1602 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:04.216310  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 141397us
I20260709 13:02:04.255750  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:04.256181  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 47163.6us
I20260709 13:02:04.215188  7291 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:04.261196  1675 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:04.255347  7265 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:04.222685  7306 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:04.190763  7230 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:04.266642  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 53710.7us
I20260709 13:02:04.222965  7304 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:04.278285  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 56327.4us
I20260709 13:02:04.254187  7202 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:04.175191  7335 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:04.293195  1701 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:02:04.265735  7299 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:04.273231  7265 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:04.278605  7196 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:04.296713  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 201923us
W20260709 13:02:04.341858  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 86244us
I20260709 13:02:04.343725  1704 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:02:04.262910  7321 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:04.352066  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:04.370150  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 80731.2us
I20260709 13:02:04.341359  7265 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:04.381040  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:04.381369  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:04.358996  7299 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:04.342650  7196 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:04.288822  6932 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:02:04.397882  1632 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:04.397980  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:04.407270  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 45049.9us
I20260709 13:02:04.407482  1628 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:02:04.295805  7304 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:04.399811  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 133471us
W20260709 13:02:04.401594  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.393072  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:04.398439  1635 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:04.289070  6571 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.103s
W20260709 13:02:04.409174  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 53813us
I20260709 13:02:04.399399  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:04.416735  1627 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.421388  1629 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:02:04.423372  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 64517.2us
I20260709 13:02:04.427750  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:04.428506  1632 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:04.430557  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 135584us
I20260709 13:02:04.405975  7196 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:04.411901  7096 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:04.408630  7188 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:04.420056  7304 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:04.487087  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:04.437062  7340 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:04.456480  7093 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:04.502669  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.458590  7096 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:04.512876  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 93602.8us
W20260709 13:02:04.495020  7324 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:02:04.519155  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:04.530678  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:04.525756  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 67828.4us
I20260709 13:02:04.375000  7339 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:04.479338  7272 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:02:04.516105  7096 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:04.418408  7291 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:04.533746  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 160932us
W20260709 13:02:04.525503  7301 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.767s	user 0.017s	sys 0.001s
W20260709 13:02:04.548272  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 90884us
W20260709 13:02:04.593278  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 161071us
I20260709 13:02:04.519496  7350 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:04.603847  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:04.602696  1527 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:04.615260  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 154030us
I20260709 13:02:04.616904  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:04.620126  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 87795us
I20260709 13:02:04.627238  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:04.632550  1560 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:04.627286  7256 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:04.636005  7256 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:04.636539  7256 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:04.627432  7340 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.638448  7340 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:04.648473  7340 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:04.625962  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:04.656888  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:04.660398  7340 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:04.662634  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:04.669800  7340 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:04.671181  7340 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:04.672660  7340 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:04.612309  7312 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.084s	user 0.003s	sys 0.007s
I20260709 13:02:04.614192  7188 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:04.689963  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 74891.3us
I20260709 13:02:04.522716  7093 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:04.599196  7230 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:04.613735  7303 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.718s	user 0.008s	sys 0.012s
I20260709 13:02:04.706980  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:04.707723  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 193996us
I20260709 13:02:04.597683  7301 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:04.687995  7202 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:02:04.710218  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47717.7us
W20260709 13:02:04.715847  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.736060  1601 leader_election.cc:400] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13.
I20260709 13:02:04.692905  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:04.726941  7343 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:02:04.718724  7256 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:04.727566  7346 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 13:02:04.730826  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:04.719143  1527 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:04.694032  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:04.738341  1555 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:04.742784  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:04.743161  7358 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:04.750388  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:04.757011  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:04.757413  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 140703us
I20260709 13:02:04.759433  2318 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:04.767584  1557 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:04.761022  1561 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:04.683734  7357 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:04.771513  2317 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:04.704840  7335 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:04.758635  7358 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:04.573395  7299 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:04.781322  7357 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:04.750808  7272 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:02:04.782730  7202 raft_consensus.cc:697] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:04.787236  7332 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:04.855087  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.806913  7333 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:04.747777  7303 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:04.811038  7358 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:04.870374  7361 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:04.871224  7361 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.871759  7361 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:04.877652  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 127427us
W20260709 13:02:04.878229  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 195641us
I20260709 13:02:04.860309  7304 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:04.840020  7357 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.880132  1632 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:04.899873  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:04.904456  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 147479us
W20260709 13:02:04.869365  7309 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.841s	user 0.013s	sys 0.002s
I20260709 13:02:04.917682  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:04.873713  7333 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:04.941776  7355 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:04.942379  7355 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:04.942912  7355 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.868108  7188 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:04.941833  7350 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:04.933856  7309 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:04.956519  1627 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.536s	user 0.007s	sys 0.006s
I20260709 13:02:05.013759  1627 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
W20260709 13:02:04.950713  6931 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.695s	user 0.013s	sys 0.001s
I20260709 13:02:04.958326  7267 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:02:05.032609  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:05.112526  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 213020us
I20260709 13:02:05.113761  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:05.112700  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 181406us
I20260709 13:02:04.938122  7353 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:04.945679  7360 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.146730  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:05.147641  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 388256us
I20260709 13:02:05.149767  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:05.064740  7267 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:04.950680  7363 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.336s
I20260709 13:02:05.139621  7353 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.079958  7377 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:05.139936  7272 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:05.146490  7340 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:05.143155  7093 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:05.172819  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 254359us
I20260709 13:02:05.178256  1706 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:05.176782  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:05.160640  6932 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:05.157435  7350 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.236380  1701 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:05.204309  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:05.160333  7301 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.168231  7353 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:05.172534  7377 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.203924  7188 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 13:02:05.176800  7193 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.006s	sys 0.028s
W20260709 13:02:05.173650  7317 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.030s	user 0.014s	sys 0.000s
I20260709 13:02:05.085870  6931 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.172853  7378 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:05.231562  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:05.173039  7340 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:05.228271  7093 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:05.283883  1707 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:05.226337  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:05.247493  1557 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:05.173169  7272 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:05.245988  7354 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.339s	user 0.002s	sys 0.000s
I20260709 13:02:05.248734  7377 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:05.249351  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:05.318725  1555 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
W20260709 13:02:05.250344  7235 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.069s	user 0.009s	sys 0.006s
I20260709 13:02:05.242870  7308 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:02:05.250749  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:05.251616  7301 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:05.252018  7372 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:02:05.252326  7345 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.341s	user 0.000s	sys 0.002s
W20260709 13:02:05.252421  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 83433.9us
I20260709 13:02:05.325006  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:05.244819  2318 connection.cc:256] RPC call timeout handler was delayed by -1.430738980000001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
W20260709 13:02:05.253376  7367 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
W20260709 13:02:05.257916  7366 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.393s	user 0.000s	sys 0.001s
I20260709 13:02:05.111304  7202 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:05.276337  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.002s	sys 0.004s
I20260709 13:02:05.280507  7303 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:05.330106  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.002s	sys 0.005s
I20260709 13:02:05.154994  7360 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:05.248188  7265 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.829s	user 0.013s	sys 0.000s
I20260709 13:02:05.315969  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:05.315538  2317 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.003s	sys 0.033s
W20260709 13:02:05.274318  7193 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.006s	sys 0.028s
I20260709 13:02:05.245589  6932 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.279515  7374 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:05.287065  7340 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:05.332775  2317 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.004s	sys 0.033s
I20260709 13:02:05.317140  7272 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.283044  7378 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:05.326990  7320 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.181s	user 0.015s	sys 0.000s
I20260709 13:02:05.321373  7384 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:05.362264  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 590846us
I20260709 13:02:05.363745  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:05.324118  7301 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:05.369745  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:05.430619  1701 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:05.371662  1706 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:05.331588  7363 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:05.435124  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51881.6us
I20260709 13:02:05.358451  7374 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:05.387137  7390 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:05.387638  7375 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:05.388075  1709 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:05.437624  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:05.388494  1554 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:05.388942  7371 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:05.447261  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3042) took 1067 ms. Trace:
I20260709 13:02:05.389083  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:05.389562  7316 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.392086  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:05.393189  1705 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:02:05.394971  1710 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:05.321017  7093 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.373587  7357 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.411312  7320 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.326473  7235 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.412725  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:05.436054  7301 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.268864  7333 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.506767  7377 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.423374  1555 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:05.510257  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.511647  2352 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:02:05.436122  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:05.513536  1527 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:05.514935  7305 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.515661  2353 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:05.359253  6932 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.436889  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3044) took 1056 ms. Trace:
I20260709 13:02:05.518016  1633 rpcz_store.cc:276] 0709 13:02:04.380948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.398651 (+ 17703us) service_pool.cc:224] Handling call
0709 13:02:04.398990 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.429450 (+ 30460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.429541 (+    91us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.511136 (+ 81595us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.807742 (+296606us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:04.807809 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.807910 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.436682 (+628772us) raft_consensus.cc:1685] finished
0709 13:02:05.436705 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.436827 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8791,"raft.queue_time_us":15597,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4588,"thread_start_us":262716,"threads_started":1,"wal-append.queue_time_us":351425}
I20260709 13:02:05.369809  7309 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.451017  1708 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:05.440963  7314 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.364s	user 0.016s	sys 0.000s
I20260709 13:02:05.520781  7309 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:05.521358  7314 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.455137  1675 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:05.521569  7309 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.447712  1636 rpcz_store.cc:276] 0709 13:02:04.379825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.380076 (+   251us) service_pool.cc:224] Handling call
0709 13:02:04.397826 (+ 17750us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:04.415187 (+ 17361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.415237 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:04.467456 (+ 52219us) log.cc:844] Serialized 52 byte log entry
0709 13:02:04.613195 (+145739us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:04.613245 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.613332 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.446998 (+833666us) raft_consensus.cc:1685] finished
0709 13:02:05.447021 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.447195 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16818,"raft.queue_time_us":17704,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":18484,"thread_start_us":99261,"threads_started":1,"wal-append.queue_time_us":277362}
W20260709 13:02:05.453539  1602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.014s	sys 0.060s
I20260709 13:02:05.354712  7265 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.488991  7379 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:05.488042  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:05.491564  7196 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.022s	user 0.012s	sys 0.000s
I20260709 13:02:05.471189  7375 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:05.495088  7093 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.488548  7374 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:05.442750  7384 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.504479  7308 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.505227  7299 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:05.504745  7378 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.506038  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.507278  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.361264  7291 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.507632  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.508633  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.556866  2344 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:05.523526  1602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.016s	sys 0.061s
W20260709 13:02:05.528839  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 196029us
I20260709 13:02:05.513171  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.450372  7363 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.527848  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.454164  7371 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.509388  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:05.468309  7316 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.534236  7379 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.588842  2351 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:05.609892  7316 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:05.555478  7228 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:05.591166  7291 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.611174  7379 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:05.611826  7378 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:05.612792  7309 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.613760  6931 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:05.517248  7377 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:05.615425  7375 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.618960  7235 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:05.619757  7265 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:05.623250  7348 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.133s	user 0.010s	sys 0.004s
I20260709 13:02:05.623620  7371 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:05.624630  7377 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.555387  7308 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.626408  7363 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.592006  2353 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:05.450526  7362 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:05.627931  7362 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.628108  7308 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.628381  7362 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:05.630069  7384 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.630761  7360 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:05.450259  7390 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:05.635380  7096 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.027s	user 0.010s	sys 0.005s
W20260709 13:02:05.636046  7358 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.734s	user 0.009s	sys 0.002s
I20260709 13:02:05.636567  7358 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.637020  7374 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:05.640537  7256 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:05.590668  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.555202  7398 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.596616  7382 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:05.653721  7272 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.597302  2347 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:02:05.558077  7299 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.657791  7272 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.658421  7299 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.663074  1701 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:05.653738  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 62453.2us
I20260709 13:02:05.666011  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:05.667366  2351 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:05.668586  7358 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.669622  7362 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:05.673216  7362 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:05.603019  7394 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.603253  7353 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.604935  7357 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:05.605307  7355 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.662s	user 0.010s	sys 0.004s
I20260709 13:02:05.534861  7305 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:02:05.534242  7388 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.138s
I20260709 13:02:05.609944  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3053) took 1214 ms. Trace:
I20260709 13:02:05.556437  2349 raft_consensus.cc:2368] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:02:05.541996  7196 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.692396  7394 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.625455  7377 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.701062  7394 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:05.589643  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 102012us
I20260709 13:02:05.706305  7196 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:05.707285  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:05.708206  7314 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:05.708737  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:05.591385  2352 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:02:05.709694  7314 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.710129  7320 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:05.711858  7340 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.645153  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:05.713845  1636 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:05.715121  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:05.715785  1636 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:05.719026  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:05.719954  7314 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:05.646411  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:05.613077  7291 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.721974  1632 raft_consensus.cc:2376] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15. Re-sending same reply.
I20260709 13:02:05.723027  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:05.723744  1632 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:05.648104  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:05.648610  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 58585.2us
W20260709 13:02:05.648762  7368 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.736s	user 0.001s	sys 0.001s
I20260709 13:02:05.649063  1629 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:05.649114  7390 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:05.650085  7344 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.253s	user 0.010s	sys 0.003s
I20260709 13:02:05.652622  7267 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.654043  7382 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:05.674268  1631 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:02:05.602439  7397 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:02:05.676501  7361 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.804s	user 0.000s	sys 0.015s
I20260709 13:02:05.678356  1628 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:05.600368  7306 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.315s	user 0.015s	sys 0.000s
I20260709 13:02:05.636397  7096 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:05.686380  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:05.696691  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 260649us
I20260709 13:02:05.699265  1704 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:02:05.595387  2345 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:05.694360  7303 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:05.702486  7305 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:05.759927  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2944) took 1134 ms. Trace:
I20260709 13:02:05.694268  7355 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.760478  7340 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:05.760370  1556 rpcz_store.cc:276] 0709 13:02:04.625936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.626148 (+   212us) service_pool.cc:224] Handling call
0709 13:02:04.626623 (+   475us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:04.627672 (+  1049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.627706 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:04.628605 (+   899us) log.cc:844] Serialized 52 byte log entry
0709 13:02:04.754165 (+125560us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:04.754206 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.754302 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.647986 (+893684us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:05.759606 (+111620us) raft_consensus.cc:1685] finished
0709 13:02:05.759628 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.759861 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":449,"raft.queue_time_us":2267,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":191,"spinlock_wait_cycles":29958912,"thread_start_us":125021,"threads_started":1,"wal-append.queue_time_us":235014}
I20260709 13:02:05.762317  7355 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.694913  1630 rpcz_store.cc:276] 0709 13:02:04.395649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.400597 (+  4948us) service_pool.cc:224] Handling call
0709 13:02:04.400918 (+   321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:04.957756 (+556838us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b18001c15c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda276d2060 00007fda276d2bfb 00007fda276dd64b
0709 13:02:04.957980 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.958010 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:04.958840 (+   830us) log.cc:844] Serialized 52 byte log entry
0709 13:02:05.361664 (+402824us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:05.361720 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.361817 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.609569 (+247752us) raft_consensus.cc:1685] finished
0709 13:02:05.609605 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.609873 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14693,"raft.queue_time_us":33598,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":13058,"spinlock_wait_cycles":24261760,"thread_start_us":422314,"threads_started":2,"wal-append.queue_time_us":572785}
I20260709 13:02:05.764772  1701 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:02:05.741950  7382 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.720431  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:05.721041  7394 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.645241  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:05.769462  1627 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:05.721268  1552 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:05.726405  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:05.729378  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 274333us
I20260709 13:02:05.735301  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:05.745805  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:05.773540  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:05.755426  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:05.757143  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:05.759199  1705 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:05.736339  7093 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.736239  7350 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:05.777710  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 57364.4us
I20260709 13:02:05.791289  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:05.778173  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 51835.8us
I20260709 13:02:05.780591  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:05.781461  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.783100  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:05.792939  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:05.794196  2351 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:05.784965  7388 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:05.789652  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:05.785002  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:05.789124  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2888) took 1626 ms. Trace:
I20260709 13:02:05.793717  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:05.795486  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:05.809753  1527 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:05.804032  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:05.760216  7305 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:05.808120  1702 rpcz_store.cc:276] 0709 13:02:04.163064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.163261 (+   197us) service_pool.cc:224] Handling call
0709 13:02:04.163650 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.192396 (+ 28746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.192430 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.193031 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.274138 (+ 81107us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:04.274190 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.274294 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.649368 (+1375074us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:05.766673 (+117305us) raft_consensus.cc:1685] finished
0709 13:02:05.766709 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.789018 (+ 22309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26047,"raft.queue_time_us":29722,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"spinlock_wait_cycles":29738880,"thread_start_us":80654,"threads_started":1,"wal-append.queue_time_us":203188}
I20260709 13:02:05.830333  1527 leader_election.cc:400] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:02:05.830857  1527 leader_election.cc:403] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:05.745970  7339 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:05.836076  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:05.836984  1632 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:05.837940  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:05.838733  1629 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:05.834156  1676 connection.cc:256] RPC call timeout handler was delayed by -1.1027112429999875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16410000000000002s for a response
W20260709 13:02:05.840018  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44598.9us
I20260709 13:02:05.840747  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:05.842425  1634 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:02:05.777330  7188 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.844491  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:05.846756  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:05.850325  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:05.850594  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:05.805811  7355 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:05.852670  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:05.804646  7314 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.854491  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:05.814108  7388 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:05.818699  7267 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.813398  7304 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:05.857132  1633 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:05.859506  1635 raft_consensus.cc:2410] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:02:05.860101  1601 leader_election.cc:400] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13.
I20260709 13:02:05.880741  7301 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.880213  7388 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.878134  1676 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:05.832180  7375 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:05.882616  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:05.867924  2348 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:02:05.881731  7291 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:02:05.896847  7375 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:05.896988  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.852221  7308 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:05.897397  7375 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.856652  7309 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:02:05.898528  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:05.813194  7350 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.899756  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.860836  7339 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.900223  7386 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:05.900489  2348 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:05.900770  7386 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.869632  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.901432  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:05.902209  7339 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.813890  7093 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.903386  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.804450  7303 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.851715  7188 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:05.905731  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:05.806064  7340 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:02:05.906437  2350 raft_consensus.cc:2376] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12. Re-sending same reply.
I20260709 13:02:05.907585  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:05.907250  7305 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.908209  2350 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:02:05.908635  7305 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:05.909096  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:05.911094  2348 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:05.870397  7364 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.874346  7406 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.755939  7361 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.847546  6932 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.876688  7096 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.876487  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:06.056890  2347 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:05.821039  7256 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:05.888584  2351 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:05.891501  1601 leader_election.cc:403] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:02:05.842533  7357 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:05.897537  7291 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:05.901212  7386 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:05.906507  7303 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.873677  7356 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:06.081728  7356 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.082149  7356 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.086052  7389 raft_consensus.cc:2764] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:02:06.087209  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2231) took 1937 ms (client timeout 1773 ms). Trace:
W20260709 13:02:06.087693  1707 rpcz_store.cc:269] 0709 13:02:04.150053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.161680 (+ 11627us) service_pool.cc:224] Handling call
0709 13:02:06.086927 (+1925247us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002a67d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:06.087078 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2377844096}
I20260709 13:02:05.861615  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:05.982762  7406 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:06.089737  1632 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:02:06.090823  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:06.091526  7406 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:06.091931  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:06.092643  1632 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:05.869627  7402 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 13:02:06.098373  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:06.099412  2347 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:02:05.899772  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 67014.8us
I20260709 13:02:05.982854  7364 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:06.101969  2317 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:05.920907  2349 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:05.855001  7333 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:02:06.102932  7306 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:06.103745  7364 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:05.983294  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 105265us
I20260709 13:02:06.104501  7364 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:06.104964  7364 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:06.105041  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:06.106228  7318 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:05.901854  7350 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:06.030651  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 148111us
I20260709 13:02:05.911252  7093 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:05.983649  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:06.109928  2346 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:06.033996  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 199967us
I20260709 13:02:05.901744  7370 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:06.036083  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:06.114867  1629 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:06.044920  1636 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:06.116765  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:06.030795  7339 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:05.982607  7416 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:06.053419  2345 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:06.031307  7362 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.933594  7202 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.117s
I20260709 13:02:06.024432  7404 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.133138  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:06.133968  1629 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:06.135054  7389 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:06.136458  7333 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:06.137055  7333 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:06.138340  7333 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:05.985188  7382 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:06.078581  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.143188  7355 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.143774  7355 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:06.079181  7357 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:06.152643  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:06.153867  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:05.914781  1557 pending_rounds.cc:85] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 5
I20260709 13:02:06.154544  1557 pending_rounds.cc:107] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:02:06.154659  1709 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:06.127022  7128 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:02:06.113511  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:06.118834  1636 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:06.156294  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:06.157440  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:06.157655  7362 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.160818  1559 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:06.059662  7256 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.161732  7256 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:06.128086  2350 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:06.133469  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:06.066295  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 209909us
I20260709 13:02:06.075044  7314 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13.
I20260709 13:02:06.065896  7291 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:06.147730  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:06.149089  7301 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:06.151186  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:06.152140  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:06.152823  1701 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:06.181310  1706 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:06.080435  7188 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:06.153611  7357 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:06.081118  6932 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:06.154879  1557 raft_consensus.cc:2889] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:06.162916  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:06.185580  1629 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:06.130146  7402 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.152168  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:06.186383  2346 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:06.110647  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:06.165740  1525 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:06.170229  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44590.1us
I20260709 13:02:06.170739  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:06.171237  1636 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:06.177210  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:06.150259  7355 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:06.189069  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 160723us
I20260709 13:02:06.192014  1702 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:06.194900  2347 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:06.198800  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:06.199695  1636 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:06.177493  7339 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:06.207332  7303 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:06.208436  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:06.124306  7370 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:06.166543  7403 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:02:06.208909  7362 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:06.459362  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 293711us
I20260709 13:02:06.454509  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:06.217351  1632 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:06.227445  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:06.454533  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:06.471580  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:06.472369  1634 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.308157  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:06.473345  2349 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:02:06.461623  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.470639  1636 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:06.466336  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:06.470458  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:06.301226  7420 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:02:06.194265  7096 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:02:06.301594  7342 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.866s	user 0.008s	sys 0.004s
I20260709 13:02:06.503185  7093 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:06.518712  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 688393us
W20260709 13:02:06.519898  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 409288us
I20260709 13:02:06.495599  7389 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.526592  7093 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:06.523087  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:06.527899  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 336227us
I20260709 13:02:06.227038  7128 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.287s
I20260709 13:02:06.505466  7362 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:06.268101  7402 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.249s
I20260709 13:02:06.504776  7350 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:06.532323  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:06.533193  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:06.538229  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:06.539530  1629 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:06.544647  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:06.548640  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:06.549338  1553 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 9.
I20260709 13:02:06.209887  7364 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:06.553716  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:06.547945  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:06.556531  7430 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:06.556689  7402 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:02:06.562311  7411 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.364s	user 0.001s	sys 0.000s
I20260709 13:02:06.563040  7093 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:06.547775  7375 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.650s	user 0.013s	sys 0.000s
I20260709 13:02:06.563899  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:06.563545  1552 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:06.601203  7375 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.603534  7375 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:06.604739  7267 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.709s	user 0.011s	sys 0.000s
I20260709 13:02:06.570256  1561 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:06.570838  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:06.622246  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:06.533963  7332 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:06.556941  7422 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 13:02:06.265309  7291 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:06.563524  7415 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.377s	user 0.000s	sys 0.001s
I20260709 13:02:06.580194  1632 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:06.537657  7421 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:06.580798  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.582548  2350 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:02:06.539049  7350 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:06.575264  7402 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:06.571089  7406 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:06.570725  7128 consensus_queue.cc:1243] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:06.591264  7361 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:06.660194  7421 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:06.663292  7356 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.581s	user 0.013s	sys 0.001s
I20260709 13:02:06.662726  7402 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.663812  7356 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.664888  7350 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.666214  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:06.667176  2348 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:06.667591  2348 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:06.667855  7421 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:06.668478  2348 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:06.597234  7424 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.364s	user 0.002s	sys 0.000s
I20260709 13:02:06.605284  7430 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:06.670512  7430 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:02:06.600220  7386 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.004s
I20260709 13:02:06.670977  7430 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:06.671240  7386 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.672144  7430 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.610255  7364 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:06.624794  7333 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:06.596549  7305 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.687s	user 0.014s	sys 0.000s
I20260709 13:02:06.677587  2353 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:06.611173  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:06.678784  1627 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:02:06.610733  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2895) took 1954 ms (client timeout 1528 ms). Trace:
W20260709 13:02:06.615787  1676 leader_election.cc:336] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.645s (SENT)
W20260709 13:02:06.680539  1704 rpcz_store.cc:269] 0709 13:02:04.656212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.656460 (+   248us) service_pool.cc:224] Handling call
0709 13:02:06.610477 (+1954017us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002e95d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:06.610636 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2392784128}
W20260709 13:02:06.681352  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 65631.7us
I20260709 13:02:06.681839  7382 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:06.553944  7388 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.647s	user 0.008s	sys 0.004s
I20260709 13:02:06.683250  1708 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:06.682961  7388 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.541803  7362 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:06.686390  7362 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.617822  1674 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:06.688710  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 71047.9us
I20260709 13:02:06.689678  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:06.625835  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55081us
I20260709 13:02:06.691298  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:06.576160  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:06.631897  1631 raft_consensus.cc:2410] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:02:06.694242  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 118188us
I20260709 13:02:06.626932  1629 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:06.658995  7428 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.659217  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:06.625007  7360 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.627369  7332 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:06.666424  7350 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:06.564348  7256 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.680030  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:06.722337  1627 raft_consensus.cc:2376] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13. Re-sending same reply.
I20260709 13:02:06.618094  1706 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:06.704066  1634 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:06.706723  2349 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:06.714370  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:06.728166  1556 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:06.728422  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:06.734179  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:06.654649  7267 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.660318  7406 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.735075  2352 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:06.741245  2350 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:06.742619  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:06.743055  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:06.743486  1630 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:06.743691  2347 raft_consensus.cc:2376] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13. Re-sending same reply.
I20260709 13:02:06.697849  7357 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:06.738516  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:06.705415  7128 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.717382  7291 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:06.698505  7388 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:06.755175  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:06.754927  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 258605us
I20260709 13:02:06.746932  7374 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:06.727872  7314 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:06.738245  7355 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:06.736927  7321 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:06.763127  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:06.759295  1629 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:06.714156  7361 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:02:06.749174  7406 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:06.772459  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:06.760843  7374 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:06.757419  7430 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:06.728272  7360 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:06.769652  7357 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.794668  7301 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:06.769217  7386 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:06.796867  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:06.764799  7305 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:02:06.805724  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:06.776990  7096 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:06.780038  7374 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.813862  1706 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:06.750092  7267 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.813239  2344 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:06.807569  7356 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:06.798139  7421 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:06.809955  7304 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:06.856292  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:06.860211  1560 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.798478  7361 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
W20260709 13:02:06.856689  1703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.013s	sys 0.011s
W20260709 13:02:06.873353  1703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.013s	sys 0.012s
I20260709 13:02:06.811173  7357 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:06.810854  7332 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:06.854161  7428 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:06.854488  7388 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:02:06.853788  7310 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:02:06.837977  7096 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:06.910816  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 221215us
I20260709 13:02:06.857651  7128 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:02:06.890226  7428 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:06.915028  7301 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.871547  7304 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:06.861459  7421 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:06.958986  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:06.965605  1559 raft_consensus.cc:2410] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
W20260709 13:02:06.978578  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.958058  7428 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:07.005815  7442 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:02:07.006424  7435 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:02:06.960456  7193 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:07.010421  1602 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:06.995486  7188 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:07.024431  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64779.4us
I20260709 13:02:07.030758  7357 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:07.031538  7357 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:07.029264  7267 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:07.032159  7357 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.033027  7267 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:07.045872  7267 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:07.047161  7267 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.010737  7442 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:07.034309  7432 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:02:06.986948  7301 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.055830  7442 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.037283  7374 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:07.059051  7421 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:07.062003  7304 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:07.016863  7193 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.061004  7357 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:07.064787  7193 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:07.066213  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 107351us
I20260709 13:02:07.070500  7096 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:07.024325  7430 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:07.132794  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.071303  7096 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 13:02:07.177668  7301 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:07.179430  7434 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:02:07.146106  7429 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:07.182644  7429 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.184005  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 949) took 1033 ms. Trace:
I20260709 13:02:07.184649  1703 rpcz_store.cc:276] 0709 13:02:06.150434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.151186 (+   752us) service_pool.cc:224] Handling call
0709 13:02:06.151534 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:06.179055 (+ 27521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:06.179100 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:06.182527 (+  3427us) log.cc:844] Serialized 26 byte log entry
0709 13:02:06.874370 (+691843us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:06.874445 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.874550 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:07.183671 (+309121us) raft_consensus.cc:1685] finished
0709 13:02:07.183705 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.183918 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32015,"raft.queue_time_us":27927,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":12068,"thread_start_us":649548,"threads_started":1,"wal-append.queue_time_us":715977}
I20260709 13:02:07.187076  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:07.187642  7416 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.027s	user 0.014s	sys 0.000s
I20260709 13:02:07.187865  1555 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:07.188094  7416 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:02:07.074958  7442 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:07.213469  7096 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.065235  7357 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.218520  7096 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:07.156111  7304 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.223646  1560 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:07.155072  7430 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:07.228628  7430 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.229368  7377 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:07.217566  7443 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.233493  7406 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:07.235212  7407 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.181s	user 0.006s	sys 0.010s
I20260709 13:02:07.233520  7443 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.238202  7304 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.244381  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2949) took 1523 ms. Trace:
I20260709 13:02:07.241287  7448 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:07.246220  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2235) took 2068 ms (client timeout 1760 ms). Trace:
W20260709 13:02:07.246711  1710 rpcz_store.cc:269] 0709 13:02:05.202326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.216450 (+ 14124us) service_pool.cc:224] Handling call
0709 13:02:07.246002 (+2029552us) spinlock_profiling.cc:243] Waited 83 ms on lock 0x7b540014d2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:07.246134 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190987008}
I20260709 13:02:07.247622  7428 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:07.250212  7350 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.523s	user 0.013s	sys 0.000s
I20260709 13:02:07.251123  7428 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:07.252434  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:07.253265  1560 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:07.249883  7448 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.258924  7429 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:07.265926  7399 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:07.266867  7399 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.250799  1558 rpcz_store.cc:276] 0709 13:02:05.721144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.723412 (+  2268us) service_pool.cc:224] Handling call
0709 13:02:05.723851 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.724094 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:05.818840 (+ 94746us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.243403 (+1424563us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b18003dbb80. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287
0709 13:02:07.243789 (+   386us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.243990 (+   201us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:07.244039 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:07.244098 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.244176 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.244314 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10205,"prepare.queue_time_us":260862,"raft.queue_time_us":85105,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":62736,"spinlock_wait_cycles":64922752,"thread_start_us":376554,"threads_started":3,"wal-append.queue_time_us":309663}
I20260709 13:02:07.281230  1560 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 10.
I20260709 13:02:07.296564  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.297343  1628 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:07.298384  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.298942  1628 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:07.299765  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.300514  1628 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:02:07.306737  1554 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:07.301360  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.312593  1628 raft_consensus.cc:2410] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:02:07.316532  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.317689  1628 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:07.319113  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.320665  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.321848  1628 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:07.323019  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.323921  1628 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:07.324898  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:07.327481  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.325807  1628 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:07.330369  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.330897  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.333287  7443 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:07.336369  7304 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:07.337378  7406 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:07.226759  7357 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.332538  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:07.340144  7355 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.560s	user 0.011s	sys 0.002s
I20260709 13:02:07.340531  1628 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:07.340793  7355 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:07.341693  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.342338  1628 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:07.342978  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.326079  7399 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:07.359956  1704 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:07.329749  7096 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:07.374864  1703 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:07.371819  1628 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:07.370682  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.380512  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.381292  1710 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:02:07.383248  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.383570  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.385064  1628 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:07.386549  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.387846  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.388864  1628 raft_consensus.cc:2410] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:02:07.390291  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.390916  1628 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:07.391736  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.393131  1628 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:07.393735  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2795) took 1521 ms. Trace:
I20260709 13:02:07.394032  1628 rpcz_store.cc:276] 0709 13:02:05.872473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.392479 (+1520006us) service_pool.cc:224] Handling call
0709 13:02:07.392813 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.393548 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.393693 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:07.395190  1628 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:07.395682  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2796) took 1523 ms. Trace:
I20260709 13:02:06.056375   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:07.395936  1628 rpcz_store.cc:276] 0709 13:02:05.872737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.394844 (+1522107us) service_pool.cc:224] Handling call
0709 13:02:07.395060 (+   216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.395515 (+   455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.395642 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:07.405948  7358 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:07.406514  7358 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:07.409978  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.411558  1708 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:07.411458  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.413928  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.415654  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.417296  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.372694  7355 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:07.426119  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:07.427175  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3067) took 1277 ms (client timeout 1539 ms). Trace:
I20260709 13:02:07.398039  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2799) took 1524 ms. Trace:
I20260709 13:02:07.428402  1629 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:07.424842  7357 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:07.427628  1632 rpcz_store.cc:269] 0709 13:02:06.149796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.157191 (+  7395us) service_pool.cc:224] Handling call
0709 13:02:07.426843 (+1269652us) spinlock_profiling.cc:243] Waited 59.3 ms on lock 0x7b540017b954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:07.427107 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136294784}
I20260709 13:02:07.428629  1628 rpcz_store.cc:276] 0709 13:02:05.876204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.396840 (+1520636us) service_pool.cc:224] Handling call
0709 13:02:07.397168 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.397402 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:07.397443 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:07.397518 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:07.397552 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.397757 (+   205us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.397866 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:07.431277  1703 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:07.431551  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.432227  1706 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:07.407507  7358 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.433923  7358 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:07.452176  7256 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:07.454823  7403 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:07.455485  7403 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:07.492110  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.497613  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.499789  1628 raft_consensus.cc:2376] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15. Re-sending same reply.
W20260709 13:02:07.495934  7412 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.462s	user 0.010s	sys 0.024s
I20260709 13:02:07.509158  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.514010  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3061) took 1370 ms. Trace:
I20260709 13:02:07.514671  1628 rpcz_store.cc:276] 0709 13:02:06.143888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.506193 (+1362305us) service_pool.cc:224] Handling call
0709 13:02:07.513098 (+  6905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.513368 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:07.513548 (+   180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:07.513623 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:07.513680 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.513765 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.513926 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:07.497582  7357 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:07.517133  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.432183  7355 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:07.504282  7403 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.521600  1628 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:07.522431  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.522521  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3062) took 1378 ms. Trace:
I20260709 13:02:07.510694  7353 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:07.523047  1628 rpcz_store.cc:276] 0709 13:02:06.144206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.516062 (+1371856us) service_pool.cc:224] Handling call
0709 13:02:07.520835 (+  4773us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.522221 (+  1386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.522453 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:07.525183  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 128509us
I20260709 13:02:07.530278  1628 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:07.531148  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.531002  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3063) took 1386 ms. Trace:
I20260709 13:02:07.533968  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:07.534574  2347 raft_consensus.cc:2376] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14. Re-sending same reply.
I20260709 13:02:07.533360  1628 rpcz_store.cc:276] 0709 13:02:06.144522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.524921 (+1380399us) service_pool.cc:224] Handling call
0709 13:02:07.529942 (+  5021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.530722 (+   780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.530948 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:07.536583  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:07.536993  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:07.537665  2344 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:02:07.537837  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:07.538740  2353 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:07.529872  7339 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.520252  7357 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.542514  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.543891  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.544566  1707 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:02:07.545795  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.546525  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.551337  1701 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:07.547326  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:07.552522  2349 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:07.552661  1708 raft_consensus.cc:2368] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:07.548031  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:07.555491  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:07.556331  7399 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:07.556804  7399 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.531787  7418 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:02:07.557188  7399 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:07.558993  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 862) took 1024 ms. Trace:
I20260709 13:02:07.559396  1559 rpcz_store.cc:276] 0709 13:02:06.534608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.535056 (+   448us) service_pool.cc:224] Handling call
0709 13:02:07.540549 (+1005493us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b540038a9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:07.558906 (+ 18357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964884352}
I20260709 13:02:07.565228  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:07.565745  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.567055  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.568164  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.556428  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3058) took 1413 ms. Trace:
I20260709 13:02:07.570734  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.572398  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.573638  1632 rpcz_store.cc:276] 0709 13:02:06.142912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.458074 (+1315162us) service_pool.cc:224] Handling call
0709 13:02:07.458404 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.458648 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:07.554193 (+ 95545us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.555300 (+  1107us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.555497 (+   197us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.555768 (+   271us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.556058 (+   290us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:07.556116 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:07.556187 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.556252 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.556362 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76,"raft.queue_time_us":225,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":154,"thread_start_us":49041,"threads_started":2}
I20260709 13:02:07.569046  7355 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:07.534008  7403 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:07.569320  7339 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.582396  7188 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:07.582633  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.582989  7188 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.583545  1632 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:07.584403  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.705770  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.682231  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.686100  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:07.684577  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2783) took 1848 ms (client timeout 1600 ms). Trace:
I20260709 13:02:07.686283  1701 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:07.686570  1636 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:02:07.574292  1628 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:07.707989  1707 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:07.697391  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:07.626994  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.708228  1631 rpcz_store.cc:269] 0709 13:02:05.835921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.836106 (+   185us) service_pool.cc:224] Handling call
0709 13:02:07.684269 (+1848163us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b540010ccd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:07.684486 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1794861440}
I20260709 13:02:07.586057  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.708611  1636 pending_rounds.cc:85] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 6
I20260709 13:02:07.595008  1632 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:07.710626  1710 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:07.588066  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.689383  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:07.690869  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:07.709614  2347 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:07.710065  1630 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
W20260709 13:02:07.636826  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.711730  2346 raft_consensus.cc:2410] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:07.709990  1636 pending_rounds.cc:107] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:02:07.614837  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.712608  1636 raft_consensus.cc:2889] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:07.713056  1635 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:07.714685  2318 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:07.715891  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:07.716008  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:07.717371  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:07.683827  7433 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.694s	user 0.001s	sys 0.001s
W20260709 13:02:07.606483  7457 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.067s	user 0.001s	sys 0.001s
I20260709 13:02:07.718365  7418 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:07.718940  7416 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:07.717806  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 94320.2us
I20260709 13:02:07.617985  7382 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:07.729149  7339 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:07.731707  7416 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.734151  1632 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:07.734412  7416 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:07.735282  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.735878  1632 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:07.735292  7418 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:07.721453  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:07.720949  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:07.722967  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.723245  7377 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:07.720022  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.754190  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:07.666718  7353 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:07.742444  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:07.754673  1706 raft_consensus.cc:2368] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18.
I20260709 13:02:07.727262  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:07.756640  1525 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:07.725003  7304 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:07.764597  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:07.765388  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.767405  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:07.665840  7406 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:02:07.770154  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.777863  1631 raft_consensus.cc:2376] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17. Re-sending same reply.
W20260709 13:02:07.771200  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 44161.2us
I20260709 13:02:07.769763  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:07.760072  7416 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:07.773064  7317 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:07.773394  1525 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:07.774895  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:07.737185  7188 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:07.786011  7350 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:07.788772  7188 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:07.789036  7399 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.793195  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:07.793380  1706 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:07.793908  2353 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:07.779659  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:07.782269  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.754335  7377 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:07.745719  7382 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:07.785228  1525 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:07.799263  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2954) took 2051 ms. Trace:
I20260709 13:02:07.737160  7418 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:07.766577  7358 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:07.766907  7339 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:07.767835  7353 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.794412  7350 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:07.836249  2353 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:07.797359  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.837776  1631 raft_consensus.cc:2376] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13. Re-sending same reply.
I20260709 13:02:07.771658  7128 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:07.770218  7428 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:07.770802  7304 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:07.801340  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:07.846014  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44802.8us
W20260709 13:02:07.846603  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 67033.7us
I20260709 13:02:07.846967  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:07.848542  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:07.848587  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.800390  7377 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.811050  7093 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:07.811405  2348 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:07.861791  2344 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:07.813941  7360 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:07.814594  7382 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:07.815562  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 864) took 1280 ms. Trace:
I20260709 13:02:07.800519  1557 rpcz_store.cc:276] 0709 13:02:05.747408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.747568 (+   160us) service_pool.cc:224] Handling call
0709 13:02:05.759928 (+ 12360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:06.198538 (+438610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:06.198585 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:06.199114 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:02:06.777303 (+578189us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:06.777347 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.964176 (+186829us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54002526d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:06.964267 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:07.683059 (+718792us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:07.799006 (+115947us) raft_consensus.cc:1685] finished
0709 13:02:07.799033 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.799195 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397584384,"thread_start_us":426118,"threads_started":1,"wal-append.queue_time_us":651766}
I20260709 13:02:07.805817  7301 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:07.782243  7444 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:07.819003  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2904) took 1628 ms. Trace:
I20260709 13:02:07.821410  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:07.823050  7399 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:07.778040  7421 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:07.849974  1706 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:07.857753  1632 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:07.863828  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.781602  7416 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:07.886584  7416 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:07.887463  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:07.874619  1633 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.869606  1554 rpcz_store.cc:276] 0709 13:02:06.535597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.535730 (+   133us) service_pool.cc:224] Handling call
0709 13:02:07.815259 (+1279529us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b540005a854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:07.815426 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1416532352}
I20260709 13:02:07.836128  7317 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:07.874329  1709 rpcz_store.cc:276] 0709 13:02:06.190976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.191140 (+   164us) service_pool.cc:224] Handling call
0709 13:02:06.207801 (+ 16661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.494544 (+286743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:06.494580 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:06.494625 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:06.494656 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.818717 (+1324061us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400223654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:07.818768 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.818937 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658843520}
I20260709 13:02:07.846431  7304 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:07.892515  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:07.864447  7333 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:07.846851  7358 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:07.867201  7093 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:07.861239  7096 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.860250  7355 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.863508  7339 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:07.893491  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 125443us
I20260709 13:02:07.874042  7301 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:07.899323  1676 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:07.874735  7350 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:07.904963  1632 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:07.915017  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3064) took 1766 ms. Trace:
I20260709 13:02:07.907364  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.915442  1632 rpcz_store.cc:276] 0709 13:02:06.148801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.902787 (+1753986us) service_pool.cc:224] Handling call
0709 13:02:07.903317 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.914701 (+ 11384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.914953 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:07.918248  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.923359  1601 leader_election.cc:400] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:07.889391  7421 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:07.894371  7428 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.923977  1601 leader_election.cc:403] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:07.915607  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:07.930099  1706 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:07.902603  7360 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:07.904588  7382 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:07.935585  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.939867  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:07.957811  1704 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:07.957386  1706 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:07.901832  7333 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:07.914592  7350 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:07.935379  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:07.914252  7301 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:07.962953  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:07.930797  7188 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:07.906888  7317 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:07.913894  7267 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.901993  7406 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:02:07.952929  6932 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:07.970324  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:07.963325  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55465.3us
I20260709 13:02:07.979244  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:07.971204  7350 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.931826  7421 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:07.913894  7355 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:08.011705  1557 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:08.006923  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:08.007486  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:07.930011  7373 consensus_queue.cc:1059] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:02:07.963310  7467 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.013825  1553 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:07.922284  7096 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:08.002038  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 153580us
I20260709 13:02:07.963266  7333 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:07.980161  7188 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.040148  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:08.051316  1560 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.972421  7339 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:07.954552  7428 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.073562  1631 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:08.074949  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 871) took 1497 ms. Trace:
I20260709 13:02:08.075129  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:08.036047  6932 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:08.038254  7358 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.007781  7399 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:08.045214  7465 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:02:08.045536  7473 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:08.046100  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.086937  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.052556  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:08.054282  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:08.065699  1704 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:08.049582  7467 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:08.050318  7464 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:02:08.004428  7128 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:08.075598  1552 rpcz_store.cc:276] 0709 13:02:06.577520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.602312 (+ 24792us) service_pool.cc:224] Handling call
0709 13:02:06.602667 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.664612 (+ 61945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:06.664664 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:06.664771 (+   107us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:06.664833 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.074563 (+1409730us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b54001ce154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:08.074636 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.074849 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141741312}
W20260709 13:02:08.040608  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 110068us
I20260709 13:02:08.094208  1558 raft_consensus.cc:2368] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12.
I20260709 13:02:08.068940  7188 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:08.127094  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 203931us
I20260709 13:02:08.008364  7267 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:08.181322  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 202156us
I20260709 13:02:08.094887  7374 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:08.234599  7468 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:02:08.156654  7353 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:08.162569  7357 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:08.384421  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.185992  7317 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:08.337013  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 282842us
I20260709 13:02:08.338181  1556 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:08.012677  7418 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:02:08.013777  7093 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:08.340096  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.395949  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:08.038754  7305 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:02:08.341991  1560 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:08.340109  1633 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:08.045576  7373 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:08.357141  1553 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:08.345882  1557 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:08.340873  1674 leader_election.cc:336] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.518s (SENT)
W20260709 13:02:08.408941  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 68113.4us
W20260709 13:02:08.346307  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 334570us
W20260709 13:02:08.373003  2348 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.511s	user 0.014s	sys 0.002s
I20260709 13:02:08.385309  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:08.345556  7473 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.414047  1633 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:08.414742  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3069) took 2264 ms. Trace:
I20260709 13:02:08.415122  1633 rpcz_store.cc:276] 0709 13:02:06.150371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.413514 (+2263143us) service_pool.cc:224] Handling call
0709 13:02:08.413848 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.414476 (+   628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.414677 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.374140  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:08.397249  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:08.410912  2348 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:08.387101  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:08.420081  6932 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.420774  7093 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:08.421362  7267 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:08.383939  7314 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.423695  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:08.424582  7355 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:08.374766  7476 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:02:08.429363  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 998) took 2484 ms. Trace:
W20260709 13:02:08.429736  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55665.6us
I20260709 13:02:08.383648  7360 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.431825  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:08.429872  2348 rpcz_store.cc:276] 0709 13:02:05.933873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.059173 (+125300us) service_pool.cc:224] Handling call
0709 13:02:06.113255 (+ 54082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.667057 (+553802us) spinlock_profiling.cc:243] Waited 77.7 ms on lock 0x7b54006bdb20. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27651dc0 00007fda276dc1c0 00007fda276dd231 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c
0709 13:02:07.400004 (+732947us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b54006bdb20. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda2764fa1d 00007fda276361a4 00007fda276b8447 00007fda276d0fe7 00007fda276d27de 00007fda276dcb49 00007fda276dd416 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419
0709 13:02:07.810977 (+410973us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b58006f1ad0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276360aa 00007fda276b8447 00007fda276d0fe7 00007fda276d27de 00007fda276dcb49 00007fda276dd416 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4
0709 13:02:08.417747 (+606770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.418052 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163452800}
I20260709 13:02:08.462604  1633 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:08.462790  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:08.463718  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3071) took 2300 ms. Trace:
I20260709 13:02:08.424348  7314 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:08.464275  1633 rpcz_store.cc:276] 0709 13:02:06.163950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.416533 (+2252583us) service_pool.cc:224] Handling call
0709 13:02:08.461622 (+ 45089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.463359 (+  1737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.463636 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.407102  7426 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:08.481194  1601 leader_election.cc:400] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18.
I20260709 13:02:08.481683  1601 leader_election.cc:403] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:02:08.482722  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:08.486105  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:08.415009  7339 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:08.407838  7373 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:08.436789  7308 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:02:08.500236  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:08.428928  7429 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:08.503940  7466 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:08.510800  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 123445us
I20260709 13:02:08.530891  1556 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:08.573393  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:08.575865  1558 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:08.579969  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.447548  7267 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:02:08.587255  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 190081us
W20260709 13:02:08.598892  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.599823  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:08.488111  7374 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:08.481422  7355 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:08.500259  7339 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:08.498896  7357 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18.
I20260709 13:02:08.579823  7428 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:08.559430  7128 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.611714  2318 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:08.615520  7399 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:08.628871  7357 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:02:08.616773  7469 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:02:08.629704  7357 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:08.617578  7472 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.226s	user 0.000s	sys 0.000s
I20260709 13:02:08.600087  7485 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:08.630887  7485 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.583012  7096 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:08.620033  7350 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.584s	user 0.009s	sys 0.005s
W20260709 13:02:08.620451  7333 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.000s
W20260709 13:02:08.425271  7444 consensus_queue.cc:1175] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" responder_term: 15 status { last_received { term: 14 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:08.623687  7470 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:08.719376  7470 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:08.720026  7470 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:08.620553  7355 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.724311  7355 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:08.628136  7339 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:08.619127  7416 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.732s	user 0.014s	sys 0.002s
I20260709 13:02:08.631910  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3065) took 2482 ms. Trace:
W20260709 13:02:08.632920  1674 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.220s (SENT) [suppressed 33 similar messages]
I20260709 13:02:08.461721  7473 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.644237  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:08.656683  1633 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:08.645785  2346 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:08.654942  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:08.560115  7426 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
W20260709 13:02:08.659657  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 173580us
I20260709 13:02:08.737450  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:08.658099  7403 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.657861  7484 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:08.666558  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:08.742602  2345 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:02:08.742987  7479 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
W20260709 13:02:08.655889  7465 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:08.744001  7426 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:02:08.665051  7488 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:02:08.744479  7426 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:08.666862  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3073) took 2471 ms. Trace:
I20260709 13:02:08.672744  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:08.745685  1634 rpcz_store.cc:276] 0709 13:02:06.199605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.665973 (+2466368us) service_pool.cc:224] Handling call
0709 13:02:08.666312 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.666517 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.666545 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.666591 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:08.666637 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.666695 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.666805 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.642836  7466 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.749881  7466 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.751559  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2790) took 2903 ms. Trace:
I20260709 13:02:08.751780  7403 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.592398  7429 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.745694  7426 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:08.654316  7485 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:08.756918  7426 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:08.695788  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:08.690671  7299 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:08.671512  7482 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.765640  7299 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:08.765741  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 72259.6us
I20260709 13:02:08.766181  7482 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.679786  7314 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:08.677922  7373 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.766971  7299 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.770434  7299 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:08.691329  7477 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 13:02:08.710671  7467 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:08.775655  7467 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:08.776191  7467 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:08.643947  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:08.659425  7350 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:08.700400  7377 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.818s	user 0.014s	sys 0.000s
I20260709 13:02:08.714300  1627 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:02:08.701226  7301 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.724s	user 0.012s	sys 0.001s
W20260709 13:02:08.699896  7382 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.002s
I20260709 13:02:08.718005  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:08.727608  1632 rpcz_store.cc:276] 0709 13:02:06.149148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.916838 (+1767690us) service_pool.cc:224] Handling call
0709 13:02:07.917233 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:07.974161 (+ 56928us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:07.974188 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:08.039094 (+ 64906us) log.cc:844] Serialized 26 byte log entry
0709 13:02:08.398941 (+359847us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:08.399007 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.399113 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.631584 (+232471us) raft_consensus.cc:1685] finished
0709 13:02:08.631615 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.631827 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":206080,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":27799,"thread_start_us":332139,"threads_started":2,"wal-append.queue_time_us":523080}
I20260709 13:02:08.678932  7188 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.481683  7360 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:08.642486  7428 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:08.752051  1629 rpcz_store.cc:276] 0709 13:02:05.848077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.775746 (+927669us) service_pool.cc:224] Handling call
0709 13:02:07.428219 (+652473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.428822 (+   603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.751322 (+1322500us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b540017b950. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:08.751489 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1458951936}
I20260709 13:02:08.657930  7096 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:08.762903  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3080) took 2306 ms. Trace:
I20260709 13:02:08.743079  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3075) took 2535 ms. Trace:
I20260709 13:02:08.798357  1702 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:02:08.739967  7355 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.671674  7357 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:08.826339  1552 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:08.829095  7267 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:08.829695  7480 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:08.838438  2351 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:08.837055  1628 rpcz_store.cc:276] 0709 13:02:06.207441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.742039 (+2534598us) service_pool.cc:224] Handling call
0709 13:02:08.742440 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.742660 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.742688 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.742732 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:08.742790 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.742854 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.743012 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.816919  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3074) took 2613 ms. Trace:
I20260709 13:02:08.809533  7314 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.846755  7448 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:08.850391  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:08.851387  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.854101  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:08.738454  7416 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:08.840080  1634 rpcz_store.cc:276] 0709 13:02:06.456810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.748891 (+2292081us) service_pool.cc:224] Handling call
0709 13:02:08.749346 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.762118 (+ 12772us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.762276 (+   158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.762370 (+    94us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:08.762446 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.762548 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.762807 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.870580  1705 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
W20260709 13:02:08.874281  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.884951  1554 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
W20260709 13:02:08.863050  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 263303us
I20260709 13:02:08.867167  1627 rpcz_store.cc:276] 0709 13:02:06.203118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.693565 (+2490447us) service_pool.cc:224] Handling call
0709 13:02:08.709638 (+ 16073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.816496 (+106858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.816835 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:08.893613  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.863507  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.740250  7333 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:08.809837  7306 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:02:08.895599  7304 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.950s	user 0.007s	sys 0.007s
I20260709 13:02:08.899899  7386 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.899658  7267 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:08.899571  7306 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:08.906599  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:08.907298  1559 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:08.907884  7306 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:08.903600  7304 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:08.909706  1559 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:08.862394  7355 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.911602  7304 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:08.912012  7355 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:08.912420  1560 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:08.836254  7096 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.825265  7301 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.903719  1561 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
W20260709 13:02:08.859979  7353 raft_consensus.cc:670] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } [suppressed 5 similar messages]
I20260709 13:02:08.909636  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:08.862141  7483 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.941673  7483 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.945096  7495 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:08.945281  7303 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:08.945880  7495 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.946262  7303 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.948912  7303 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:08.777146  7467 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.831817  7374 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.820976  7377 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.956166  7374 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.958001  7377 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:08.866662  7484 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:08.958698  7374 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:08.862287  7093 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:08.919100  1628 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:08.897316  7428 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.959899  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3084) took 1949 ms. Trace:
I20260709 13:02:08.959751  7428 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.960400  1628 rpcz_store.cc:276] 0709 13:02:07.010654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.902143 (+1891489us) service_pool.cc:224] Handling call
0709 13:02:08.918748 (+ 16605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.959552 (+ 40804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.959824 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.961694  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:08.962100  7428 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:08.939924  1555 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:08.963128  7306 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:08.959071  7484 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.963855  7306 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.873400  7370 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:08.963153  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:08.970516  1628 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:08.917361  7470 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:08.971385  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:08.972147  1559 raft_consensus.cc:2376] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16. Re-sending same reply.
W20260709 13:02:08.973116  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 50571.8us
W20260709 13:02:08.973932  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 341016us
I20260709 13:02:08.944231  7483 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:08.978626  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:08.981166  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3082) took 2363 ms. Trace:
I20260709 13:02:08.988992  1632 rpcz_store.cc:276] 0709 13:02:06.617353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.814145 (+2196792us) service_pool.cc:224] Handling call
0709 13:02:08.814589 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.814893 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:08.940951 (+126058us) log.cc:844] Serialized 14 byte log entry
0709 13:02:08.980603 (+ 39652us) log.cc:844] Serialized 14 byte log entry
0709 13:02:08.980777 (+   174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.980823 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:08.980883 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.980970 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.981105 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15158,"raft.queue_time_us":24159,"thread_start_us":127990,"threads_started":2}
I20260709 13:02:08.991096  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:08.905221  7386 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:08.989076  1557 raft_consensus.cc:2376] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15. Re-sending same reply.
I20260709 13:02:08.991885  1632 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:08.920046  7096 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:08.897020  7314 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:08.981089  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:08.949260  7495 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:09.003057  7314 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:08.993252  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:08.998718  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.005435  1632 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:09.006361  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.007289  1632 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:02:09.014063  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.014964  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:08.995604  7421 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.954s	user 0.014s	sys 0.002s
I20260709 13:02:09.026461  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:09.027141  1553 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:09.036197  1553 raft_consensus.cc:2410] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:09.007248  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:08.999732  7484 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:09.023362  1632 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:09.047382  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:09.022119  1628 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:08.981042  7093 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:09.023906  7370 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:09.004994  7416 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:09.037024  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.030272  7304 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.040616  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 129601us
I20260709 13:02:09.041174  7470 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:09.042740  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:09.079551  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3085) took 2068 ms. Trace:
I20260709 13:02:09.080062  1553 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:09.044286  1556 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:09.048873  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:09.080075  1632 rpcz_store.cc:276] 0709 13:02:07.011053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.078656 (+2067603us) service_pool.cc:224] Handling call
0709 13:02:09.078994 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.079203 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.079230 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.079276 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:09.079319 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.079381 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.079490 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.082475  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.082782  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.083887  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:09.084589  1628 raft_consensus.cc:2376] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14. Re-sending same reply.
I20260709 13:02:09.073185  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.086282  1557 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:09.086725  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.087517  1628 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:09.088413  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.094188  1632 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 12: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:09.095443  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:09.097450  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.098134  1553 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:09.051457  7336 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:02:09.103935  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:09.097450  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:09.101538  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.105026  1632 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.105252  1628 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:02:09.105911  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.015719  7485 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:09.106855  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.033951  7421 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:09.108291  1628 raft_consensus.cc:2410] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:09.109681  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.111191  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.109372  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:09.109949  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:09.111049  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 76356.5us
I20260709 13:02:09.152534  1557 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:09.111948  1628 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:09.155004  1632 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:09.175957  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3087) took 1877 ms. Trace:
I20260709 13:02:09.183578  1632 rpcz_store.cc:276] 0709 13:02:07.298505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.174715 (+1876210us) service_pool.cc:224] Handling call
0709 13:02:09.175064 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.175496 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.175540 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.175602 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:09.175661 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.175742 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.175884 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.197595  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.198551  1553 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:09.082043  7304 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:09.262779  1632 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:09.384141  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2815) took 2066 ms. Trace:
I20260709 13:02:09.385129  1632 rpcz_store.cc:276] 0709 13:02:07.317330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.188787 (+1871457us) service_pool.cc:224] Handling call
0709 13:02:09.189229 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.383759 (+194530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.384046 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.386920  1632 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:09.325554  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:09.382259  1628 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:09.303993  1556 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:09.389169  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2811) took 2283 ms. Trace:
I20260709 13:02:09.389940  1628 rpcz_store.cc:276] 0709 13:02:07.105572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.156811 (+2051239us) service_pool.cc:224] Handling call
0709 13:02:09.381869 (+225058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.388548 (+  6679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.389092 (+   544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.391914  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1028) took 1814 ms. Trace:
I20260709 13:02:09.392434  1632 rpcz_store.cc:276] 0709 13:02:07.578024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.386297 (+1808273us) service_pool.cc:224] Handling call
0709 13:02:09.386677 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.391591 (+  4914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.391847 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.326002  1558 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:09.397563  1628 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:09.405661  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1029) took 1827 ms. Trace:
I20260709 13:02:09.414286  7188 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:09.415971  7093 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:09.418057  7508 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.322s	user 0.000s	sys 0.001s
I20260709 13:02:09.418815  1632 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:09.419605  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1031) took 1840 ms. Trace:
I20260709 13:02:09.419917  1632 rpcz_store.cc:276] 0709 13:02:07.579096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.408088 (+1828992us) service_pool.cc:224] Handling call
0709 13:02:09.408525 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.419312 (+ 10787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.419543 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.409839  1628 rpcz_store.cc:276] 0709 13:02:07.578445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.390976 (+1812531us) service_pool.cc:224] Handling call
0709 13:02:09.391322 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.405231 (+ 13909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.405574 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.422324  7386 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:09.422971  7503 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:02:09.422369  7512 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.270s	user 0.000s	sys 0.000s
W20260709 13:02:09.428025  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.336194  7426 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.560s	user 0.006s	sys 0.006s
W20260709 13:02:09.324921  7339 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.599s	user 0.006s	sys 0.006s
I20260709 13:02:09.431504  1557 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:09.099318  7317 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:02:09.454821  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 447596us
I20260709 13:02:09.355036  7333 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:09.104502  7350 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:09.331573  7477 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:02:09.562664  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 406540us
I20260709 13:02:09.565508  1559 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:09.321624  7448 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.574627  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 69667.4us
W20260709 13:02:09.307308  7476 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.784s	user 0.007s	sys 0.005s
I20260709 13:02:09.110533  7374 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.589926  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.603101  1553 raft_consensus.cc:2368] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:09.601106  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:09.595819  2347 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.011s	sys 0.037s
W20260709 13:02:09.603917  2347 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.012s	sys 0.037s
I20260709 13:02:09.264086  7483 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:09.455688  7370 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:09.605449  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2791) took 3733 ms. Trace:
W20260709 13:02:09.579628  7504 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:02:09.579227  7317 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:09.685511  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 123539us
I20260709 13:02:09.589620  7477 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:09.627537  1636 rpcz_store.cc:276] 0709 13:02:05.851537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.391698 (+1540161us) service_pool.cc:224] Handling call
0709 13:02:07.686088 (+294390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.713600 (+ 27512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.584643 (+1871043us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b540010ccd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:09.584848 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":676008704}
I20260709 13:02:09.648180  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:09.692209  7498 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
W20260709 13:02:09.693089  7513 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.314s	user 0.000s	sys 0.000s
I20260709 13:02:09.696687  1553 raft_consensus.cc:2410] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:02:09.694327  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 93307.1us
I20260709 13:02:09.563071  7339 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:09.658923  7514 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:02:09.714607  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44835.7us
I20260709 13:02:09.722790  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.610558  7476 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:09.723534  1636 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:09.722976  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:09.603572  7188 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.117s
I20260709 13:02:09.730234  1553 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:09.606364  7448 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:09.610350  7318 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.122s
I20260709 13:02:09.734387  1636 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:02:09.737720  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1034) took 2157 ms. Trace:
I20260709 13:02:09.736030  7521 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:02:09.738158  1636 rpcz_store.cc:276] 0709 13:02:07.579867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.727147 (+2147280us) service_pool.cc:224] Handling call
0709 13:02:09.734162 (+  7015us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:09.737185 (+  3023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.737658 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.740033  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.740705  1553 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:09.741945  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.646871  7374 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:09.747798  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1035) took 2167 ms. Trace:
I20260709 13:02:09.749487  1636 rpcz_store.cc:276] 0709 13:02:07.580147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.739751 (+2159604us) service_pool.cc:224] Handling call
0709 13:02:09.747206 (+  7455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.747411 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.747438 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.747482 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:09.747525 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.747602 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.747741 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:09.751636  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 149771us
I20260709 13:02:09.751829  1629 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:09.696635  7483 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:09.738883  7500 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.389s	user 0.001s	sys 0.000s
I20260709 13:02:09.756687  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.757421  1553 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:09.762542  1636 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:09.763465  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3089) took 2037 ms. Trace:
I20260709 13:02:09.763859  1636 rpcz_store.cc:276] 0709 13:02:07.726212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.750849 (+2024637us) service_pool.cc:224] Handling call
0709 13:02:09.762302 (+ 11453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.763100 (+   798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.763392 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.767804  1632 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:09.746572  7304 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:09.742791  7362 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:02:09.754132  7515 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:09.771227  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.783896  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.766621  7516 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:09.748860  7356 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:02:09.754838  7322 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:02:09.790138  7516 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:09.795493  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:09.786252  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 87670.4us
W20260709 13:02:09.789696  7096 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.783s	user 0.007s	sys 0.005s
I20260709 13:02:09.745363  7477 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:09.796742  1674 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:09.789917  7515 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:09.805538  7096 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:09.813270  7524 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:09.813804  7524 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:09.812866  7515 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:09.814888  7524 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:09.816334  7524 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:09.817705  7518 consensus_queue.cc:1175] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d62ce354d09a4be59c29094425d09bc1" responder_term: 13 status { last_received { term: 12 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:09.818879  7448 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:09.768549  7370 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:09.808646  7521 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:02:09.808780  7519 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.001s
I20260709 13:02:09.761554  7317 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:09.801151  7516 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:09.810650  7339 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:09.811699  7374 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:09.813875  1628 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 14. Ops: []
I20260709 13:02:09.807793  7483 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:09.822592  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1017) took 1175 ms. Trace:
I20260709 13:02:09.831249  2352 rpcz_store.cc:276] 0709 13:02:08.644188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.644541 (+   353us) service_pool.cc:224] Handling call
0709 13:02:08.671302 (+ 26761us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:08.872333 (+201031us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:08.916186 (+ 43853us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.096006 (+179820us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:09.105030 (+  9024us) log.cc:844] Serialized 26 byte log entry
0709 13:02:09.105176 (+   146us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:09.105382 (+   206us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.109575 (+  4193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.109647 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:09.819656 (+710009us) raft_consensus.cc:1685] finished
0709 13:02:09.819676 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.819809 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6010,"prepare.queue_time_us":42552,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":728,"raft.queue_time_us":110745,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":156618,"thread_start_us":307057,"threads_started":2,"wal-append.queue_time_us":409923}
I20260709 13:02:09.822649  7521 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:09.823496  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.834508  7521 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:09.834949  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.835917  7521 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:09.836314  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:09.837961  1553 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:09.823366  1675 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:09.839488  7362 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:09.840082  7362 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:09.840564  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:09.841168  7362 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:09.841696  6932 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:09.841744  7322 consensus_queue.cc:1059] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:09.842293  7528 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:09.842242  6932 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:09.842824  6932 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:09.842941  7528 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:09.843072  7322 raft_consensus.cc:3055] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:09.844128  7322 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:09.843971  7528 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:09.826256  7477 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:09.845089  7322 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:09.846472  7322 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:09.847402  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2958) took 1833 ms. Trace:
I20260709 13:02:09.847817  1561 rpcz_store.cc:276] 0709 13:02:08.013984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.014523 (+   539us) service_pool.cc:224] Handling call
0709 13:02:08.069340 (+ 54817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:08.920466 (+851126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.847076 (+926610us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b54002ff654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:09.847332 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1911255808}
I20260709 13:02:09.850282  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:09.851541  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.853428  1634 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:09.856926  1634 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:09.880584  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:09.886373  2317 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:09.861236  7096 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:09.894964  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:09.864768  7516 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:09.893638  7386 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:09.902421  2317 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:09.885329  7481 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:09.905680  7481 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.906246  7481 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:09.906533  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2967) took 1541 ms. Trace:
I20260709 13:02:09.909615  7386 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.910228  7386 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:09.900074  7299 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:09.913939  7299 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.914043  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1032) took 2334 ms. Trace:
I20260709 13:02:09.914343  7299 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.914455  1632 rpcz_store.cc:276] 0709 13:02:07.579353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.420790 (+1841437us) service_pool.cc:224] Handling call
0709 13:02:09.421107 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.768159 (+347052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.913775 (+145616us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54001b6f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:09.913982 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796525312}
W20260709 13:02:09.917079  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 2955) took 1952 ms (client timeout 1539 ms). Trace:
W20260709 13:02:09.917544  1554 rpcz_store.cc:269] 0709 13:02:07.964386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.971971 (+  7585us) service_pool.cc:224] Handling call
0709 13:02:09.916890 (+1944919us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b5400271fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:09.917004 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912548608}
I20260709 13:02:09.907179  1560 rpcz_store.cc:276] 0709 13:02:08.365416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.401461 (+ 36045us) service_pool.cc:224] Handling call
0709 13:02:08.401804 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.912795 (+510991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.906261 (+993466us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b54002530d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:09.906463 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173884160}
I20260709 13:02:09.928318  7421 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:09.928087  6932 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:09.929565  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2974) took 1321 ms. Trace:
I20260709 13:02:09.929952  1558 rpcz_store.cc:276] 0709 13:02:08.608235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.608416 (+   181us) service_pool.cc:224] Handling call
0709 13:02:08.608815 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.399796 (+790981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.929311 (+529515us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b5400330ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:09.929507 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1648771968}
I20260709 13:02:09.932520  7308 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:09.932932  7308 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.933293  7308 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:09.948832  2344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1406) took 2187 ms (client timeout 2113 ms). Trace:
W20260709 13:02:09.944584  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 104100us
W20260709 13:02:09.949460  2344 rpcz_store.cc:269] 0709 13:02:07.761150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.761323 (+   173us) service_pool.cc:224] Handling call
0709 13:02:09.948578 (+2187255us) spinlock_profiling.cc:243] Waited 77.1 ms on lock 0x7b54004ed754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:09.948707 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177364224}
I20260709 13:02:09.951166  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:09.956033  7481 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:09.957896  7429 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:09.951359  7403 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:09.959404  7466 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:09.967126  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3093) took 2098 ms. Trace:
I20260709 13:02:09.967859  1636 rpcz_store.cc:276] 0709 13:02:07.868552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.764835 (+1896283us) service_pool.cc:224] Handling call
0709 13:02:09.765209 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.765533 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:09.830686 (+ 65153us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.966694 (+136008us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.966758 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:09.966819 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.966895 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.967058 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":189747,"threads_started":2,"wal-append.queue_time_us":133659}
W20260709 13:02:09.974735  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.970487  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 108403us
I20260709 13:02:09.980841  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:09.981801  1632 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.983908  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:09.984733  1636 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:09.986420  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:09.987403  2352 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:09.989104  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:09.992523  2344 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:09.949155  7386 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.007324  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:10.009064  1630 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:09.983589  7528 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:10.011574  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:10.012035  2344 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:10.014662  2352 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:10.021896  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:10.026629  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:10.026968  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:10.028904  2344 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:10.031287  7299 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.042725  2348 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:10.034476  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:10.043661  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 141301us
I20260709 13:02:10.044387  2345 raft_consensus.cc:2410] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 13:02:10.036229  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.045096  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:10.023672  6932 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:10.053396  1632 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:10.056053  1674 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.060525  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:10.058164  7429 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:10.060115  1674 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.062929  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:10.063174  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.066293  2352 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.020380  7528 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:10.064682  7481 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:10.068315  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:10.074602  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3092) took 2209 ms (client timeout 2625 ms). Trace:
I20260709 13:02:10.069408  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.077394  1602 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.065498  7429 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:10.075187  1631 rpcz_store.cc:269] 0709 13:02:07.860721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.887097 (+ 26376us) service_pool.cc:224] Handling call
0709 13:02:10.069405 (+2182308us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b540010ccd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:10.069590 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425739008}
I20260709 13:02:10.079892  7448 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:10.079710  7483 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:10.080899  7388 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:10.079710  7480 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:10.081866  7301 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:10.071138  7374 raft_consensus.cc:2764] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:02:10.084981  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1038) took 1008 ms. Trace:
I20260709 13:02:10.072319  7516 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:10.066473  7317 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:02:10.059090  7193 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:02:10.098430  1634 rpcz_store.cc:276] 0709 13:02:09.076453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.881451 (+804998us) service_pool.cc:224] Handling call
0709 13:02:09.881824 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.882105 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:09.971271 (+ 89166us) log.cc:844] Serialized 14 byte log entry
0709 13:02:10.084559 (+113288us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:10.084620 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:10.084683 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.084761 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.084905 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":198154,"prepare.run_cpu_time_us":179,"prepare.run_wall_time_us":178,"raft.queue_time_us":98,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":3832,"thread_start_us":184846,"threads_started":2}
I20260709 13:02:10.101141  1674 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.085464  7448 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:10.083755  7532 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:10.147460  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46432.7us
W20260709 13:02:10.167894  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.169262  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:10.089180  7480 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:10.083873  7494 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:02:10.098839  7516 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:10.088589  7429 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.174024  7524 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:10.193209  7524 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:10.194070  7524 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.173368  7356 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:02:10.199993  7299 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:10.199692  7308 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.178400  7521 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:10.172468  7448 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.167609  7532 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:10.202754  7521 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.171353  7480 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:10.212114  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 154331us
I20260709 13:02:10.208993  2352 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:10.186349  7516 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:10.227119  1602 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.213938  7521 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.213614  7532 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:10.246980  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.238338  7494 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:10.256157  2352 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:10.238833  7386 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:10.261915  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.264014  1560 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:10.204771  7528 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.265561  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:10.266484  1559 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:10.267457  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:10.267851  1525 leader_election.cc:336] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.726s (SENT)
I20260709 13:02:10.268428  1554 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:02:10.274271  2351 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1016) took 1630 ms (client timeout 2046 ms). Trace:
I20260709 13:02:10.268421  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:10.282397  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:10.274852  2351 rpcz_store.cc:269] 0709 13:02:08.643846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.644010 (+   164us) service_pool.cc:224] Handling call
0709 13:02:10.269312 (+1625302us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540002bf54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:10.274187 (+  4875us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240816000}
I20260709 13:02:10.306681  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:10.307628  2345 raft_consensus.cc:2368] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:02:10.308432  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:10.282785  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.310093  7541 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.085s	user 0.001s	sys 0.001s
I20260709 13:02:10.309561  7339 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.317373  7339 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.317854  7374 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:10.319811  7339 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:10.311635  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:10.321025  1561 raft_consensus.cc:2368] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12.
I20260709 13:02:10.321539  7494 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:10.322054  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:10.322896  1561 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:10.322679  7494 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.323846  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.325021  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.304816  7496 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.326053  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:10.326610  7480 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.315899  7429 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:10.328073  7373 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:10.326072  7496 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.326053  2352 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:10.331779  7306 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:10.332267  7496 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:10.311311  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.313282  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:10.335703  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.312747  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:10.336854  1627 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.337563  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:10.340333  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:10.341497  1561 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:10.279844  7301 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:10.343405  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:10.342281  2345 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:10.327244  7308 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:10.348984  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.304464  7421 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.356127  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:10.360343  1553 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:10.362766  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:10.338366  7486 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.370913  1627 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:10.341131  7339 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.385766  7339 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:10.386891  7267 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:10.387712  7267 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:10.376299  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:10.376485  7480 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:10.377712  7524 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:10.390091  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:10.390440  7267 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:10.391096  7524 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:10.391330  7473 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:10.391863  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:10.392396  7482 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:10.394223  1630 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:10.381642  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:10.395080  2351 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:10.396848  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1033) took 2817 ms. Trace:
I20260709 13:02:10.397258  1628 rpcz_store.cc:276] 0709 13:02:07.579613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.422011 (+1842398us) service_pool.cc:224] Handling call
0709 13:02:09.422260 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.831120 (+408860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.396589 (+565469us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b54002dbed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:10.396786 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":900187392}
I20260709 13:02:10.398619  7360 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:10.406056  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2273) took 1381 ms. Trace:
I20260709 13:02:10.398664  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:10.406919  1628 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.408622  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1018) took 1718 ms. Trace:
I20260709 13:02:10.347460  7496 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.378558  7256 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:10.415843  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:10.424971  1632 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:10.421816  2345 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:10.426008  2351 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:10.426558  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:10.427359  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.427793  2352 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:10.427380  2345 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:10.422564  1630 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:10.408998  2347 rpcz_store.cc:276] 0709 13:02:08.690309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.692747 (+  2438us) service_pool.cc:224] Handling call
0709 13:02:08.693212 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:09.011318 (+318106us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:09.011380 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:09.034655 (+ 23275us) log.cc:844] Serialized 26 byte log entry
0709 13:02:09.604455 (+569800us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:09.604512 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.408216 (+803704us) spinlock_profiling.cc:243] Waited 74.4 ms on lock 0x7b54004b7154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:10.408266 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:10.408317 (+    51us) raft_consensus.cc:1685] finished
0709 13:02:10.408330 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.408559 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9703,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":13054,"spinlock_wait_cycles":171075456,"thread_start_us":767152,"threads_started":2,"wal-append.queue_time_us":738972}
I20260709 13:02:10.431176  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:10.432211  2347 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:10.434507  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:10.435878  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:10.436193  2353 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:10.436591  2346 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:10.406481  1552 rpcz_store.cc:276] 0709 13:02:09.024969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.025082 (+   113us) service_pool.cc:224] Handling call
0709 13:02:09.025389 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.099923 (+ 74534us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:09.112427 (+ 12504us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.310651 (+198224us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.310736 (+    85us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:09.310806 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.839561 (+528755us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b18001b5980. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:10.037095 (+197534us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b18001b5980. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419
0709 13:02:10.405781 (+368686us) spinlock_profiling.cc:243] Waited 74.2 ms on lock 0x7b54002521d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:10.405830 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.405963 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12781,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":140,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":9211,"spinlock_wait_cycles":253414912,"thread_start_us":197764,"threads_started":1,"wal-append.queue_time_us":620418}
I20260709 13:02:10.439384  2349 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:02:10.373404  7301 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:10.448772  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.449632  1628 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:10.434168  7374 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:02:10.458622  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 745487us
I20260709 13:02:10.409791  7496 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:10.451658  7403 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.461572  7477 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:10.461799  7403 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.463344  7477 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:10.463842  7477 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.464244  7477 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:10.464432  7403 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:10.444178  7442 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.440568  7518 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.465757  7442 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.466140  7518 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:10.468258  7442 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:10.438568  7256 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:10.476838  7256 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.423928  7486 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:10.479728  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:10.456295  7301 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:10.480152  7486 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:10.481089  7301 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:10.481618  7527 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:10.482396  7527 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.483312  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.484158  1553 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:10.486900  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:10.465859  7193 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:02:10.487704  1553 raft_consensus.cc:2376] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13. Re-sending same reply.
I20260709 13:02:10.486937  7485 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:10.489960  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2965) took 2367 ms. Trace:
I20260709 13:02:10.490140  7317 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:10.490672  7317 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:10.490412  1555 rpcz_store.cc:276] 0709 13:02:08.122859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.365899 (+243040us) service_pool.cc:224] Handling call
0709 13:02:08.400090 (+ 34191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.963505 (+563415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.489709 (+1526204us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b5400206f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:10.489897 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195818368}
I20260709 13:02:10.492067  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:10.492260  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.492928  1627 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:10.493036  1559 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:10.493443  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:10.494024  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:10.491894  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.494385  1630 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:10.494695  1627 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:10.495051  1553 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:10.491885  7317 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:10.495836  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:10.496284  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.480690  1552 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:10.497023  7317 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.499053  7448 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.497023  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:10.503319  1627 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:10.444612  7308 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:10.505637  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:10.505792  7374 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:10.365931  7421 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.506548  1559 raft_consensus.cc:2410] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:02:10.506642  1636 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:02:10.504381  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:10.516638  1627 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.504326  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.508847  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:10.509667  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.517647  1552 raft_consensus.cc:2368] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:10.513351  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:10.514281  7552 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:02:10.514644  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.504956  7308 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.549985  7470 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.550679  7470 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.552356  7308 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:10.562306  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.565733  7306 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.565912  7339 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:10.517935  1554 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:10.568809  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.569896  1554 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:10.448314  7473 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.519523  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.520241  7448 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:10.521103  1553 raft_consensus.cc:2410] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:10.521623  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:10.605213  2352 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.522043  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.522317  7516 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:10.522804  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.524493  7428 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:10.618140  1627 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:10.526911  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:10.619947  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 93101.9us
I20260709 13:02:10.620926  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.528703  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:10.530419  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3072) took 4350 ms. Trace:
I20260709 13:02:10.485351  7527 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:10.534193  1561 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:10.533535  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:10.537282  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.631029  1628 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.538975  7402 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:10.542776  7521 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:10.545545  7532 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:10.547473  7374 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.548616  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:10.646488  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2296) took 1549 ms (client timeout 1852 ms). Trace:
W20260709 13:02:10.646889  1557 rpcz_store.cc:269] 0709 13:02:09.096646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.548231 (+1451585us) service_pool.cc:224] Handling call
0709 13:02:10.646394 (+ 98163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:10.566569  7306 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.469210  7518 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:10.574416  1555 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:10.570894  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:10.621227  2317 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.622646  1633 rpcz_store.cc:276] 0709 13:02:06.179883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.479136 (+2299253us) service_pool.cc:224] Handling call
0709 13:02:08.513867 (+ 34731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.733540 (+219673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.530121 (+1796581us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540010c7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:10.530348 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327535360}
W20260709 13:02:10.516986  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 58146.7us
W20260709 13:02:10.535295  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.618487  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.631309  1556 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:10.633762  1553 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:10.633994  2352 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:10.643442  1636 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:02:10.643865  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.641505  1630 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
W20260709 13:02:10.644766  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 116161us
I20260709 13:02:10.648510  1702 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:10.649652  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:10.617782  7428 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.756105  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:10.770879  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.757418  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:10.771368  1628 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
W20260709 13:02:10.758258  1556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2299) took 1428 ms (client timeout 1515 ms). Trace:
I20260709 13:02:10.737082  1703 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:10.760876  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 139734us
I20260709 13:02:10.740478  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:10.642437  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 185171us
I20260709 13:02:10.762343  1602 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.727505  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:10.774586  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:10.774962  1674 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.775308  1706 raft_consensus.cc:2376] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12. Re-sending same reply.
I20260709 13:02:10.761623  1552 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
W20260709 13:02:10.760145  1527 connection.cc:256] RPC call timeout handler was delayed by -1.0523112159999641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
I20260709 13:02:10.776437  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:10.776854  1559 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:10.773913  1556 rpcz_store.cc:269] 0709 13:02:09.329967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.536818 (+1206851us) service_pool.cc:224] Handling call
0709 13:02:10.757813 (+220995us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:10.776772  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:10.760418  1627 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:10.777791  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:10.777290  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.764979  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:10.781167  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.781876  1709 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:10.619532  7470 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:10.783130  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.789924  1708 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:10.791005  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.784438  1630 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:10.784574  1701 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:02:10.792357  1705 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:02:10.791610  1708 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:10.794219  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:10.794562  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.795861  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:10.773164  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.785876  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:10.788166  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.797518  1703 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:10.603842  7473 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.799752  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.773878  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:10.801102  1706 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 10.
I20260709 13:02:10.640929  7386 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:10.804214  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:10.648510  7322 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.804638  1527 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:10.806342  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 450338us
I20260709 13:02:10.799837  7360 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:10.883601  1703 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:10.884006  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:10.883548  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 109773us
I20260709 13:02:10.782896  7428 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:10.866312  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.866797  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.868366  1707 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:10.650053  7267 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:10.869079  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.786274  7308 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:10.863031  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.753386  7336 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:10.874493  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 69964.6us
I20260709 13:02:10.787921  7357 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:10.770428  7339 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.787662  7535 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:02:10.876277  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:10.802219  7524 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:02:10.959002  2352 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:10.867002  7429 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:10.951378  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:10.978716  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 166485us
I20260709 13:02:10.787757  7480 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:10.861755  7485 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:02:10.967736  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 91550.5us
I20260709 13:02:10.996326  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:10.998191  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:10.815166  7386 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.968984  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:10.862315  7559 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:10.959949  7267 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.980548  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:10.881713  7553 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:10.987680  7554 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:02:10.882652  7301 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:10.923918  7188 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:10.862761  7427 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 13:02:10.961033  7360 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:10.992940  7551 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:02:10.980489  7256 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:10.874918  7466 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.127s
I20260709 13:02:11.022778  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2806) took 4037 ms. Trace:
I20260709 13:02:10.972117  7357 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:11.001346  7308 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:11.001937  7448 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:02:10.967151  7336 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:10.995446  7470 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:11.008484  7494 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:02:10.883258  7322 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:10.891525  7096 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.118s
I20260709 13:02:10.972208  7339 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.012511  7267 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:11.056267  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 253548us
I20260709 13:02:11.051340  1629 rpcz_store.cc:276] 0709 13:02:06.985306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.855268 (+1869962us) service_pool.cc:224] Handling call
0709 13:02:08.872847 (+ 17579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.753072 (+880225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.022375 (+1269303us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b54003cd7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:11.022688 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2020927872}
I20260709 13:02:11.010926  7429 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:11.070981  7561 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:02:11.135072  7429 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.030495  7188 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:11.026454  7485 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:11.137295  1710 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:11.004498  7386 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.126930  7573 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.043435  7516 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:11.138630  1702 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
W20260709 13:02:11.102619  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 119688us
W20260709 13:02:11.113283  7558 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:02:11.151348  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:11.152746  7448 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:11.154783  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:11.105311  7486 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:11.145468  2352 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:11.052103  7356 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:11.073702  7568 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:11.106426  7546 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.106184  7532 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:11.152916  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:11.113973  7470 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:11.112900  7361 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:11.082163  7301 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:11.142007  7188 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:11.154240  7573 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:11.173125  7317 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.674s	user 0.008s	sys 0.003s
I20260709 13:02:11.175026  7521 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:11.127666  7575 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.135365  7322 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:11.052731  7256 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:11.182256  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 295625us
I20260709 13:02:11.156780  7308 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.201143  7473 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:11.137403  7360 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:11.199420  7575 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.191898  7361 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.137028  7336 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:11.074389  7524 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 13:02:11.185001  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:11.187614  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 188298us
I20260709 13:02:11.214776  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:11.162894  7486 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.219491  7486 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.221204  7361 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:11.221920  7402 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.576s	user 0.006s	sys 0.006s
I20260709 13:02:11.213994  7521 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:02:11.168179  7356 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:11.204728  6932 consensus_queue.cc:1059] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:02:11.242946  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 170760us
W20260709 13:02:11.243775  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.281222  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.290441  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:11.184536  7568 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:11.293347  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 108427us
I20260709 13:02:11.294409  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:11.218462  7575 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:11.295943  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:11.224133  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.188350  7448 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:11.194617  7301 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:11.234186  7524 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:11.222433  7447 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.238652  1703 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:11.213323  1704 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:11.210387  7256 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:11.183054  7546 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:11.342257  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 361753us
I20260709 13:02:11.223251  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:11.343849  1527 leader_election.cc:400] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:02:11.344408  1527 leader_election.cc:403] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:11.303030  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 58551.5us
I20260709 13:02:11.303448  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:11.311246  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:11.354591  1707 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:11.320154  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:11.328964  1634 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:11.330344  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:11.356596  1628 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:11.357100  2351 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:11.302618  7096 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:11.343587  1701 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.297513  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:11.346505  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 120593us
I20260709 13:02:11.348595  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:11.229229  7332 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
I20260709 13:02:11.320154  7402 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.320324  7524 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:11.310316  7577 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:11.369328  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:11.459309  1709 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:11.324756  7575 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.368001  1630 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:11.196597  7357 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.333084  7448 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:11.456686  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:11.201135  7485 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.467792  7578 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:02:11.501880  7339 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:11.502741  7339 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.504487  7306 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:11.507340  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:11.508216  2351 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:11.509704  1676 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:11.510881  7532 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:11.511319  7532 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.511421  7421 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:11.511713  7532 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.513959  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:11.514779  1707 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:11.518239  2351 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:02:11.471974  1701 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:11.480342  7575 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:11.483376  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:11.521796  1709 raft_consensus.cc:2410] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:02:11.483940  7587 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:02:11.484409  7428 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.003s
I20260709 13:02:11.471663  7577 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:11.303736  7470 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.370476  7427 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.142s
I20260709 13:02:11.347121  7447 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.355966  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:11.548062  1703 raft_consensus.cc:606] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:11.548462  1703 raft_consensus.cc:614] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:11.514319  7096 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:11.459122  7402 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.564699  7402 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:11.565899  2353 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:11.512651  7332 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:02:11.575996  7332 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:11.576858  7332 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:11.579890  7332 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:11.519572  7301 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:11.581746  7448 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:11.545006  7524 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.585785  7524 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:11.558660  7546 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:11.591732  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:11.593170  1632 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:11.564706  2350 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:11.542856  7336 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:02:11.572705  1559 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:11.557848  7573 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:11.597055  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:11.598032  1556 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:11.598603  1556 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:11.601253  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:11.542973  7577 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.602087  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:11.603626  1560 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:11.604239  1554 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:11.605368  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:11.599639  1556 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:11.578029  7564 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.388s	user 0.000s	sys 0.001s
I20260709 13:02:11.606070  1552 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:11.606829  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:11.609767  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:11.608011  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:11.616430  1559 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:11.588850  7386 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.612928  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:11.543579  7428 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.608815  1556 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.619792  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:11.605804  7563 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.050s	sys 0.047s
I20260709 13:02:11.645709  2349 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:11.627163  7563 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.051s	sys 0.047s
I20260709 13:02:11.552651  7568 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:11.687273  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 389851us
I20260709 13:02:11.628060  7402 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:11.688832  1601 leader_election.cc:400] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:11.691962  7476 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:11.692555  7546 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.695650  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:11.698580  2347 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:02:11.700852  7553 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:11.703032  7386 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:11.620483  7486 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.757763  1632 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:02:11.688982  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 326803us
I20260709 13:02:11.614730  7188 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.708705  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:11.713306  2346 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:11.709033  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:11.762346  1634 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.706949  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:11.762846  1636 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:11.743237  7477 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.278s	user 0.009s	sys 0.007s
W20260709 13:02:11.769088  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 425443us
I20260709 13:02:11.770301  1557 raft_consensus.cc:2368] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16.
I20260709 13:02:11.764854  1525 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:11.759867  1675 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.009s	sys 0.007s
I20260709 13:02:11.706171  7402 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:11.766294  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59432.3us
I20260709 13:02:11.705403  7428 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:11.777577  1627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.007s	sys 0.017s
I20260709 13:02:11.777448  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:11.777945  1627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.007s	sys 0.017s
I20260709 13:02:11.780164  1633 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:11.781662  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:11.782814  1633 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:11.787019  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:11.771234  1675 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.009s	sys 0.007s
I20260709 13:02:11.709555  7476 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:11.775254  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:11.796321  1632 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.803932  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3117) took 1019 ms. Trace:
I20260709 13:02:11.804667  1630 rpcz_store.cc:276] 0709 13:02:10.784199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.797012 (+ 12813us) service_pool.cc:224] Handling call
0709 13:02:10.797411 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.470646 (+673235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.803655 (+333009us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b5400010ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:11.803858 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311726976}
I20260709 13:02:11.806722  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:11.774380  7267 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.627s	user 0.013s	sys 0.000s
I20260709 13:02:11.818945  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:11.821758  1630 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:11.769345  7374 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:11.761695  7573 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:11.824723  7594 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.825054  7357 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:11.779326  7402 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:11.750977  7546 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.750131  7590 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:02:11.808943  7575 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:11.858201  7592 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:11.826927  7532 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.823576  7267 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:11.840440  7332 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:02:11.836751  7301 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:11.804677  7485 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
W20260709 13:02:11.887908  7574 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.747s	user 0.003s	sys 0.003s
W20260709 13:02:11.841920  7559 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.796s	user 0.004s	sys 0.010s
I20260709 13:02:11.900211  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3112) took 1117 ms. Trace:
I20260709 13:02:11.841054  7594 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:11.788887  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 647340us
I20260709 13:02:11.772998  7477 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:11.926443  2351 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:11.956444  1634 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:11.961000  1630 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:11.927299  1627 rpcz_store.cc:276] 0709 13:02:10.782597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.784473 (+  1876us) service_pool.cc:224] Handling call
0709 13:02:10.784797 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.800156 (+ 15359us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:11.101806 (+301650us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.797211 (+695405us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.809056 (+ 11845us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.809249 (+   193us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:11.809295 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.809370 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.899912 (+ 90542us) raft_consensus.cc:1685] finished
0709 13:02:11.899944 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.900134 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84934,"prepare.queue_time_us":207802,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":14524,"raft.queue_time_us":61251,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":47698,"thread_start_us":809549,"threads_started":2,"wal-append.queue_time_us":780496}
W20260709 13:02:11.950193  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.948534  1636 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:11.832172  7256 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:11.927371  1632 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:11.966434  1634 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:02:11.965332  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:11.965355  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 187977us
W20260709 13:02:11.968598  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 203862us
I20260709 13:02:11.974913  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:11.976941  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:11.977613  1632 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:11.978981  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:11.983083  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:11.990850  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:11.984314  7374 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:11.994839  1708 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:11.995621  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:11.993530  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:11.839376  7573 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:11.989885  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:11.998096  1627 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:11.986791  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:11.925772  7486 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:11.997153  2348 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:02:11.991914  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:11.960374  7559 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:12.000659  7096 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:12.007709  7096 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:12.008213  7096 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:12.009667  1633 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:12.011938  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:12.012651  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:12.013415  1627 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:12.014319  7477 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:12.012693  1631 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:12.003615  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.016044  1627 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:02:12.016311  7546 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:11.892205  7476 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.018272  7188 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:12.006475  7573 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:12.012674  7594 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:12.011966  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3102) took 2026 ms. Trace:
I20260709 13:02:11.960700  7339 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14.
I20260709 13:02:12.005266  1705 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:12.026680  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:12.019407  1635 rpcz_store.cc:276] 0709 13:02:09.985255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.015664 (+ 30409us) service_pool.cc:224] Handling call
0709 13:02:10.024579 (+  8915us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.025606 (+  1027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.025654 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:10.759277 (+733623us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b2c00070338. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:02:10.759734 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:02:10.991670 (+231936us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:10.991722 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.991825 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.765671 (+773846us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:12.011745 (+246074us) raft_consensus.cc:1685] finished
0709 13:02:12.011770 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.011905 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":446,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":15569,"spinlock_wait_cycles":23561344,"thread_start_us":116150,"threads_started":1,"wal-append.queue_time_us":311722}
I20260709 13:02:11.978610  7428 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:11.967790  7593 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:11.963840  7301 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:12.024346  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:12.024796  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 252784us
I20260709 13:02:12.024453  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.038426  2348 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:12.004949  7362 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:12.029673  1634 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:02:12.016045  1631 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:02:12.032028  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:11.999801  1632 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:02:11.993914  7386 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:11.985225  7575 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:12.039527  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:12.034130  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 260808us
I20260709 13:02:12.041510  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:12.041388  1556 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:11.969775  7357 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:02:11.999682  7470 raft_consensus.cc:3055] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:12.049741  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:12.055127  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:12.056085  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:12.056485  1635 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:12.056936  7609 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:12.038759  7601 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:02:12.045156  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:12.048756  1703 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:12.060316  1705 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:12.034472  7593 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:12.049165  7374 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.064945  7374 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:12.053115  7476 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:12.058512  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:12.056183  2351 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.052745  7532 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:12.072481  7593 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:12.073814  7593 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.058871  7386 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:12.076712  7386 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:12.082656  7581 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:02:12.083943  7301 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:12.092456  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47378.5us
W20260709 13:02:12.094278  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 54917.7us
I20260709 13:02:12.055090  1633 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:12.061985  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:12.094321  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:12.047319  7496 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:12.105839  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.060787  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:12.062419  7518 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:02:12.100327  2348 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:12.104323  7609 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:12.101451  2351 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:12.142458  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.142800  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.143211  1560 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:12.143424  1709 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:12.144124  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:12.145797  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.146431  1553 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
W20260709 13:02:12.146333  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 85637.6us
I20260709 13:02:12.062638  7470 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:12.140602  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:12.100239  7339 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.037799  7428 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.136795  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.146512  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:12.147504  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:12.145296  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.046427  7448 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:12.241632  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.242358  1560 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:12.242158  7356 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.243547  1560 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:12.243547  7356 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.148092  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:12.244593  7592 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:12.244980  1560 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:02:12.246105  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 98092.5us
I20260709 13:02:12.134626  1556 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:12.248565  1556 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:02:12.148664  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 54414.9us
I20260709 13:02:12.165371  1676 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:12.169155  1555 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:12.250085  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:12.250217  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 84959.7us
I20260709 13:02:12.169837  7267 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:12.251605  7267 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:12.252136  7267 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.174077  1709 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:12.253657  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.254135  2348 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEARNER]: Advancing to term 16
I20260709 13:02:12.256458  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.100077  7611 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:12.176676  7532 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:12.165371  7518 consensus_queue.cc:1243] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer 95fc6a4cf5874b7eb91280dffcd3b3fb log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:02:12.173305  7470 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.183702  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:12.272485  7618 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:02:12.196697  7572 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.069s	user 0.001s	sys 0.004s
W20260709 13:02:12.201538  7570 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.068s	user 0.006s	sys 0.009s
I20260709 13:02:12.204406  7428 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:12.146132  7600 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.315s	user 0.000s	sys 0.002s
W20260709 13:02:12.204675  2350 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1400) took 4668 ms (client timeout 3212 ms). Trace:
I20260709 13:02:12.206022  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:12.151386  7590 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:12.299268  7590 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:12.299887  7590 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:12.177137  7614 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:02:12.213887  7603 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.323s	user 0.004s	sys 0.000s
I20260709 13:02:12.214213  7402 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.306032  7402 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:12.306141  7447 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:12.218230  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 2941) took 1454 ms (client timeout 1534 ms). Trace:
W20260709 13:02:12.307832  1704 rpcz_store.cc:269] 0709 13:02:10.764258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.764694 (+   436us) service_pool.cc:224] Handling call
0709 13:02:12.217870 (+1453176us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b540016ba54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:12.218148 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":955051264}
W20260709 13:02:12.308565  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2259) took 1539 ms (client timeout 1515 ms). Trace:
I20260709 13:02:12.308744  1602 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:12.309128  1702 rpcz_store.cc:269] 0709 13:02:10.769026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.770590 (+  1564us) service_pool.cc:224] Handling call
0709 13:02:12.308357 (+1537767us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b540016c454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:12.308480 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844104448}
I20260709 13:02:12.310322  2344 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:12.310680  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:12.310365  7448 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:12.312003  7448 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.312395  7448 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:12.316651  7612 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.293s	user 0.001s	sys 0.001s
I20260709 13:02:12.319876  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1029) took 1174 ms. Trace:
I20260709 13:02:12.321673  7470 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.325229  7430 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:12.325903  7430 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:12.327837  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.219569  7607 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:02:12.222121  7256 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:12.227103  7374 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:12.330803  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 2999) took 2033 ms. Trace:
I20260709 13:02:12.331162  7448 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.332515  1628 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:12.331285  1558 rpcz_store.cc:276] 0709 13:02:10.297093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.340342 (+ 43249us) service_pool.cc:224] Handling call
0709 13:02:10.340697 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.629100 (+288403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.629152 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:10.643565 (+ 14413us) log.cc:844] Serialized 26 byte log entry
0709 13:02:10.962428 (+318863us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:10.962488 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.962580 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.196038 (+1233458us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:12.330472 (+134434us) raft_consensus.cc:1685] finished
0709 13:02:12.330497 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.330731 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14258,"raft.queue_time_us":37225,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":29513600,"thread_start_us":439930,"threads_started":2,"wal-append.queue_time_us":370012}
I20260709 13:02:12.294410  7611 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:12.334060  7611 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.337029  7611 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:12.228399  7524 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.005s
I20260709 13:02:12.342356  7448 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:12.235328  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.344039  7096 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.345764  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:12.347788  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:12.349687  7532 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:12.350071  1701 raft_consensus.cc:2410] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:02:12.351125  7532 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.352620  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:12.354791  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:12.356593  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.235805  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:12.358125  1710 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:12.358952  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.359752  1710 raft_consensus.cc:2410] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:02:12.360448  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.361164  1710 raft_consensus.cc:2410] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:02:12.235757  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.362001  1705 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
W20260709 13:02:12.362609  7577 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.746s	user 0.014s	sys 0.000s
I20260709 13:02:12.251506  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:12.362871  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.363569  1705 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:12.363288  7577 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:12.363822  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 112385us
I20260709 13:02:12.364176  7535 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:12.364492  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.281261  7362 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.364885  7611 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:12.366026  7362 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:12.368743  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.368397  7362 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:12.252178  7317 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:12.085970  7559 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:02:12.375633  1631 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:02:12.262221  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.264865  2348 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:12.267851  1553 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:12.377326  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:12.273615  7360 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.069s	user 0.020s	sys 0.027s
I20260709 13:02:12.195631  1635 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:02:12.380657  1601 connection.cc:256] RPC call timeout handler was delayed by -1.8199846830000297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
I20260709 13:02:12.176235  7339 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:12.384217  1634 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:12.390113  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:12.272085  7267 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:12.419788  1705 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.310672  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.364496  1629 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:12.297245  2350 rpcz_store.cc:269] 0709 13:02:07.535808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.536146 (+   338us) service_pool.cc:224] Handling call
0709 13:02:12.204330 (+4668184us) spinlock_profiling.cc:243] Waited 4 s on lock 0x7b5400762d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:12.204512 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9195893632}
I20260709 13:02:12.411454  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:12.375020  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.349298  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:12.421452  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 136075us
I20260709 13:02:12.320554  2346 rpcz_store.cc:276] 0709 13:02:11.145680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.145817 (+   137us) service_pool.cc:224] Handling call
0709 13:02:11.149102 (+  3285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.762390 (+613288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.319446 (+557056us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b5400756554. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:12.319778 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1296884608}
I20260709 13:02:12.369918  7317 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:12.426606  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.382406  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.390902  7096 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:12.370211  7193 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:12.392485  7577 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:12.432318  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:12.436569  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.437243  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.440483  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.444300  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:12.445660  2349 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:12.435055  7193 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:12.380817  7360 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.176s	user 0.021s	sys 0.027s
I20260709 13:02:12.364614  7535 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:12.419476  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:12.421674  1710 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:12.375865  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:12.428042  1709 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:12.465847  2348 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:12.430481  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:12.426812  7611 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:12.466960  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2951) took 1699 ms. Trace:
I20260709 13:02:12.467566  1708 rpcz_store.cc:276] 0709 13:02:10.767810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.861820 (+ 94010us) service_pool.cc:224] Handling call
0709 13:02:10.862338 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.998430 (+1136092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.466617 (+468187us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400247054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:12.466877 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2603913472}
I20260709 13:02:12.440127  7430 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.430296  7336 consensus_queue.cc:1059] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:02:12.474968  1710 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:02:12.476697  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 100916us
I20260709 13:02:12.431121  7516 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:02:12.498348  7447 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:12.432003  7306 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:02:12.483106  7611 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:12.487800  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1033) took 1123 ms. Trace:
I20260709 13:02:12.487946  7430 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.490562  7336 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:12.493691  7428 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:12.495500  7317 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.439965  7524 consensus_queue.cc:1059] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:02:12.516168  7360 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:12.503875  7336 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:12.505264  7447 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:12.554128  7447 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.561996  7590 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.565296  7590 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:12.574577  7611 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:12.638863  1457 catalog_manager.cc:5697] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 7 to 15, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 15 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:12.643551  1457 catalog_manager.cc:5697] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:12.646756  1457 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 8 to 11, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 11 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:12.649670  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:12.268257  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:12.652611  1457 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 14 to 17, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 17 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:12.654089  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 406815us
I20260709 13:02:12.664197  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:12.670879  1456 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 274) took 37123 ms (client timeout 15000 ms). Trace:
I20260709 13:02:12.239141  1554 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:12.677134  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.678465  1554 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
W20260709 13:02:12.680043  1557 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.013s	sys 0.035s
W20260709 13:02:12.680624  1557 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.013s	sys 0.035s
I20260709 13:02:12.680760  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.660748  1553 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:02:12.682195  1553 pending_rounds.cc:85] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 6
I20260709 13:02:12.682358  1558 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:12.682631  1553 pending_rounds.cc:107] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:02:12.683053  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.683149  1553 raft_consensus.cc:2889] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:12.683770  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.684249  1555 raft_consensus.cc:2376] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14. Re-sending same reply.
I20260709 13:02:12.685180  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.685317  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:12.685422  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:12.686312  1555 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.686699  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:12.686826  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.687554  1553 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.687727  1558 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:12.688822  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.673264  1457 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:12.692612  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.693154  1552 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:12.284897  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.701445  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.701733  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.702659  1552 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:12.703521  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:12.704201  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.693805  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.696972  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:12.705960  1557 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:02:12.706223  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.707290  1554 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:12.693843  1559 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:12.692270  1457 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:02:12.707325  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.704303  1560 raft_consensus.cc:2376] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14. Re-sending same reply.
I20260709 13:02:12.721815  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.723397  1554 raft_consensus.cc:2410] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:02:12.723651  1457 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:12.729861  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1063) took 1403 ms. Trace:
I20260709 13:02:12.730486  1558 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:12.726553  1630 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:12.726910  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:12.777383  1634 raft_consensus.cc:2410] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:02:12.392740  1636 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:12.780586  7096 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.781687  7096 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.727586  1631 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:12.375133  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:12.722151  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:12.657377  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:12.730368  1629 rpcz_store.cc:276] 0709 13:02:11.326031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.326178 (+   147us) service_pool.cc:224] Handling call
0709 13:02:11.364594 (+ 38416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.425378 (+1060784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.729153 (+303775us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b5400139cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:12.729594 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2299041024}
I20260709 13:02:12.413513  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.428200  1675 leader_election.cc:400] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12.
I20260709 13:02:12.830163  1675 leader_election.cc:403] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:02:12.720852  1456 rpcz_store.cc:269] 0709 13:01:35.546736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.546889 (+   153us) service_pool.cc:224] Handling call
0709 13:02:12.670428 (+37123539us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:02.240429 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:02.241458 (+  1029us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:02.241591 (+   133us) write_op.cc:435] Acquired schema lock
0709 13:02:02.241639 (+    48us) tablet.cc:662] Decoding operations
0709 13:02:02.243719 (+  2080us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:02.243857 (+   138us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:02.243899 (+    42us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:02:02.343170 (+ 99271us) tablet.cc:701] Row locks acquired
0709 13:02:02.343238 (+    68us) write_op.cc:260] PREPARE: finished
0709 13:02:02.343528 (+   290us) write_op.cc:270] Start()
0709 13:02:02.343741 (+   213us) write_op.cc:276] Timestamp: P: 1783602122343453 usec, L: 0
0709 13:02:02.343791 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 13:02:02.345217 (+  1426us) log.cc:844] Serialized 4775 byte log entry
0709 13:02:02.737346 (+392129us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:03.737072 (+999726us) op_driver.cc:464] REPLICATION: finished
0709 13:02:04.007444 (+270372us) write_op.cc:301] APPLY: starting
0709 13:02:04.023301 (+ 15857us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:04.944390 (+921089us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:04.944440 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:05.476078 (+531638us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:05.613468 (+137390us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 13:02:05.613544 (+    76us) write_op.cc:312] APPLY: finished
0709 13:02:05.613991 (+   447us) log.cc:844] Serialized 113 byte log entry
0709 13:02:05.614271 (+   280us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:05.614443 (+   172us) write_op.cc:454] Released schema lock
0709 13:02:05.614513 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:05.615247 (+   734us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:05.615282 (+    35us) write_op.cc:454] Released schema lock
0709 13:02:05.619502 (+  4220us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b180037ebe0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
Metrics: {"mutex_wait_us":9962,"reported_tablets":48,"spinlock_wait_cycles":19584,"child_traces":[["op",{"apply.queue_time_us":229654,"apply.run_cpu_time_us":36958,"apply.run_wall_time_us":1629815,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54287,"num_ops":12,"prepare.queue_time_us":976316,"prepare.run_cpu_time_us":68495,"prepare.run_wall_time_us":327720,"raft.queue_time_us":436825,"raft.run_cpu_time_us":18623,"raft.run_wall_time_us":189220,"replication_time_us":1357497,"spinlock_wait_cycles":74216576,"thread_start_us":1722827,"threads_started":4,"wal-append.queue_time_us":313726,"wal-append.run_cpu_time_us":42824,"wal-append.run_wall_time_us":2960809}]]}
I20260709 13:02:12.831283  7339 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:12.423830  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 44921.2us
I20260709 13:02:12.373941  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:12.843014  7516 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:02:12.734534  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.432894  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:12.372110  1703 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
W20260709 13:02:12.711347  1527 connection.cc:256] RPC call timeout handler was delayed by -2.316161013999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32120000000000004s for a response
I20260709 13:02:12.747399  1710 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:12.749229  1708 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:12.847314  1708 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:12.847961  1556 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.011s	sys 0.008s
W20260709 13:02:12.848623  1556 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.011s	sys 0.008s
I20260709 13:02:12.499361  2344 rpcz_store.cc:276] 0709 13:02:11.363918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.364348 (+   430us) service_pool.cc:224] Handling call
0709 13:02:12.310027 (+945679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.310960 (+   933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.487506 (+176546us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b540002bf50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:12.487721 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173650688}
W20260709 13:02:12.754091  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.850795  1561 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:12.759419  1674 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:12.852782  7427 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.853452  1709 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:02:12.771032  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:12.854370  2353 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:12.854498  7437 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 11. Current term is 12.
I20260709 13:02:12.772195  1705 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:12.773761  1557 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:12.724659  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:12.773887  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 104994us
I20260709 13:02:12.788857  1552 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:12.731036  1457 catalog_manager.cc:5697] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 9 to 12, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:12.785950  7362 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:12.802333  1559 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:12.808302  1553 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:12.786653  7096 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:12.539867  7535 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.816496  7480 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:12.817590  7332 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:12.818127  7306 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.541999  7360 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:12.819233  7559 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:12.786065  7193 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.867997  1457 catalog_manager.cc:5697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 4 to 11, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 11 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:12.825057  1555 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
W20260709 13:02:12.819762  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 438222us
W20260709 13:02:12.835856  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 178956us
I20260709 13:02:12.847065  1703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:12.507812  2317 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:12.751252  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.828876  7356 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:02:12.867089  1554 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:12.868848  7535 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:12.859540  7336 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:12.893294  7336 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.895965  7332 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.882809  7306 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:12.898337  7563 consensus_queue.cc:1175] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba0548b7d2a94e8598475a46c279fe0e" responder_term: 13 status { last_received { term: 13 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:12.900149  7306 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:12.902810  1704 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:02:12.903301  1552 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:12.881995  7521 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:12.904721  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:12.871086  7480 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.906879  7470 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:12.907369  7470 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:12.888154  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 463702us
W20260709 13:02:12.888885  1601 connection.cc:256] RPC call timeout handler was delayed by -1.614784683000039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
I20260709 13:02:12.882000  7421 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.881949  7193 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.849251  1708 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.896788  7332 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:12.860204  7362 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:12.907704  7470 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:12.922214  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4470703640000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2398s for a response
I20260709 13:02:12.819270  7532 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:02:12.914295  1703 raft_consensus.cc:606] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:12.877065  7559 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:12.928599  2353 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:12.853770  7427 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:12.873805  7418 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:12.933825  1555 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:12.939541  1559 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:12.941313  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:12.947229  1708 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.955359  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:12.957137  7301 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:12.957703  7496 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:02:12.962011  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 203091us
W20260709 13:02:12.969421  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 53821.9us
I20260709 13:02:12.938339  7476 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.973706  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.974121  7516 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.928220  7559 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.972704  1457 catalog_manager.cc:5697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:12.983690  1704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:12.923491  7421 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:12.938522  7418 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:12.985009  7362 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:12.986965  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 479150us
I20260709 13:02:12.991075  7096 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:12.995014  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:12.979594  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:13.014675  7256 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:13.015646  7332 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:13.016763  7480 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:13.022516  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:13.022192  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:13.030704  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.017215  7362 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:13.040899  7317 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:02:13.049726  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 74495.2us
I20260709 13:02:13.056948  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:13.052412  7332 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:13.030313  7476 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.065394  7332 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.089756  7332 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:13.090049  7427 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:13.091534  1457 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:13.096760  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:13.099119  7430 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.577s	user 0.012s	sys 0.005s
I20260709 13:02:13.061335  7516 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.100031  7430 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:13.100415  1704 raft_consensus.cc:606] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Received request to transfer leadership to TS 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:13.101013  1704 raft_consensus.cc:614] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:13.103518  7516 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:13.104732  7614 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:13.107194  7339 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:13.080309  7494 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:13.081149  7634 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:02:13.108207  7256 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:13.113075  7256 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:13.107709  7339 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:13.119338  7267 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.689s	user 0.010s	sys 0.010s
I20260709 13:02:13.121183  7267 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:13.068470  7418 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.088912  1708 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:02:13.067001  7301 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.127187  1708 pending_rounds.cc:85] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 9
I20260709 13:02:13.130295  1708 pending_rounds.cc:107] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 13.10
W20260709 13:02:13.121745  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.132203  1708 raft_consensus.cc:2889] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:13.113560  7256 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:13.113806  7476 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:13.145233  1457 catalog_manager.cc:5697] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 6 to 12, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 12 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:13.081792  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 197353us
I20260709 13:02:13.119294  7339 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.085601  7481 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:13.141916  1704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:13.141772  7336 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:13.136058  7421 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:13.113950  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 66520.5us
I20260709 13:02:13.129115  7301 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:13.146615  1629 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:13.153120  7402 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:13.151286  7481 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.154008  1704 raft_consensus.cc:606] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:13.185456  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 190639us
I20260709 13:02:13.193889  7447 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.122s
I20260709 13:02:13.197461  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:13.198411  2352 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:13.198865  2353 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:13.200971  2351 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:02:13.202484  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:13.202800  7481 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:13.204123  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.207222  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:13.213452  1674 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:13.214304  1634 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:13.208211  7532 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:13.215902  7532 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:13.216967  1704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:13.217928  7518 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:13.218835  7518 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:13.220201  7518 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.221557  7528 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.222318  7528 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.225015  7528 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:13.226259  7339 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.216979  7532 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.231493  7193 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:13.232013  7193 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:13.187086  1457 catalog_manager.cc:5697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:13.217893  1704 raft_consensus.cc:606] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:13.217914  1630 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:02:13.235064  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.135919  7267 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:13.242990  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61312.2us
I20260709 13:02:13.186600  7614 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.188897  7402 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:13.248189  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66886.4us
I20260709 13:02:13.253096  1457 catalog_manager.cc:5697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 4 to 11. New cstate: current_term: 11 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:13.257261  7360 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.259413  7360 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:13.258999  1457 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:13.261744  7418 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:13.261626  1457 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 9 to 14, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 14 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:13.262576  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:13.263670  7096 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:13.265793  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:13.265919  7096 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:13.269702  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:13.268738  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:13.270780  7332 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.275215  7362 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:13.275739  7332 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:13.271612  7096 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:13.279876  7096 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.280361  7096 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:13.275807  7362 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:13.281919  7362 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:13.282663  7421 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:13.285959  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 215707us
I20260709 13:02:13.269675  1675 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:13.290220  7373 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:13.292227  1675 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:13.233047  7193 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.294407  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:13.263794  1457 catalog_manager.cc:5697] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 7 to 11, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196), pending config changed. New cstate: current_term: 11 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:13.285418  7421 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.252257  7356 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:02:13.322392  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:13.324404  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:13.255977  7614 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.341704  1457 catalog_manager.cc:5697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:02:13.365080  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.321638  7421 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:13.312103  7373 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:13.347833  7193 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:13.375025  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 106503us
W20260709 13:02:13.375805  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 98879.2us
W20260709 13:02:13.374951  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 82847.7us
W20260709 13:02:13.369172  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 57790.6us
I20260709 13:02:13.328049  7301 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:13.379675  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.380321  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.384991  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.312778  7437 consensus_queue.cc:1059] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
W20260709 13:02:13.385448  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.386237   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:13.389179  7593 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:13.368674  7317 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.390255  1636 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:13.396505  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:13.399308  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:13.398193  2346 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:13.401300  2347 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:02:13.404733  7256 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.406317  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:13.406751  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44375.1us
I20260709 13:02:13.426283  2349 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:13.426383  7361 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:02:13.427556  1628 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:13.429183  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:13.430591  2352 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:13.430110  7402 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.360819  7528 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:02:13.432003  7402 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:13.432226  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:13.433919  1630 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:13.379366  1457 catalog_manager.cc:5697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 9 to 11, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 11 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:13.404318  2350 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:13.326622  7447 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:02:13.444936  7336 raft_consensus.cc:3055] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:13.444530  7317 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.445825  7336 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:13.446090  7614 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:13.447078  7336 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.448626  7317 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:13.449132  7336 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:13.453838  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:13.455303  7535 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.528s	user 0.016s	sys 0.000s
I20260709 13:02:13.455659  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:13.460423  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:13.461737  2349 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:13.457702   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:13.371768  7559 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.376500  7470 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.467957  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:13.472934  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:13.482201  2348 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
I20260709 13:02:13.482439  1707 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:02:13.487574  1629 raft_consensus.cc:2410] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 18 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:02:13.487684  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:13.488627  1635 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.496354  7573 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:13.470124  7494 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:13.497195  7573 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.499626  7573 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:13.494504  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:13.502787  1636 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:13.503804  7332 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:13.504285  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:13.509584  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:13.514259  7613 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:13.515133  7613 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.494223  7470 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:13.463443  7563 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.532819  1628 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:13.534767  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 61039.8us
I20260709 13:02:13.537268  1675 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:13.540639  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2279) took 1170 ms. Trace:
I20260709 13:02:13.541369  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:13.542671  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.535692  1457 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 16, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:02:13.546325  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 92640.7us
I20260709 13:02:13.541100  1707 rpcz_store.cc:276] 0709 13:02:12.370507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.736418 (+365911us) service_pool.cc:224] Handling call
0709 13:02:12.737189 (+   771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.482946 (+745757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.538369 (+ 55423us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b540036e2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:13.540579 (+  2210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1676302080}
I20260709 13:02:13.553964  1631 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:02:13.554394  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:13.555949  2349 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:02:13.559736  7447 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:13.562067  2318 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:13.563050  7447 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:13.564608  7559 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:13.565295  7559 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:13.565707  7559 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.566061  7559 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:13.552711  7563 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.569922  7563 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:13.589965  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52823.9us
I20260709 13:02:13.590233  1457 catalog_manager.cc:5697] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 4 to 11. New cstate: current_term: 11 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:13.533167  7614 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:02:13.533038  7496 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:02:13.593715  1635 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
W20260709 13:02:13.648648  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 86706.6us
I20260709 13:02:13.593433  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:13.651672  7476 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.594808  2349 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:13.630054  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:13.627200  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:13.619860  1636 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:13.626194  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:13.608354  7267 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:13.651047  7193 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:13.673424  7429 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:02:13.550374  7481 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.685717  7402 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:13.617965  1527 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:13.580893  7362 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:13.628041  7640 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:02:13.635612  7361 raft_consensus.cc:3055] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:13.558624  7613 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:13.652371  7476 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.633037  7256 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:13.722698  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:13.732182  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:13.721422  7573 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:13.733444  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:13.735333  2348 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:13.651674  7447 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:02:13.629176  7535 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:02:13.595036  7336 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 13:02:13.629527  7521 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 13:02:13.761422  1636 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:13.733536  1628 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:13.756004  1635 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:02:13.761058  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 167751us
I20260709 13:02:13.731884  7361 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:13.762758  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 328399us
I20260709 13:02:13.782424  2344 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:13.785112  7362 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:13.786397  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 168616us
I20260709 13:02:13.787791  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:13.787986  7361 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.788672  1527 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:13.790831  7301 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:13.790982  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:13.791275  7418 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:13.792208  7470 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:13.792843  7470 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.793296  7470 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.793654  7418 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:13.796530  7418 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:13.797983  7301 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:13.798112  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:13.798499  7301 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:13.799058  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.800166  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.800414  7535 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:13.800953  7535 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:13.802170  7481 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:13.802309  1674 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:13.802038  7535 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.803772  7535 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:13.804013  7481 raft_consensus.cc:2764] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:02:13.804144  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:13.756875  7532 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:13.828766  7532 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:13.829352  7532 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:13.773681  7573 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.830216  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:13.832068  7563 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:13.832630  7573 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:13.823287  7256 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:13.834478  7256 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:13.835041  7476 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:13.835264  7336 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:13.835747  7336 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:13.835959  7521 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:13.836421  7521 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:13.751255  7267 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.837388  7521 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.836782  7336 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.839025  7521 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:13.839093  7336 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:13.839891  7267 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:13.805397  7362 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:13.797323  7494 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:13.833362  7573 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:13.841563  7360 consensus_queue.cc:1175] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba0548b7d2a94e8598475a46c279fe0e" responder_term: 17 status { last_received { term: 17 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 17 index: 11 } }
W20260709 13:02:13.849943  7096 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.005s
W20260709 13:02:13.871626  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 68112.4us
I20260709 13:02:13.835073  7256 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:13.847486  7362 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:13.866153  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 361911us
I20260709 13:02:13.799553  7301 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:13.879478  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 81514.9us
I20260709 13:02:13.873088  7518 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.893332  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:13.900359  7470 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.917860  7301 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:13.918320  7421 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.868284  7573 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:13.867960  7496 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:13.926440  7256 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.878436  7096 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:13.880391  7362 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:13.938501  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44215.2us
W20260709 13:02:13.949934  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.946806  7418 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.952781  7373 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:13.953967  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:13.940505  7573 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:13.956634  1632 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:13.960642  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 72874.7us
I20260709 13:02:13.970108  1635 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:02:13.955904  7373 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.950673  7614 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:13.941903  7518 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:13.954128  7559 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:14.003885  7496 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:14.016314  7373 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:13.957608  7418 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:14.058732  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 406274us
W20260709 13:02:14.056617  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 429529us
W20260709 13:02:14.064848  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46477us
W20260709 13:02:14.079658  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 123500us
I20260709 13:02:14.024170  7614 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:14.105903  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 315830us
I20260709 13:02:14.068838  7496 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:14.109905  7332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.011s	sys 0.006s
W20260709 13:02:14.159921  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.106290  7650 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.200888  7655 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:02:14.203589  7362 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:14.204530  7559 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:14.033949  7317 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:02:14.206596  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:14.211158  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.264322  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 57827.8us
I20260709 13:02:14.215124  7096 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:14.223807  7651 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.128s	user 0.001s	sys 0.003s
I20260709 13:02:14.224079  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:14.224545  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 542547us
I20260709 13:02:14.223869  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:14.267452  2344 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:14.228665  7532 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.252437  7650 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:14.230173  7659 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.255846  7614 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:14.257864  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:14.216732  7332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.012s	sys 0.007s
I20260709 13:02:14.230098  7653 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.248708  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51681.9us
W20260709 13:02:14.264482  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 67979.8us
W20260709 13:02:14.270658  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 74099.6us
I20260709 13:02:14.274592  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:14.276731  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.281183  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:14.283630  2348 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:14.251655  7429 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.286320  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:14.290155  7447 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:14.292814  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:14.293633  7652 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:14.294389  7652 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:14.295540  7652 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.298075  7614 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:02:14.298750  7654 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:02:14.299510  7614 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:14.314608  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:14.316260  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:14.334033  1708 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:14.301281  7614 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:14.339215  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:14.339938  1602 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:14.302910  7656 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:02:14.343859  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:14.344501  7658 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:14.344772  1702 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:14.345160  7658 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.345741  7658 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:14.350261  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:14.338312  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:14.352195  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:14.356393  1709 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:14.276053  7659 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.367614  1707 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:14.370151  1706 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:02:14.371129  1704 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:14.317668  7332 raft_consensus.cc:2764] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:02:14.373099  1710 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:14.375609  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:14.380816  1703 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:14.383916  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:14.313795  7613 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:02:14.391507  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:14.292697  7653 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.384404  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:14.387421  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:14.342164  7336 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:02:14.399056  1705 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:14.394831  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:14.305610  7429 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.418618  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:14.419523  1707 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:14.390552  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:14.422250  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52487.8us
I20260709 13:02:14.423717  1675 leader_election.cc:400] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16.
I20260709 13:02:14.426201  1675 leader_election.cc:403] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:02:14.424013  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 158528us
I20260709 13:02:14.340934  7614 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:14.429790  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.428189  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:14.438645  1710 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:14.446678  1703 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:14.443499  1527 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:14.455535  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:14.399772  7518 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:14.466270  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:14.467293  2349 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:14.348688  7496 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:14.494069  7671 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:14.494426   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:14.494796  7671 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.495183  7671 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:14.497623  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 122131us
I20260709 13:02:14.442713  7332 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:14.449592  7418 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:14.496055  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:14.503065  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 146986us
I20260709 13:02:14.508242  7653 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:14.508935  7653 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:14.509749  7667 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:14.510491  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:14.511229  1701 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:14.513322  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:14.513867  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 62682.1us
I20260709 13:02:14.514369  1709 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:14.514889  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:14.524856  7657 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:02:14.526763  7659 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:14.531792  1703 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:14.537395  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:14.539800  7429 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:14.464090  7559 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 13:02:14.540921  7559 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:14.541126  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:14.541400  7559 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:14.545650  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:14.545953  7559 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.547407  7559 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:14.551551  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:14.509311  7653 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:14.497346  7362 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:14.538367  7521 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.699s	user 0.015s	sys 0.004s
I20260709 13:02:14.581229  1704 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:14.578130  7664 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:14.585928  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:14.586221  7306 raft_consensus.cc:993] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
W20260709 13:02:14.587181  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 46139.3us
I20260709 13:02:14.587893  7650 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:14.592002  1705 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:14.587198  7306 raft_consensus.cc:1081] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:02:14.600360  7267 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:14.600972  7267 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:14.601380  7267 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:14.601733  7667 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:14.609004  1709 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 11.
W20260709 13:02:14.586263  7676 log.cc:927] Time spent T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:02:14.571867  7653 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:14.621765  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:14.570080  7532 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:14.584506  7193 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16.
I20260709 13:02:14.632652  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:14.535487  7332 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.605971  7535 raft_consensus.cc:993] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:02:14.605615  7361 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:14.688345  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 128177us
W20260709 13:02:14.627539  7573 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.007s
W20260709 13:02:14.710085  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.707219  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 91188.2us
W20260709 13:02:14.640625  7373 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.572s	user 0.014s	sys 0.000s
W20260709 13:02:14.711591  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 122331us
I20260709 13:02:14.643935  7306 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.695462  1676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.531s (SENT) [suppressed 66 similar messages]
I20260709 13:02:14.642489  7447 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:14.715133  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3145) took 1094 ms. Trace:
I20260709 13:02:14.717564  7447 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.718184  7447 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:14.717677  1633 rpcz_store.cc:276] 0709 13:02:13.621130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.735235 (+114105us) service_pool.cc:224] Handling call
0709 13:02:13.759273 (+ 24038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.846701 (+ 87428us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:13.877113 (+ 30412us) log.cc:844] Serialized 14 byte log entry
0709 13:02:14.216059 (+338946us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.216145 (+    86us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:14.216238 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.714859 (+498621us) spinlock_profiling.cc:243] Waited 82.6 ms on lock 0x7b5400196c54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:14.714913 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.715070 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27778,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":45,"raft.queue_time_us":38983,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":201256576,"thread_start_us":280749,"threads_started":1,"wal-append.queue_time_us":404171}
I20260709 13:02:14.697569  7362 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.698567  7563 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.702350  7664 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:14.695168  7535 raft_consensus.cc:1081] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:02:14.712576  7532 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.716754  7676 log.cc:927] Time spent T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:02:14.712986  1636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 13:02:14.626883  7614 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:14.751235  1636 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:14.764221  1636 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:14.769968  7668 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.405s	user 0.004s	sys 0.000s
I20260709 13:02:14.777567  1710 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:14.713423  7573 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.793035  1703 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:14.727536  7306 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:14.737447  7664 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:02:14.798266  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 102838us
I20260709 13:02:14.763090  7532 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:14.804947  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.722383  7373 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:14.806885  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 75262.6us
I20260709 13:02:14.808043  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:14.812592  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 588953us
I20260709 13:02:14.738694  7362 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.824576  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:14.824887  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.825438  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:14.825817  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:14.826584  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:14.827402  7658 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.828604  1629 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:02:14.802438  7664 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:14.848500  1633 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:14.849035  1632 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:14.849790  1635 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:14.851068  7650 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:14.851686  7666 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.861094  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 131635us
I20260709 13:02:14.766072  7563 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:14.863379  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 131876us
I20260709 13:02:14.870894  7373 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:14.871718  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 93637.3us
I20260709 13:02:14.871801  7650 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.872714  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:14.835776  7614 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:14.871134  7301 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.930s	user 0.009s	sys 0.005s
I20260709 13:02:14.873889  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:14.878566  7694 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:14.879031  7694 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:14.879452  7694 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:14.885247  1635 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:14.888458  1632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 13:02:14.889114  1632 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:14.889544  1632 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:14.871078  7573 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:14.873693  7494 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:14.891096  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:14.892109  1633 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:14.894439  7593 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:14.894941  7593 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:14.895951  7593 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:14.897662  7532 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:14.898734  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:14.891314  7494 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:14.876900  7256 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.846990  7682 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:02:14.866731  7666 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:14.968266  7662 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.796038  7665 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 13:02:14.969184  7662 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:14.972172  7563 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:14.972568  7662 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:14.927022  7494 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:14.929529  7481 raft_consensus.cc:2764] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:02:14.985759  7362 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:14.939095  7573 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:14.941941  7682 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:14.942757  7658 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:14.989163  7682 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:14.991933  7682 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:14.944339  7653 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.946050  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:14.950493  1632 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.969177  7666 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:14.957716  7267 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.960175  7671 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.965829  7332 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:14.923775  7256 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.897409  7593 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:14.929199  7689 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:02:14.973372  7694 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:14.997458  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:14.997303  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:15.014739  1703 raft_consensus.cc:2410] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:14.996428  1632 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:15.015111  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 86559.4us
I20260709 13:02:14.997557  1636 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:15.000813  7697 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.113s	user 0.007s	sys 0.000s
I20260709 13:02:14.993453  7682 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:14.993813  7373 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.028450  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.035874  1636 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:15.039296  1710 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:15.038189  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:15.042587  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:15.055176  7694 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:15.056854  7687 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:02:15.060058  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:14.995929  7664 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.062379  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.063745  1710 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 13:02:15.066787  2318 connection.cc:256] RPC call timeout handler was delayed by -1.9082402989999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15510000000000002s for a response
W20260709 13:02:15.067313  2318 connection.cc:256] RPC call timeout handler was delayed by -1.8111775250000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2219s for a response
I20260709 13:02:15.069839  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.071103  1702 raft_consensus.cc:2410] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:02:15.071859  7650 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:15.073515  7256 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:15.050879  7373 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.078715  7373 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:15.081334  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:15.097621  2318 leader_election.cc:336] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.532s (SENT)
W20260709 13:02:15.129884  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.147564  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 60785.6us
I20260709 13:02:15.147912  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:15.098145  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.148629  1636 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
W20260709 13:02:15.085860  7559 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.519s	user 0.016s	sys 0.000s
I20260709 13:02:15.153903  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:15.148540  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.154698  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.156375  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:15.061916  7671 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:15.158582  1702 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:15.097208  7573 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:15.164108  7447 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.151573  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:15.158609  1630 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:15.178817  1635 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:15.189990  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 92407.4us
W20260709 13:02:15.192487  2318 leader_election.cc:336] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.591s (SENT)
I20260709 13:02:15.194180  2318 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.193372  1703 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:15.195742  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.196452  1710 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:15.202255  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:15.202472  7267 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:15.203285  1702 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:15.177769  7671 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:15.208274  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 173277us
W20260709 13:02:15.210425  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.217402  7361 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:15.232546  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:15.234781  7703 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 13:02:15.236016  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.238260  7653 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:15.240760  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 89316.7us
I20260709 13:02:15.243016  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3139) took 1776 ms. Trace:
I20260709 13:02:15.243520  1627 rpcz_store.cc:276] 0709 13:02:13.466250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.466445 (+   195us) service_pool.cc:224] Handling call
0709 13:02:13.482504 (+ 16059us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:13.482998 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.483028 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:13.483776 (+   748us) log.cc:844] Serialized 52 byte log entry
0709 13:02:13.762388 (+278612us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:13.782592 (+ 20204us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.238447 (+1455855us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b18003bb4e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda2b0d23c7 00007fda276e49a7 00007fda276f40bf
0709 13:02:15.238664 (+   217us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.238919 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.239011 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.242783 (+  3772us) raft_consensus.cc:1685] finished
0709 13:02:15.242805 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.242957 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5350,"prepare.queue_time_us":40377,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":131,"raft.queue_time_us":57997,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":205,"spinlock_wait_cycles":89027456,"thread_start_us":678645,"threads_started":3,"wal-append.queue_time_us":525164}
W20260709 13:02:15.245882  7686 log.cc:927] Time spent T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.435s	user 0.006s	sys 0.008s
I20260709 13:02:15.247042  7693 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:15.237387  7573 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:15.250146  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:15.251791  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:15.252466  7573 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:15.253269  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:15.253558  7573 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:15.254317  7660 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:15.254848  7660 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:15.255472  7660 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:15.254789  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:15.191532  7692 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:15.256721  1632 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:15.197824  7664 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:15.211814  7678 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:02:15.267683  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 75242.5us
W20260709 13:02:15.268345  2318 connection.cc:256] RPC call timeout handler was delayed by -1.7220895130000144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 13:02:15.268702  2318 connection.cc:256] RPC call timeout handler was delayed by -1.6725895130000481s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19730000000000003s for a response
W20260709 13:02:15.269006  2318 connection.cc:256] RPC call timeout handler was delayed by -1.5699895129999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20870000000000002s for a response
I20260709 13:02:15.271904  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:15.272929  1627 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:15.264314  7678 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.258097  7692 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.274112  7678 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:15.274185  7692 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.277841  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.278889  7559 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:15.279376  7559 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.279825  7559 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:15.240908  7096 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:15.264452  7667 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.259502  7267 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 13:02:15.298950  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.339783  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.342110  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64374.2us
I20260709 13:02:15.343227  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:15.350705  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.353960  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.354295  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.354846  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.355772  1705 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:15.354362  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:15.354872  1702 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:15.361917  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.362069  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.364189  1627 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:15.364497  1707 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:15.364954  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.365662  1701 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:15.365244  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.366919  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.367566  1708 raft_consensus.cc:2376] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17. Re-sending same reply.
I20260709 13:02:15.367866  2344 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:15.368625  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:15.370646  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.371315  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.371411  1710 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:15.368440  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:15.374064  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.374692  1634 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:15.375774  7650 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:15.351298  7653 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:15.377143  2346 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:15.378481  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:15.379125  2350 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:15.364373  7652 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:15.381798  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.381922  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:15.382649  1705 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:15.382869  2353 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:15.383973  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:15.384778  2349 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:15.370697  7496 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:15.351660  7563 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.384087  7332 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:15.388829  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:15.392836  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 138125us
I20260709 13:02:15.394531  1704 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:15.397310  7332 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.400319  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.400621  7332 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:15.403901  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:15.420027  2348 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:15.423837  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:15.425110  1705 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:15.416108  7267 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:02:15.426648  7470 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:15.426532  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:15.423894  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:15.431213  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:15.432638  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:15.440501  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:15.441453  1558 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:15.446036  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:15.450307  7671 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:15.416509  7693 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.452121  2345 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:15.416615  7659 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:02:15.453775  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:15.401571  7563 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:15.454674  1559 raft_consensus.cc:2410] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:15.429177  7573 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:02:15.455819  7563 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:15.457306  1556 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:02:15.457877  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:15.453274  7662 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:15.459059  1703 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:15.458827  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:15.459213  1554 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:15.410236  7650 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:15.458554  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:15.429203  7496 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.480501  2351 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:15.482640  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:15.483505  2345 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:15.487385  1527 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:15.488209  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 118571us
I20260709 13:02:15.491597  7301 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:15.493439  7301 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:15.464200  7593 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:15.499477  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.500144  1704 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:15.500496  2349 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
W20260709 13:02:15.500041  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 137005us
I20260709 13:02:15.477237  7661 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:15.473443  7650 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.506027  1602 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:15.520359  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.496661  7563 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.539597  2352 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:15.476521  7256 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:02:15.499014  7593 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.544926  1527 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:15.548182  2318 connection.cc:256] RPC call timeout handler was delayed by -1.1024539599999912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 13:02:15.548657  2318 connection.cc:256] RPC call timeout handler was delayed by -1.0452990279999881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
I20260709 13:02:15.517376  7650 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.515620  7661 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:15.555321  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.565146  2350 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:15.513080  7301 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
W20260709 13:02:15.568362  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45654.7us
I20260709 13:02:15.569235  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.570457  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.550171  7593 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:15.578094  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.578896  1704 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:15.557231  7714 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:15.576648  7653 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:15.599128  7661 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.556041  7563 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.605418  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:15.583947  7267 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:15.585343  7402 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:02:15.638780  7715 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:15.638274  7694 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:02:15.586447  7666 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:02:15.639878  1704 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:15.662043  1552 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:15.662410  1702 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:02:15.655398  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:15.656487  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:15.693593  1555 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:15.657934  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:15.694470  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 124053us
I20260709 13:02:15.669533  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:15.695768  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.651077  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:15.697775  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.702176  1558 raft_consensus.cc:2410] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:02:15.697866  1633 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:02:15.709913  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.661032  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:15.757567  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 272454us
I20260709 13:02:15.757700  1710 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.762247  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:15.766119  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:15.766733  7653 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:15.766841  1709 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:15.651062  7714 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:15.768285  1709 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:15.670761  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:15.672528  7682 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.638s	user 0.014s	sys 0.004s
I20260709 13:02:15.772559  1704 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:15.766809  1710 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:15.805716  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:15.770707  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.807468  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:15.650964  7563 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:15.669852  7267 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:15.670235  7715 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:15.668088  7659 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:15.816290  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:15.752646  7193 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.815495  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:15.826197  1558 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 11: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:02:15.826790  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.674742  7661 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:15.646718  7664 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.133s
I20260709 13:02:15.752223  7301 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:15.844327  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:15.875563  1702 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:15.843533  1709 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.877609  1709 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:15.875959  1702 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:15.881467  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 110614us
I20260709 13:02:15.882751  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:15.806957  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 301059us
I20260709 13:02:15.804708  7714 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.643831  7720 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:15.807171  7362 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.846213  1557 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:15.806886  7532 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.778851  7447 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:15.820218  7559 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.540s	user 0.020s	sys 0.003s
I20260709 13:02:15.757970  7256 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:02:15.831873  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 95044.8us
I20260709 13:02:15.906457  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:15.836999  1555 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:15.804183  7653 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.907728  7653 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:15.908228  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:15.909324  1555 raft_consensus.cc:2410] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:15.910344  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:15.911944  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:15.913949  7716 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:02:15.916664  7715 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.919044  1709 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:15.922186  7306 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:15.923584  7429 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:15.924088  7429 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:15.925225  7429 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:15.871057  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 326211us
W20260709 13:02:15.864351  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 94673.9us
I20260709 13:02:15.845229  7659 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:15.690230  7660 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:15.758217  7666 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.118s
W20260709 13:02:15.899150  7679 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:02:15.898990  1702 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:15.925810  7721 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:15.926712  7267 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.867538  1704 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:15.899112  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:15.932798  1553 raft_consensus.cc:2368] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17.
I20260709 13:02:15.927639  7724 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:15.935726  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:15.677042  7692 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.939079  1702 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:15.889433  7496 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:15.926615  7532 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.958467  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:15.958834  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:15.962515  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 72650.9us
I20260709 13:02:15.927452  7559 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:15.896306  7714 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:15.948767  7720 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:15.960652  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:15.963409  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:15.971607  1703 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:15.873826  7193 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:15.954875  7721 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:15.977833  7743 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:15.977152  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:15.956758  7724 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:15.991467  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:15.991987  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:15.993405  1552 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:15.983649  7692 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:15.985761  7718 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:15.969587  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:15.983346  7532 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:15.995807  1526 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:15.994793  1558 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:15.934182  7362 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:15.968986  7429 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:16.001546  1555 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:02:16.001058  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45869.6us
I20260709 13:02:15.998068  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:15.991279  7721 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:16.020570  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:16.003506  7332 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.602s	user 0.005s	sys 0.009s
I20260709 13:02:16.024709  1704 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:16.026497  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:15.995486  7724 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:16.022166  7743 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:16.031417  7660 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:16.032485  7667 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.746s	user 0.017s	sys 0.000s
I20260709 13:02:16.032985  7667 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:16.033632  7729 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:16.034173  7732 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:16.034413  7729 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:16.034832  7732 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:16.035540  7729 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.035782  7732 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.036346  7653 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.037591  7559 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.039062  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:16.039232  7653 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.039778  1560 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:16.017504  7301 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.046741  7301 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.047349  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:16.013859  7470 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:16.022373  7678 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.748s	user 0.014s	sys 0.000s
I20260709 13:02:16.000584  7720 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:16.014468  7718 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:16.011734  7532 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.053435  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:16.062124  1702 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:16.063598  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:15.991005  7659 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:16.070205  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.053615  7743 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:16.077548  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:16.077715  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:16.053508  7737 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:16.087770  7737 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:16.088351  7737 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:16.095432  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 77059.7us
I20260709 13:02:16.029172  7332 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.065336  7682 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.075629  7470 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.120797  1560 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:16.142372  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:16.061576  7678 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.065294  7532 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.150635  1555 raft_consensus.cc:2368] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14.
I20260709 13:02:16.151451  1557 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:16.160835  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:16.155087  7653 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.155316  7682 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.171895  7682 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.173794  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:16.174958  7684 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:16.117319  7193 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:16.175896  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:16.177018  7684 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:16.120626  7650 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.002s
I20260709 13:02:16.178159  7650 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.181854  7496 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:16.183297  7593 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.597s	user 0.007s	sys 0.005s
I20260709 13:02:16.183866  7559 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.183826  7593 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.184675  7559 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.185338  7470 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.185885  7653 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.188220  7559 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.176148  7332 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.182749  7496 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:16.193899  7741 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:02:16.197038  1708 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:16.188223  7470 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:16.198027  7678 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:16.189172  7718 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.166365  7301 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.205752  1558 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 19: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:16.116961  7743 raft_consensus.cc:697] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:16.196445  7724 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.202337  7532 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:16.222425  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 175134us
I20260709 13:02:16.121089  7715 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:16.235044  1557 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:16.234099  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:16.203334  7559 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.201285  7496 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:16.188306  7653 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.238258  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:16.076084  7662 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:16.239331  1556 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
W20260709 13:02:16.251722  7740 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:02:16.258980  1675 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:16.263976  1554 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:16.237619  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:16.266819  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:16.279191  7665 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:16.280035  7665 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:16.280308  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:16.281021  7665 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.284053  7660 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:16.284797  7660 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:16.286091  7660 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.291836  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:16.297506  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:16.298062  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:16.298388  1633 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:16.298775  1630 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:16.300674  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:16.302806  1632 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:16.224318  7715 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.305359  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:16.306502  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:16.306273  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:16.307799  1635 raft_consensus.cc:2410] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:02:16.305663  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:16.244650  7650 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:16.234938  7720 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.278451  7662 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.314481  7717 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:02:16.315876  7717 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:02:16.316061  7722 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.454s	user 0.005s	sys 0.000s
I20260709 13:02:16.316385  7717 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:16.307175  1627 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:16.317725  7362 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:16.305925  7692 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:16.320942  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:16.321657  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 62793.1us
W20260709 13:02:16.321800  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 90924.7us
I20260709 13:02:16.290470  7653 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:02:16.323863  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:16.324836  7663 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.325045  1561 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:16.325433  7663 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.325779  7402 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:16.325896  7743 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.326423  7664 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:16.332087  7715 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.332716  1560 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:16.321919  1558 raft_consensus.cc:2393] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:16.266692  7421 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 11 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:16.338784  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:16.275271  7573 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:16.337929  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:16.349045  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.364952  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.365581  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.349247  1635 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:16.353412  2318 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:16.354785  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:16.244601  7559 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.356514  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:16.300632  7684 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:16.371948  1561 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:16.372627  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:16.362879  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:16.290256  7532 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:02:16.370646  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:16.374320  1557 raft_consensus.cc:2410] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:02:16.361316  1633 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:16.378332  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:16.380591  1558 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:16.381407  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:16.382073  1633 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:16.382273  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.379194  1627 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:16.384012  7521 raft_consensus.cc:993] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: : Instructing follower ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:02:16.361949  7301 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.405776  1675 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:16.407827  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:16.409847  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:16.410714  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:16.411005  1556 raft_consensus.cc:2410] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:16.411564  1559 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:16.412128  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:16.413802  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:16.409978  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56757us
I20260709 13:02:16.414534  1552 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:16.415465  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:16.415934  1552 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:16.418154  2318 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:16.419045  1556 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:16.417301  7694 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:16.426026  1633 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:16.379829  7650 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:16.375268  7573 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:16.436960  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:16.441133  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 70648.1us
I20260709 13:02:16.443717  1704 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:16.402706  7681 raft_consensus.cc:1081] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Signalling peer ba0548b7d2a94e8598475a46c279fe0e to start an election
I20260709 13:02:16.317385  7717 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.447014  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:16.447840  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:16.454456  1702 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:16.447561  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:16.448191  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:16.450503  1555 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:02:16.443686  7447 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:16.457692  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:16.457947  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:16.458389  1706 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:16.458775  1707 raft_consensus.cc:2410] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:02:16.422458  7301 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.460587  1552 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:16.461786  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:16.462361  1710 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:16.463032  7301 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:16.464836  1552 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:02:16.421458  7336 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:16.465866  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:16.466375  1552 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:16.418344  7593 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.468348  1552 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:02:16.468982  1552 pending_rounds.cc:85] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 6
I20260709 13:02:16.470108  1552 pending_rounds.cc:107] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:02:16.470422  1552 raft_consensus.cc:2889] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:16.471946  7573 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.474893  7663 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:16.475035  7559 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.471935  1552 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:16.447228  7421 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:16.465883  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:16.481096  1552 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:02:16.453977  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 173726us
I20260709 13:02:16.481593  1633 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:16.482676  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:16.491786  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44857.6us
I20260709 13:02:16.493681  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:16.495713  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:16.441666  7684 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:16.445214  7193 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:16.481146  7724 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:16.500923  7752 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:02:16.502102  7714 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.000s
I20260709 13:02:16.503321  7751 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:16.503608  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:16.504321  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:16.482452  7421 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:16.511731  7759 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:02:16.516789  7717 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:16.518108  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:16.512212  7721 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.523780  1558 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
W20260709 13:02:16.535020  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 52457.8us
I20260709 13:02:16.536466  1627 raft_consensus.cc:2410] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 13:02:16.537251  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 119191us
I20260709 13:02:16.537163  1556 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:16.538497  1633 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:02:16.513243  7429 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.500s	user 0.003s	sys 0.007s
I20260709 13:02:16.437865  7687 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:02:16.439358  7650 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.536576  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:16.589461  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:16.593345  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 63246.2us
I20260709 13:02:16.553232  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:16.553170  1710 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:16.543361  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:16.540477  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:16.526716  7336 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:16.602545  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 62121.6us
I20260709 13:02:16.603791  7593 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:16.554803  1702 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:16.586454  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:16.605464  1558 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:16.605543  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:16.610494  1676 leader_election.cc:336] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.531s (SENT)
I20260709 13:02:16.720283  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:16.721563  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:16.722656  2353 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
W20260709 13:02:16.722769  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 117328us
W20260709 13:02:16.721628  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 111168us
I20260709 13:02:16.596680  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:16.603859  7650 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:16.543519  7532 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.187s
I20260709 13:02:16.733456  1636 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:16.543834  7667 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:16.558528  7421 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:16.540846  7681 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:16.741916  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.564863  7494 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:16.583863  7559 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:16.680029  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 77145.2us
I20260709 13:02:16.754642  1675 leader_election.cc:400] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17.
I20260709 13:02:16.543454  7694 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.201s
W20260709 13:02:16.779253  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.604349  7593 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.605042  7687 consensus_queue.cc:1243] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:02:16.544566  7751 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.797938  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2304) took 1036 ms. Trace:
I20260709 13:02:16.799052  1705 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:02:16.803151  7721 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.802958  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:16.544205  7714 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.719216  7193 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:16.731544  7301 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:16.742648  7336 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:16.606223  7733 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.197s
I20260709 13:02:16.839052  7660 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:16.849452  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.886271  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:16.887048  2348 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:16.878278  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:16.888985  2351 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:16.855993  7593 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:16.845647  1708 rpcz_store.cc:276] 0709 13:02:15.761657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.761836 (+   179us) service_pool.cc:224] Handling call
0709 13:02:16.791616 (+1029780us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b5400224554. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:16.797823 (+  6207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":996611840}
I20260709 13:02:16.754675  7689 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.760370  7753 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:16.839545  7559 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.906993  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:16.810884  7681 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:16.865940  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 280000us
I20260709 13:02:16.908780  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:16.868420  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:16.869565  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 264623us
W20260709 13:02:16.880035  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.910260  2353 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:16.881398  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:16.911041  1554 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:16.883242  2349 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:16.583287  1561 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:16.885710  1560 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:16.852357  1636 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:16.853999  7762 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.893396  1527 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:16.861104  7751 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.901199  2348 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:16.871253  7301 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:16.925166  7760 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:02:16.925442  7714 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.926213  7714 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:16.926637  7714 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:16.927002  7766 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:02:16.927080  7714 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:16.926882  7763 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:02:16.894546  7660 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.929670  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:16.931402  7660 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.931644  7715 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:16.925248  7301 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:16.921398  7665 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:16.933976  7665 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:16.934355  7665 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:16.941258  7684 raft_consensus.cc:2764] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:02:16.942639  7301 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.877873  7193 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:16.895233  7496 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.943971  7496 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:16.944408  7193 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.924204  7762 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.947263  7762 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:16.948603  7496 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.950059  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:16.918193  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:16.953428  7714 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.954684  7301 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:16.957062  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.959367  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.962982  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.964646  2349 raft_consensus.cc:3055] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:16.965121  2349 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:16.966030  2349 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:16.967084  2349 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:16.975627  7429 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:16.976387  7429 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:16.976910  7770 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:16.898597  7733 consensus_queue.cc:1243] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:02:16.987416  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 69338.1us
I20260709 13:02:16.988471  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:16.988759  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 78588.4us
I20260709 13:02:16.984020  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:16.982203  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:16.997025  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:16.998245  2348 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:17.002350  2351 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:17.003070  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:17.004596  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 407971us
I20260709 13:02:16.913727  7681 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:17.007052  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:16.990748  1675 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:17.011365  1527 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:17.011965  2349 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:02:17.012461  2349 pending_rounds.cc:85] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 9
I20260709 13:02:17.012734  2349 pending_rounds.cc:107] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:02:16.916736  7765 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:17.013007  2349 raft_consensus.cc:2889] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:17.014271  1554 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:16.990833  7660 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:17.113895  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 102661us
W20260709 13:02:17.123703  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 133056us
W20260709 13:02:17.114487  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 73612.8us
I20260709 13:02:17.124912  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:17.114871  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 96563.6us
I20260709 13:02:17.003175  7532 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:17.126863  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 130209us
I20260709 13:02:17.134127  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:17.137189  1601 leader_election.cc:400] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14.
I20260709 13:02:17.012805  7652 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:17.004489  7096 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:17.007143  7494 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:17.008347  7687 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:17.154688  7687 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:17.146117  7762 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:02:17.155301  7687 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:17.003428  7714 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:17.142400  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:17.113168  7765 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.158505  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:17.159104  1631 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:02:17.159888  7765 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:16.982571  7429 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.162226  7429 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:17.162580  1627 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:17.163920  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:17.166260  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 161128us
I20260709 13:02:17.015163  7336 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:17.015254  7667 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:17.026489  7733 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:17.148895  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:17.026851  7661 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:17.190371  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:17.152459  7573 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.145195  7678 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.148795  7096 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:17.006770  7753 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.142s
I20260709 13:02:17.123965  7720 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:17.133406  7660 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.145129  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.200335  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.200613  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:17.123310  7763 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:16.918991  7689 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.130223  7532 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.196447  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:17.197443  2351 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:02:17.199833  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:17.212288  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 80831.6us
I20260709 13:02:17.199570  7678 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:02:17.197073  7096 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:17.207453  7737 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.118s	user 0.014s	sys 0.004s
W20260709 13:02:17.249960  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.198045  7301 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:17.253777  2344 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:17.254820  2353 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:17.212301  7532 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:17.254208  7737 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.194532  7733 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.211766  7762 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:17.287135  7762 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.266919  7559 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.266656  7667 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.323782  1628 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:17.268685  7689 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.270053  7678 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:17.270795  7421 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.379238  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:17.260855  7301 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.380010  2346 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 12.
I20260709 13:02:17.276350  7096 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.277093  1631 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:17.280228  7665 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.281786  7660 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.238507  7720 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:17.400707  2347 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:17.213100  7715 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:02:17.402945  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:17.265800  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 128734us
I20260709 13:02:17.322104  2348 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
W20260709 13:02:17.345885  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.346590  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:17.272565  7593 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.394306  1552 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:17.394760  7775 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.385902  7573 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.403223  1631 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:17.254329  7763 raft_consensus.cc:697] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:17.406726  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:17.409895  2346 raft_consensus.cc:2410] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:02:17.292011  7687 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.411777  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:17.411938  1527 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:17.422952  7763 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.424510  7593 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:17.425192  7653 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:17.425782  7653 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:17.427048  7687 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.426729  7653 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.427804  2349 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:17.431787  7661 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:17.432310  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:17.432404  7661 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:17.433032  2345 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:17.433854  7661 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:17.435609  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:17.436236  2347 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:17.437232  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:17.437666  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:17.438289  1554 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:17.438220  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:17.440348  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:17.444162  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:17.448557  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:17.448282  2346 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:02:17.442408  7650 consensus_meta.cc:287] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.680s	user 0.005s	sys 0.006s
W20260709 13:02:17.462909  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 210919us
I20260709 13:02:17.463269  1554 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:17.449502  7494 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:17.461241  7717 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.873s	user 0.011s	sys 0.002s
I20260709 13:02:17.414486  7660 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:02:17.474231  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:17.442835  7267 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:02:17.420905  7775 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.420181  7573 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.459967  7778 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:17.494802  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:17.467738  7784 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
W20260709 13:02:17.503958  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.508903  2318 connection.cc:256] RPC call timeout handler was delayed by -1.0359934079999675s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 13:02:17.512060  2318 connection.cc:256] RPC call timeout handler was delayed by -1.0332934079999632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
I20260709 13:02:17.473814  7494 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:17.460001  7650 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:17.524981  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 71546.3us
W20260709 13:02:17.530253  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.544677  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:17.553411  2345 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:17.521148  7494 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:17.558997  1559 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:17.518178  7733 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.561112  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:17.555215  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:17.565745  1556 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:17.474864  7720 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.541738  7737 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.554903  7782 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:02:17.588515  1599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.013s	sys 0.010s
W20260709 13:02:17.588896  1599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.014s	sys 0.010s
I20260709 13:02:17.515861  7778 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.557246  7421 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.591885  2318 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:17.559201  7783 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:02:17.599474  7267 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:17.568459  7447 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:17.613823  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:17.620272  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:17.610143  1556 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:17.652587  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:17.655875  1556 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:17.659679  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 164140us
I20260709 13:02:17.587368  7782 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:17.591176  7762 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:17.700776  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.703042  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:17.643143  7786 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:02:17.705581  7797 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:17.704023  7593 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.611194  7733 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:17.610204  7301 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:17.645031  7765 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:17.583302  7792 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.131s
W20260709 13:02:17.717617  7787 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:02:17.717833  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:17.643708  7193 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:02:17.718618  1561 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:17.719710  7447 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:17.719996  7778 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:17.720794  7193 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.721554  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:17.722061  7783 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:17.722474  7783 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:17.723278  7783 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.719755  1561 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:17.611737  7429 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.611052  7791 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.607214  7687 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.100s
I20260709 13:02:17.664440  7663 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.740290  7663 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.707751  7797 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.741752  7797 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:17.743499  7663 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.647344  7665 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.721313  7193 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.745505  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.745437  7791 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.645807  7717 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:17.746783  7717 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.747381  7193 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:17.747992  7193 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.748178  1701 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:17.748566  7193 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.727347  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:17.729694  1634 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:17.731796  7336 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.539s	user 0.007s	sys 0.003s
I20260709 13:02:17.732013  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:17.749867  1702 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:17.750022  7336 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:17.750816  7336 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.751214  7336 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.751605  7494 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.752560  7193 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:17.753153  7336 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:17.760607  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:17.761016  1703 raft_consensus.cc:2368] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14.
I20260709 13:02:17.761080  1702 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:17.732246  7795 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.748144  7791 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.762187  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:17.761902  7795 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.764333  7795 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.744918  7573 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:17.735222  7782 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.765614  7782 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:17.766611  7681 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.738s	user 0.010s	sys 0.000s
I20260709 13:02:17.767014  7681 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.768635  1527 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:17.610108  7796 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 13:02:17.737713  7800 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.771212  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.644680  7662 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:17.772007  7764 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.621s	user 0.005s	sys 0.008s
I20260709 13:02:17.772043  1629 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 19: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:17.724292  7733 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.772974  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.753933  7650 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.773631  1629 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:17.774173  7650 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:17.774439  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.774699  7650 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:17.775815  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:17.775663  7650 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.776153  1630 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:17.776862  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:17.755646  7783 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:17.777341  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.777799  1710 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:17.778215  1630 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:17.649102  7267 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.779419  1629 raft_consensus.cc:2410] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:02:17.757231  7532 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.714911  7593 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.761395  1709 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:17.734373  7762 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:17.730319  7614 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:17.770998  7800 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.749985  7765 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:17.733467  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:17.817350  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 83990.7us
I20260709 13:02:17.817672  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:17.819046  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:17.749128  7717 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.759421  7775 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:17.780701  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.820722  1635 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:17.824604  1635 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:17.791257  1556 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
W20260709 13:02:17.738256  7780 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 13:02:17.792404  1632 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:17.793432  1630 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 19. Current term is 20. Ops: []
W20260709 13:02:17.806283  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 186083us
I20260709 13:02:17.831265  7593 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.831029  7765 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.831910  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:17.832782  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:17.834246  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:17.834398  7336 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:17.834887  7336 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.835296  7336 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:17.835747  1701 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:02:17.836714  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:17.837198  1701 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:17.837687  7805 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.838068  1707 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: [13.11-14.12]
I20260709 13:02:17.768949  7681 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:17.816200  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:17.826210  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:17.842195  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:17.827376  1635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448"
dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:54746
I20260709 13:02:17.843261  1635 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:02:17.807113  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.812539  7614 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:17.814256  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2896) took 1532 ms. Trace:
I20260709 13:02:17.840193  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:17.842828  1703 raft_consensus.cc:2376] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14. Re-sending same reply.
W20260709 13:02:17.831368  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.847334  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.822288  7715 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:17.835011  1706 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:17.820946  7733 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.844790  1635 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:17.816934  7494 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.845809  1628 rpcz_store.cc:276] 0709 13:02:16.278053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.278173 (+   120us) service_pool.cc:224] Handling call
0709 13:02:17.809996 (+1531823us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400344254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:17.810131 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2332675840}
I20260709 13:02:17.849165  1707 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:17.843463  1709 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:17.838176  7805 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.851135  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3046) took 1333 ms. Trace:
I20260709 13:02:17.823478  7762 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:17.844276  7796 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:17.858587  1552 rpcz_store.cc:276] 0709 13:02:16.513408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.517657 (+  4249us) service_pool.cc:224] Handling call
0709 13:02:17.846107 (+1328450us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b54003c0fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:17.846222 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1867810560}
I20260709 13:02:17.861373  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:17.841079  7267 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:17.861982  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:17.868072  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:17.870333  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:17.872112  1709 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:17.875881  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.876453  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:17.876879  1628 raft_consensus.cc:2410] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:02:17.917212  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:17.918115  7532 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:17.921622  7429 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.922170  7429 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:17.924706  7429 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:17.850610  7193 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.926530  7762 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:17.965412  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.918161  1632 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:17.945788  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:17.966346  1706 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:17.962306  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.925642  7790 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:02:17.953042  1635 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.971060  1710 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:02:17.975935  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:17.841130  7797 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:17.980432  1635 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:17.980832  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 106196us
I20260709 13:02:17.878357  7662 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:17.984858  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:17.985291  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:17.985734  1703 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:17.964869  7532 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:17.969306  7267 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.014181  1632 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:18.015569  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2353) took 1237 ms. Trace:
I20260709 13:02:18.016687  1709 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:18.019452  1560 rpcz_store.cc:276] 0709 13:02:16.763681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.842532 (+ 78851us) service_pool.cc:224] Handling call
0709 13:02:16.861936 (+ 19404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.913926 (+ 51990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.000428 (+1086502us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b54003377d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:18.000612 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54090624}
I20260709 13:02:18.020061  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:18.020642  1709 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:17.981768  7429 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.030699  1709 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:18.036964  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.037834  1709 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:18.038780  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.039361  1709 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:18.040222  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.046334  1703 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
W20260709 13:02:18.046356  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 169977us
I20260709 13:02:18.048918  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:18.054847  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:18.056102  1703 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:18.002978  7694 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:02:18.006232  7804 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.977707  7614 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.058279  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.058766  1674 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:18.018117  7662 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.027707  7267 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.053867  7789 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:02:18.116217  1703 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:18.055294  7532 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.030957  7806 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:02:18.123327  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:18.036064  7429 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.126577  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.132143  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.132781  1704 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:18.138146  7806 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:18.138592  7806 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:18.139392  7806 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.140798  7806 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.126513  7807 raft_consensus.cc:2764] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:02:18.083392  7614 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.162184  1703 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:18.162393  1708 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:18.123029  7663 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:18.163796  1703 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:18.164055  7663 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.165038  1703 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:18.166225  7663 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:18.169919  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:18.170574  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.171274  2345 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:18.172200  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.172441  7662 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:18.173381  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:18.179106  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.179288  7664 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:18.179845  7814 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:18.057236  7762 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.182195  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:18.182984  7687 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.184319  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:18.179919  2346 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:18.188504  2346 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:18.159994  1709 pending_rounds.cc:85] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 9
I20260709 13:02:18.190587  1709 pending_rounds.cc:107] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:02:18.190876  1709 raft_consensus.cc:2889] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:18.193451  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:18.196957  7804 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:18.198372  7804 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:18.198626  7812 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:18.198928  7804 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.197465  7470 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:18.203531  7805 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:18.205756  7429 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:18.182212  7267 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:18.215694  1702 raft_consensus.cc:2410] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:18.216328  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:18.220904  1636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2892) took 1944 ms (client timeout 1689 ms). Trace:
W20260709 13:02:18.221489  1636 rpcz_store.cc:269] 0709 13:02:16.276637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.276781 (+   144us) service_pool.cc:224] Handling call
0709 13:02:18.220710 (+1943929us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b540010e0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:18.220833 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980033792}
I20260709 13:02:18.223593  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.225034  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:18.226599  1708 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:18.227000  1708 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:18.227839  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:18.228816  1703 raft_consensus.cc:2368] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17.
I20260709 13:02:18.151526  7694 consensus_queue.cc:1243] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:02:18.183521  7687 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.214458  7532 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:18.235641  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.219094  7664 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:18.245021  1601 leader_election.cc:400] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14.
I20260709 13:02:18.246240  1601 leader_election.cc:403] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:02:18.237999  1708 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.247802  1708 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:18.243924  1703 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:18.254529  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.255750  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:18.262501  7791 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14.
W20260709 13:02:18.247792  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 123393us
I20260709 13:02:18.245591  7694 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:18.243484  7532 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:18.268831  1701 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:18.243116  7447 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:02:18.236881  7267 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:18.255446  7662 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:02:18.265568  7664 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:18.282800  1708 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:18.277563  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.283829  1701 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:18.283697  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:18.274492  7729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.774s	user 0.024s	sys 0.017s
I20260709 13:02:18.284379  1708 raft_consensus.cc:2368] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17.
I20260709 13:02:18.272415  7797 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:18.285794  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:18.287707  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.276348  7782 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:18.293080  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.294342  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.280189  7336 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.282303  7573 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 13:02:18.306437  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.295567  7820 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:02:18.302939  1708 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:18.283339  7694 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:18.302500  7821 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:02:18.329985  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.330296  7687 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:18.286230  7729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.786s	user 0.024s	sys 0.018s
I20260709 13:02:18.334287  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.336596  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:18.293649  7791 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:18.338274  7791 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:18.344852  7769 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.086s	user 0.000s	sys 0.012s
I20260709 13:02:18.346177  7791 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:18.347770  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1048) took 1180 ms. Trace:
I20260709 13:02:18.348176  2353 rpcz_store.cc:276] 0709 13:02:17.167743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.167854 (+   111us) service_pool.cc:224] Handling call
0709 13:02:17.169200 (+  1346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.256243 (+ 87043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.347505 (+1091262us) spinlock_profiling.cc:243] Waited 74.2 ms on lock 0x7b540055ffd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:18.347689 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170746112}
I20260709 13:02:18.337795  1708 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:18.355399  1701 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:18.358137  7336 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:18.331276  7790 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:18.359702  7790 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.361099  1709 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:18.361955  7790 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:18.362412  1709 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:02:18.362691  7772 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.105s	user 0.006s	sys 0.014s
I20260709 13:02:18.363581  1709 raft_consensus.cc:1217] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 10.10->[11.11-14.12]   Dedup: 11.11->[14.12-14.12]
I20260709 13:02:18.364535  1709 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: [11.11-14.12]
I20260709 13:02:18.365789  1709 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 13:02:18.293257  7751 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:18.328069  7532 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.367981  7751 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.368546  7532 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:18.330415  7782 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:18.325414  7800 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:18.371744  7694 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.382048  7804 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.383472  7096 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:18.384369  7804 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:18.390628  7659 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:18.367187  7336 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.373114  7751 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:18.321923  7664 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:18.417626  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 201375us
W20260709 13:02:18.418865  7825 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:02:18.422752  7782 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.402055  7336 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.433740  7718 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:18.434193  7718 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:18.438891  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 66451.9us
I20260709 13:02:18.435091  7718 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.451809  1708 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:18.437845  7336 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:18.455014  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.458997  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.443611  7267 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:18.468017  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.473308  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:18.474795  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:18.475596  1557 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:18.477820  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.478552  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:18.461278  7718 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.483964  1557 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.482560  1709 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
W20260709 13:02:18.495827  7776 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.341s	user 0.012s	sys 0.007s
I20260709 13:02:18.498279  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3037) took 2536 ms. Trace:
I20260709 13:02:18.498741  1555 rpcz_store.cc:276] 0709 13:02:15.961754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.457122 (+495368us) service_pool.cc:224] Handling call
0709 13:02:16.799742 (+342620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.800059 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:16.800458 (+   399us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.007508 (+207050us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.007586 (+    78us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:17.007648 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.007741 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.495005 (+1487264us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:18.498078 (+  3073us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b540038a4d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:18.498216 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39680,"prepare.queue_time_us":466,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":3588,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":96308,"spinlock_wait_cycles":1012951296,"thread_start_us":206518,"threads_started":1,"wal-append.queue_time_us":353699}
I20260709 13:02:18.503364  7667 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:18.504045  7667 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:18.504060  2345 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:18.506278  7496 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.506623  7729 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:18.506917  7496 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:18.511730  7785 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.952s	user 0.000s	sys 0.002s
I20260709 13:02:18.523609  1601 leader_election.cc:400] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17.
I20260709 13:02:18.524780  1601 leader_election.cc:403] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:18.454430  7829 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:02:18.527312  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:18.530954  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1453) took 1356 ms. Trace:
I20260709 13:02:18.531360  2350 rpcz_store.cc:276] 0709 13:02:17.174846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.175017 (+   171us) service_pool.cc:224] Handling call
0709 13:02:17.201239 (+ 26222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.201749 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.201795 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.202377 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.437739 (+235362us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:17.437778 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.437846 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.511143 (+1073297us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:18.513321 (+  2178us) raft_consensus.cc:1685] finished
0709 13:02:18.513343 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.530889 (+ 17546us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5897,"spinlock_wait_cycles":4695424,"thread_start_us":178224,"threads_started":1,"wal-append.queue_time_us":244102}
I20260709 13:02:18.473645  7336 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:02:18.538631  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:18.539204  1634 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:18.541523  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:18.541805  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:18.542096  1636 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:18.543005  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:18.543787  1632 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.543119  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:18.546710  1601 leader_election.cc:400] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17.
I20260709 13:02:18.528580  7267 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:18.549656  1601 leader_election.cc:403] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:18.551184  1632 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:18.534858  7832 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:18.538470  7421 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.594913  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1452) took 1446 ms. Trace:
I20260709 13:02:18.594702  7421 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.595288  2349 rpcz_store.cc:276] 0709 13:02:17.148708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.157558 (+  8850us) service_pool.cc:224] Handling call
0709 13:02:17.157865 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.430357 (+272492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.594659 (+1164302us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54006310d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:18.594854 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620354304}
I20260709 13:02:18.596392  7267 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17.
I20260709 13:02:18.596812  7421 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:18.598068  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.600929  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.601554  2350 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:18.561787  1636 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:18.554273  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:18.563351  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:18.557026  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:18.608919  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51931.3us
I20260709 13:02:18.564289  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 2986) took 2550 ms. Trace:
I20260709 13:02:18.569434  1634 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:18.570194  2351 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:18.570680  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.572264  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2903) took 1431 ms. Trace:
I20260709 13:02:18.612138  1633 rpcz_store.cc:276] 0709 13:02:17.140882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.168666 (+ 27784us) service_pool.cc:224] Handling call
0709 13:02:17.196778 (+ 28112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.323920 (+127142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.323967 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.324028 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:17.324082 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.571970 (+1247888us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54001383d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:18.572023 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.572205 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291816320}
I20260709 13:02:18.610253  1705 rpcz_store.cc:276] 0709 13:02:16.014860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.022505 (+  7645us) service_pool.cc:224] Handling call
0709 13:02:16.798626 (+776121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.854027 (+ 55401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.564066 (+1710039us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b5400224550. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:18.564208 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1783649024}
I20260709 13:02:18.608150  1627 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:18.573354  7689 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:18.575428  7652 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:18.577942  7762 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:18.578289  1703 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
W20260709 13:02:18.579728  7830 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:18.581439  7494 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.543335  7829 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:18.548858  7756 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:18.586793  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3022) took 2899 ms. Trace:
I20260709 13:02:18.587294  7832 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:18.588676  7758 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 2.118s	user 0.010s	sys 0.015s
I20260709 13:02:18.590240  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:18.562288  7496 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:18.505309  7667 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.593968  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:18.594127  7820 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:18.539642  7729 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:18.615334  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.614776  7784 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.556279  1557 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:18.623234  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.629359  7829 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:18.639739  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.626078  7652 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:18.553956  7660 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:02:18.592546  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.643471  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.643487  2353 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEARNER]: Advancing to term 15
W20260709 13:02:18.630849  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.629590  7756 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.612716  7751 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17.
I20260709 13:02:18.616281  2351 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:18.615911  7718 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:18.623788  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.627162  7267 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:02:18.635210  7820 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:18.629760  1559 rpcz_store.cc:276] 0709 13:02:15.686315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.412198 (+725883us) service_pool.cc:224] Handling call
0709 13:02:16.412543 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.560316 (+1147773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.585143 (+1024827us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400208354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:18.585331 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2635958656}
I20260709 13:02:18.616312  7689 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:18.627015  7661 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:02:18.674604  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.680625  1552 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:02:18.663682  7667 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:18.681977  7829 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.683223  7829 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.641121  7729 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.686087  7689 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.686605  1553 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: [14.10-16.12]
I20260709 13:02:18.687186  7689 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:18.687299  7729 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:18.625272  7743 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:02:18.628010  7096 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:02:18.655018  1555 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 13:02:18.707949  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 153515us
I20260709 13:02:18.624519  7791 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:02:18.669776  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:18.710134  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:18.651340  7718 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:18.714028  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 64631.6us
I20260709 13:02:18.702224  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:18.715775  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:18.654186  1705 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:18.650014  7756 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.690258  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:18.720170  1553 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:18.689800  2353 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:18.635881  7494 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.704820  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:18.650429  7660 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:18.708523  1554 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:02:18.711711  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:18.667678  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:18.716506  1555 raft_consensus.cc:2368] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17.
I20260709 13:02:18.642241  7784 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.664377  7732 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.714311  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:18.730108  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 62517.5us
I20260709 13:02:18.726718  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:18.730970  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:18.732167  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:18.741235  7784 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:18.743758  7835 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:18.751616  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:18.717479  7532 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:02:18.722420  7718 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:18.695089  7832 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:18.754735  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:18.766657  1629 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:18.777791  1553 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:18.753798  7614 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.787361  7831 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:18.787221  7784 consensus_queue.cc:1059] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:18.787927  7831 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.788319  7831 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:18.788443  7784 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:18.788909  7784 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:18.792979  2318 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:18.794485  7694 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:18.794878  7694 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.795478  7694 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:18.796829  7840 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.109s	user 0.001s	sys 0.001s
I20260709 13:02:18.797139  7831 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.794858  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:18.801975  7832 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:18.802332  7832 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.802673  7832 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:18.804023  7694 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.805646  7714 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:18.806373  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.806185  7714 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.807188  1705 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:18.807922  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:18.808343  7714 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:18.809252  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54630.9us
I20260709 13:02:18.811618  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.812000  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.812243  1710 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:18.813679  1702 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:18.813254  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:18.815011  7842 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:18.815243  1704 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:18.815593  7842 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:18.815850  2351 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:18.808894  7714 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.816933  7714 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.818331  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:18.819016  2349 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:18.820109  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.820969  2348 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:18.821851  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.821914  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.822618  2353 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:18.823555  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.823659  1705 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:18.824231  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:18.824846  1708 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:18.825111  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:18.825678  2347 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:18.826082  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:18.745512  7820 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.828851  2351 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
W20260709 13:02:18.807227  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.830837  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:18.830607  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:18.837755  1705 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.833284  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:18.833501  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:18.755071  7838 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:18.850589  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:18.757540  7447 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:18.760870  7801 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:18.843547  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.844360  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:18.849272  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.853003  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:18.765851  7421 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:18.862358  2349 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:18.839375  7666 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.849303  7521 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:18.800762  7831 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:18.891959  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:18.910993  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:18.911288  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:18.806242  7694 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:18.933758  1708 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:18.905447  1554 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:18.912972  2352 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:02:18.935320  1705 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:18.869616  7664 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:18.928912  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:18.867264  7807 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:18.942587  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:18.761636  7732 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:18.858798  7838 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.868870  7801 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:18.816716  7842 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.903689  7447 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:18.891597  7844 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.915102  7729 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:18.914909  7421 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:18.963733  1561 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:18.941570  7834 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:18.940809  7664 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:18.949079  7792 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:18.944742  7821 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:18.973975  7421 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.974887  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:18.975354  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:18.915552  7666 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:18.981562  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:18.982389  1555 raft_consensus.cc:2376] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17. Re-sending same reply.
I20260709 13:02:18.983498  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:18.970649  2352 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:18.955991  7532 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:18.946038  7807 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:18.970333  7664 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:18.900624  7784 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:19.014430  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.969134  7729 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:18.952133  7838 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:18.987265  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:18.940969  7660 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:19.030750  1554 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:19.012027  7666 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:19.012486  1553 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.001266  7807 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.988996  7821 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.987536  7792 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:19.017460  7532 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:19.057979  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:19.071597  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 238371us
I20260709 13:02:19.072654  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:19.075155  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.081569  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.071853  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 161607us
I20260709 13:02:19.094937  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:19.029714  7714 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:19.102339  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:19.103592  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:19.055713  7784 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.107697  7714 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:19.108870  2318 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:02:19.109601  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.110466  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:19.118582  1708 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:02:19.118999  1708 pending_rounds.cc:85] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 11
I20260709 13:02:19.119231  1708 pending_rounds.cc:107] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:02:19.119451  1708 raft_consensus.cc:2889] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:19.094347  7814 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:19.125425  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 449191us
W20260709 13:02:19.140173  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.120040  1553 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:19.014134  7801 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:19.146648  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 217799us
I20260709 13:02:19.110165  7856 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.118853  7663 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:19.148169  7831 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:19.143513  7814 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:19.122408  7652 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:19.176260  7831 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:19.176045  7856 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:19.185169  7666 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:19.186044  7666 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:19.186501  7856 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:19.187384  7666 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:19.188678  7732 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:19.189381  7732 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:19.193148  7801 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:19.197645  7532 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:19.201640  7854 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:19.202255  7854 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.202667  7854 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:19.176687  7663 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:19.070137  7821 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:19.109730  7447 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:19.223165  7821 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:19.227039  7756 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:19.227517  7821 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:19.228380  7821 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:19.149420  1602 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:19.230806  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 81472.3us
I20260709 13:02:19.232052  2347 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:19.232508  1602 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:19.145357  7521 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:19.133561  7662 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.103s
W20260709 13:02:19.231954  7667 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.550s	user 0.001s	sys 0.010s
W20260709 13:02:19.267539  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.235385  7714 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:19.267444  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:19.232694  7862 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:19.209244  7096 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 13:02:19.288585  7857 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:19.295869  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:19.148475  7729 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:19.298219  7521 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:19.299294  7663 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:19.300698  7832 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:19.302173  7814 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:19.318945  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 208543us
W20260709 13:02:19.303329  7811 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.185s	user 0.003s	sys 0.000s
I20260709 13:02:19.304549  7652 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:19.306489  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:19.307962  7756 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:19.255342  7849 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:19.311705  1560 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:19.268324  7743 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:19.318965  7689 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.001s
I20260709 13:02:19.323963  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:19.348801  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:19.350307  2351 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: [15.15-15.15]
I20260709 13:02:19.268823  7659 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:02:19.183017  7792 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:19.321516  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:19.362336  2350 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:19.351645  1558 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:19.364423  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:19.361609  7832 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:19.368701  1558 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:19.273077  7850 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 13:02:19.375463  7807 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:19.369851  7842 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:19.379312  7718 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.004s
I20260709 13:02:19.379333  7807 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:19.379846  7718 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:19.324225  7831 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:19.393095  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:19.393500  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:19.394050  7869 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:19.394192  7470 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:19.395560  7869 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:19.398394  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:19.359848  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:19.312885  7857 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.358902  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:19.374402  7829 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.691s	user 0.012s	sys 0.000s
I20260709 13:02:19.401412  1560 raft_consensus.cc:2376] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17. Re-sending same reply.
W20260709 13:02:19.420008  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.428397  7614 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:19.428267  7714 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:02:20.588894  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.459733  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:19.960927  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:19.642604  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.656366  1632 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:19.707176  1675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.771s (SENT) [suppressed 12 similar messages]
I20260709 13:02:19.932289  1633 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:02:20.588706  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2381) took 1891 ms (client timeout 1956 ms). Trace:
I20260709 13:02:19.961678  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3056) took 1301 ms. Trace:
I20260709 13:02:19.705322  1631 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:19.429705  1554 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
W20260709 13:02:20.594345  1560 rpcz_store.cc:269] 0709 13:02:18.696954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.344261 (+647307us) service_pool.cc:224] Handling call
0709 13:02:20.588639 (+1244378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.949563  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1078) took 1129 ms. Trace:
I20260709 13:02:19.501380  2347 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:19.485113  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1016) took 1660 ms. Trace:
W20260709 13:02:19.867444  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 351340us
I20260709 13:02:19.734941  1630 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:19.486543  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:18.070945   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:19.647663  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:20.626562  1636 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:20.530705  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:20.436520  1553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3060) took 1761 ms (client timeout 1771 ms). Trace:
I20260709 13:02:20.631306  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:20.603367  2351 rpcz_store.cc:276] 0709 13:02:18.820592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.906292 (+ 85700us) service_pool.cc:224] Handling call
0709 13:02:18.906678 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.350873 (+444195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.949154 (+598281us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54006446d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:19.949465 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1019525376}
I20260709 13:02:20.632125  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:20.593715  1561 rpcz_store.cc:276] 0709 13:02:18.660663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.669362 (+  8699us) service_pool.cc:224] Handling call
0709 13:02:19.961437 (+1292075us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400389ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:19.961615 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":571085184}
I20260709 13:02:20.605801  1559 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
W20260709 13:02:20.629261  1553 rpcz_store.cc:269] 0709 13:02:18.675171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.974480 (+299309us) service_pool.cc:224] Handling call
0709 13:02:20.436284 (+1461804us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x7b54002ff654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:20.436439 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85128064}
W20260709 13:02:20.634860  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2358) took 1948 ms (client timeout 2150 ms). Trace:
W20260709 13:02:20.635373  1559 rpcz_store.cc:269] 0709 13:02:18.686702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.460355 (+773653us) service_pool.cc:224] Handling call
0709 13:02:20.634806 (+1174451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:20.633927  1704 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:20.634771  1634 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:20.699455  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3022) took 1239 ms. Trace:
I20260709 13:02:20.696105  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:20.717280  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:20.718022  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2355) took 2040 ms (client timeout 2185 ms). Trace:
W20260709 13:02:20.718003  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 1.01101e+06us
W20260709 13:02:20.717468  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 112374us
I20260709 13:02:20.591552  7614 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:20.720646  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.604687  1707 rpcz_store.cc:276] 0709 13:02:17.824160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.865884 (+ 41724us) service_pool.cc:224] Handling call
0709 13:02:17.866311 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.877274 (+ 10963us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.877318 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.877889 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.999784 (+121895us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:18.084132 (+ 84348us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.237012 (+152880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.237112 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.302693 (+1065581us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:19.472468 (+169775us) raft_consensus.cc:1685] finished
0709 13:02:19.484829 (+ 12361us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.485013 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31941,"prepare.queue_time_us":115875,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":137783,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":26128,"spinlock_wait_cycles":7308160,"thread_start_us":243028,"threads_started":3,"wal-append.queue_time_us":117851}
I20260709 13:02:20.720911  7614 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:20.639950  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3020) took 1193 ms. Trace:
W20260709 13:02:20.721125  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.721146  1553 raft_consensus.cc:2410] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:02:19.837706  7782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.391s	user 0.025s	sys 0.034s
I20260709 13:02:19.754678  7663 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:19.479522  7858 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:19.671950  7811 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
W20260709 13:02:19.752593  7873 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:02:20.720125  1555 rpcz_store.cc:269] 0709 13:02:18.677647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.530218 (+1852571us) service_pool.cc:224] Handling call
0709 13:02:20.717960 (+187742us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:19.754169  7718 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:20.721704  1708 rpcz_store.cc:276] 0709 13:02:19.397244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.459359 (+ 62115us) service_pool.cc:224] Handling call
0709 13:02:20.590198 (+1130839us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:19.753376  7871 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
W20260709 13:02:20.077144  7792 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.012s	sys 0.005s
I20260709 13:02:20.718400  1704 rpcz_store.cc:276] 0709 13:02:19.397990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.399371 (+  1381us) service_pool.cc:224] Handling call
0709 13:02:20.636964 (+1237593us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:20.085723  7831 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.006s	sys 0.007s
W20260709 13:02:20.764142  7831 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.272s	user 0.012s	sys 0.018s
I20260709 13:02:20.192543  7751 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.735661  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:20.569016  7801 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.375s	user 0.005s	sys 0.006s
I20260709 13:02:20.736766  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:20.781046  1560 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:20.737211  1631 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:20.737393  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:19.471575  7855 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 2, Time since last communication: 1.294s
I20260709 13:02:20.738368  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:20.784984  1633 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:20.738484  7862 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.636417  7336 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:20.739318  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:20.789999  1630 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:19.472003  7470 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 1.269s
I20260709 13:02:19.495344  7664 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:20.750542  2353 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1079) took 1929 ms (client timeout 1859 ms). Trace:
W20260709 13:02:20.793089  2353 rpcz_store.cc:269] 0709 13:02:18.820891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.823268 (+  2377us) service_pool.cc:224] Handling call
0709 13:02:20.750262 (+1926994us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b54006310d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:20.750429 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1760961408}
I20260709 13:02:20.751860  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2322) took 2566 ms. Trace:
I20260709 13:02:20.775616  7861 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.722025  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2360) took 2034 ms. Trace:
I20260709 13:02:20.796263  1553 rpcz_store.cc:276] 0709 13:02:18.687201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.695664 (+2008463us) service_pool.cc:224] Handling call
0709 13:02:20.721968 (+ 26304us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:20.395856  7841 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.570s	user 0.002s	sys 0.000s
W20260709 13:02:20.571005  7784 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.453s	user 0.008s	sys 0.004s
I20260709 13:02:19.872730  7832 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.758687  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:20.237732  7666 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.000s	sys 0.017s
I20260709 13:02:19.803673  7870 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:19.364037  7743 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:20.757906  7792 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.382s	user 0.016s	sys 0.018s
I20260709 13:02:20.737867  7806 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:20.761593  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3059) took 2086 ms. Trace:
I20260709 13:02:20.814675  1555 rpcz_store.cc:276] 0709 13:02:18.674914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.760554 (+2085640us) service_pool.cc:224] Handling call
0709 13:02:20.760922 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.761142 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.761179 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:20.761238 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:20.761284 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.761355 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.761522 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:20.722043  7782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.278s	user 0.045s	sys 0.064s
W20260709 13:02:19.579020  7763 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.148s	user 0.013s	sys 0.005s
I20260709 13:02:19.697707  7869 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.820727  2349 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:20.769539  1632 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:02:20.770336  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:20.268501  7494 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:19.628026  7659 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:20.778509  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:20.782471  1561 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:20.789734  7336 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:20.737843  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.811057  7666 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.161s	user 0.004s	sys 0.031s
I20260709 13:02:19.468315  7842 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 1.316s
I20260709 13:02:19.506292  7854 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:19.459051  7857 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:20.794067  1701 rpcz_store.cc:276] 0709 13:02:18.185092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.365038 (+179946us) service_pool.cc:224] Handling call
0709 13:02:18.808079 (+443041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.808245 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.808272 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.808325 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:18.808354 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.808401 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.751633 (+1943232us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b540036e2d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:20.751783 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1018271616}
W20260709 13:02:20.779105  7839 raft_consensus.cc:670] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } [suppressed 10 similar messages]
W20260709 13:02:20.802376  7843 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.911s	user 0.000s	sys 0.010s
I20260709 13:02:20.757124  7663 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:20.802999  1634 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:20.819536  7870 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:20.783744  7662 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.810781  1559 raft_consensus.cc:2410] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 13:02:20.822686  1561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2371) took 2128 ms (client timeout 2378 ms). Trace:
W20260709 13:02:20.917011  1561 rpcz_store.cc:269] 0709 13:02:18.694442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.641194 (+1946752us) service_pool.cc:224] Handling call
0709 13:02:20.822634 (+181440us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:20.782863  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2369) took 2088 ms (client timeout 2229 ms). Trace:
I20260709 13:02:20.792423  7762 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.823982  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2359) took 2129 ms. Trace:
I20260709 13:02:20.919729  1557 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:19.497138  7838 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:20.778262  7751 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:20.860855  1631 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: [13.9-14.10]
I20260709 13:02:20.860939  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:20.804628  7718 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.824872  7806 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:20.894356  7844 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.925881  7844 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:20.926790  7664 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:20.928093  7844 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:20.928989  7806 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:20.930368  7751 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:20.860546  7659 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:20.932586  7838 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:20.932756  7659 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:20.822369  7763 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.391s	user 0.022s	sys 0.025s
I20260709 13:02:20.814281  7559 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.933887  7559 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:20.923278  7762 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:20.935150  1552 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:20.936180  7559 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:20.817291  7821 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:20.892561  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:20.936713  7762 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:20.937783  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:20.938488  1630 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:20.938903  7663 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:20.939316  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:20.940253  1630 raft_consensus.cc:2410] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:02:20.941032  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:20.894984  2316 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.018s	sys 0.016s
W20260709 13:02:20.941684  2316 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.018s	sys 0.017s
I20260709 13:02:20.942981  7870 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:20.943540  7870 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:20.944337  2353 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:20.944829  2344 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:20.945523  7806 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.945719  2348 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:20.922787  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:20.946048  7806 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:20.947113  1631 raft_consensus.cc:2410] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:20.947623  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:20.947911  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:20.948254  7890 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.948834  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:20.948954  2351 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:02:20.817095  7832 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:20.896733  7743 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:20.897697  7888 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:02:20.722616  7858 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:20.793740  7862 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:20.916544  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:20.917914  1560 rpcz_store.cc:269] 0709 13:02:18.693966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.640689 (+1946723us) service_pool.cc:224] Handling call
0709 13:02:20.782801 (+142112us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:20.919487  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3065) took 2177 ms (client timeout 2434 ms). Trace:
I20260709 13:02:20.864522  7792 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.947738  7857 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:20.919901  1555 rpcz_store.cc:276] 0709 13:02:18.686960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.815683 (+2128723us) service_pool.cc:224] Handling call
0709 13:02:20.815943 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.816241 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.816265 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:20.816304 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:20.816341 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.816389 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.816498 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101984}
I20260709 13:02:20.863688  7782 raft_consensus.cc:476] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Not starting pre-election -- already a leader
I20260709 13:02:20.923185  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1466) took 2409 ms. Trace:
I20260709 13:02:21.062938  2345 rpcz_store.cc:276] 0709 13:02:18.509741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.509889 (+   148us) service_pool.cc:224] Handling call
0709 13:02:18.510277 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:18.527011 (+ 16734us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:18.527058 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:18.532836 (+  5778us) log.cc:844] Serialized 52 byte log entry
0709 13:02:18.649939 (+117103us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:18.650001 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.650093 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:20.801839 (+2151746us) spinlock_profiling.cc:243] Waited 8.61 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:20.906089 (+104250us) raft_consensus.cc:1685] finished
0709 13:02:20.906110 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.919356 (+ 13246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16261,"raft.queue_time_us":21249,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":4925,"spinlock_wait_cycles":19809152,"thread_start_us":64024,"threads_started":1,"wal-append.queue_time_us":216203}
I20260709 13:02:21.038995  1630 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 19: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:21.038573  7890 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.039544  7751 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:20.806483  7861 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:21.041273  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.067395  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.042899  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 61122.4us
I20260709 13:02:20.925949  7336 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:20.944000  7870 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:21.046083  1559 rpcz_store.cc:269] 0709 13:02:18.742344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.648508 (+1906164us) service_pool.cc:224] Handling call
0709 13:02:20.919430 (+270922us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:20.919507  7732 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:21.048527  7879 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:21.050635  7854 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:21.054092  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2917) took 2512 ms. Trace:
I20260709 13:02:20.827386  7869 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:20.924238  7855 raft_consensus.cc:2764] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:02:21.045800  1634 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:21.069450  1555 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:21.071251  1635 rpcz_store.cc:276] 0709 13:02:18.542054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.542160 (+   106us) service_pool.cc:224] Handling call
0709 13:02:18.542451 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.554205 (+ 11754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.554233 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.554794 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.638295 (+ 83501us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:18.638335 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.638405 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:20.395121 (+1756716us) spinlock_profiling.cc:243] Waited 8.44 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:21.053756 (+658635us) raft_consensus.cc:1685] finished
0709 13:02:21.053780 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.054010 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11372,"raft.queue_time_us":11996,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":38930,"spinlock_wait_cycles":19418368,"thread_start_us":68380,"threads_started":1,"wal-append.queue_time_us":144703}
I20260709 13:02:21.075553  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2380) took 2378 ms. Trace:
I20260709 13:02:21.075886  1555 rpcz_store.cc:276] 0709 13:02:18.696716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.067601 (+2370885us) service_pool.cc:224] Handling call
0709 13:02:21.067872 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.075287 (+  7415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.075500 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:21.076927  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:21.077553  1630 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:21.078783  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:21.072841  1631 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:21.079646  1635 raft_consensus.cc:2410] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:02:21.080263  1558 raft_consensus.cc:2368] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17.
I20260709 13:02:21.080538  1555 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:21.087433  7858 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:20.922859  7662 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.092361  1561 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:02:21.081213  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 938) took 2059 ms. Trace:
I20260709 13:02:21.097333  1555 rpcz_store.cc:276] 0709 13:02:19.021354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.079967 (+2058613us) service_pool.cc:224] Handling call
0709 13:02:21.080235 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.080908 (+   673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.081150 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:21.101629  1632 raft_consensus.cc:2410] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:21.040903  7718 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.087788  7862 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:21.075062  7336 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:21.048595  7806 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:21.102557  1559 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:02:21.129467  1634 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:21.065428  7792 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.070221  7732 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:21.048007  7659 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:02:21.145272  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:21.247884  2345 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:21.184751  1631 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
W20260709 13:02:21.229310  7884 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:02:21.229827  7890 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:21.231168  7878 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:02:21.233068  7869 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:21.233856  7882 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:02:21.234207  7832 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:21.235230  7874 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:02:21.236756  7718 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:21.241669  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3066) took 2226 ms. Trace:
I20260709 13:02:21.242919  1634 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:21.317240  1559 rpcz_store.cc:276] 0709 13:02:19.015722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.069289 (+2053567us) service_pool.cc:224] Handling call
0709 13:02:21.098248 (+ 28959us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.241352 (+143104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.241608 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:21.319094  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.005s	sys 0.016s
I20260709 13:02:21.234246  7861 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:21.304749  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 534491us
W20260709 13:02:21.306674  7804 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.842s	user 0.001s	sys 0.001s
W20260709 13:02:21.319576  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.005s	sys 0.016s
I20260709 13:02:21.319572  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 940) took 2297 ms. Trace:
I20260709 13:02:21.327337  1634 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:21.327049  1559 rpcz_store.cc:276] 0709 13:02:19.022098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.318589 (+2296491us) service_pool.cc:224] Handling call
0709 13:02:21.318988 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.319198 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.319230 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:21.319279 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:21.319311 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.319391 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.319514 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:21.327392  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.330283  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 941) took 2307 ms. Trace:
I20260709 13:02:21.330979  1559 rpcz_store.cc:276] 0709 13:02:19.022423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.328591 (+2306168us) service_pool.cc:224] Handling call
0709 13:02:21.329013 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.329245 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.329287 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:21.329349 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:21.329419 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.329770 (+   351us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.330210 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:21.312214  7714 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.006s	sys 0.000s
W20260709 13:02:21.317519  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.010s	sys 0.006s
I20260709 13:02:21.341336  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:21.343199  1559 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:21.344110  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 942) took 2321 ms. Trace:
I20260709 13:02:21.344548  1559 rpcz_store.cc:276] 0709 13:02:19.022671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.332014 (+2309343us) service_pool.cc:224] Handling call
0709 13:02:21.342932 (+ 10918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.343794 (+   862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.344040 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:21.347362  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:21.321190  1705 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.002s	sys 0.013s
I20260709 13:02:21.278465  7659 consensus_queue.cc:1243] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer 95fc6a4cf5874b7eb91280dffcd3b3fb log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:02:21.332564  7714 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.006s	sys 0.001s
W20260709 13:02:21.349177  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.010s	sys 0.006s
W20260709 13:02:21.349525  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.012s
W20260709 13:02:21.349628  1705 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.002s	sys 0.014s
W20260709 13:02:21.326596  7804 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.001s	sys 0.001s
I20260709 13:02:21.351222  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:21.353866  1630 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:02:21.355605  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.000s	sys 0.012s
I20260709 13:02:21.357550  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3026) took 1958 ms. Trace:
W20260709 13:02:21.359022  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.359611  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.010s	sys 0.005s
W20260709 13:02:21.362887  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.010s	sys 0.005s
I20260709 13:02:21.339879  7732 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.363039  7762 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:21.364312  7857 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:21.354367  7842 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.365371  7842 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.358670  7858 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.354977  7899 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.376612  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 943) took 2353 ms. Trace:
I20260709 13:02:21.377043  7899 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.378458  7892 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:21.374465  7861 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:02:21.379699  7899 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:21.379231  7892 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.380762  7854 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:21.381649  7892 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:21.369828  7855 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:02:21.382365  7855 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:21.382705  7855 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:21.383479  7855 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.384642  7855 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:21.385878  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:21.387010  7896 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:02:21.388136  7733 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:21.388576  7733 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:21.389394  7733 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.389984  7896 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:21.390386  7733 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:21.390492  7896 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:21.391719  7896 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.393304  7896 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:21.384020  1703 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.036s	sys 0.059s
W20260709 13:02:21.398497  1703 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.037s	sys 0.059s
I20260709 13:02:21.338815  7784 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:21.377043  1559 rpcz_store.cc:276] 0709 13:02:19.022928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.349988 (+2327060us) service_pool.cc:224] Handling call
0709 13:02:21.375897 (+ 25909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.376164 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.376207 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:21.376270 (+    63us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:21.376332 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.376423 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.376545 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:21.362470  1705 rpcz_store.cc:276] 0709 13:02:19.399201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.427502 (+ 28301us) service_pool.cc:224] Handling call
0709 13:02:19.427838 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.428195 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:19.428224 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:19.428716 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:02:20.590889 (+1162173us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:20.724106 (+133217us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.357076 (+632970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.357186 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:21.357251 (+    65us) raft_consensus.cc:1685] finished
0709 13:02:21.357271 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.357455 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":352041,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":243959,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":20530,"thread_start_us":1070861,"threads_started":2,"wal-append.queue_time_us":1202431}
W20260709 13:02:21.408238  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.002s	sys 0.018s
I20260709 13:02:21.409883  7892 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.413165  7834 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.441s	user 0.046s	sys 0.076s
I20260709 13:02:21.370817  7792 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:21.414135  7714 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.381311  7854 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.414937  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:21.413865  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 422684us
I20260709 13:02:21.367926  7842 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:21.415303  7861 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 13:02:21.368824  7858 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.425732  7336 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:21.426405  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1071) took 2607 ms. Trace:
I20260709 13:02:21.426787  7870 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:21.431042  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.002s	sys 0.018s
I20260709 13:02:21.460546  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:21.461232  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.461678  1634 raft_consensus.cc:2410] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:02:21.418808  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:21.452665  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.456054  1553 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.015s	sys 0.009s
I20260709 13:02:21.456310  2350 rpcz_store.cc:276] 0709 13:02:18.818583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.819315 (+   732us) service_pool.cc:224] Handling call
0709 13:02:18.819632 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.362736 (+543104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.426123 (+2063387us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54004bf0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:21.426327 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1247344896}
W20260709 13:02:21.434682  7834 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.463s	user 0.046s	sys 0.077s
W20260709 13:02:21.467685  1553 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.015s	sys 0.009s
I20260709 13:02:21.435506  7844 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:21.442472  7792 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.459456  7614 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.000s
I20260709 13:02:21.470741  7834 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:21.480131  7834 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.415685  7784 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.462992  7336 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:21.494949  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2376) took 2799 ms (client timeout 1855 ms). Trace:
I20260709 13:02:21.495536  7336 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:21.495342  1554 rpcz_store.cc:269] 0709 13:02:18.695697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.057620 (+361923us) service_pool.cc:224] Handling call
0709 13:02:21.494724 (+2437104us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b540005a854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:21.494852 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839828736}
I20260709 13:02:21.496213  7336 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.499396  7881 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.580s	user 0.014s	sys 0.035s
I20260709 13:02:21.500172  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:21.493619  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.500842  1632 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.435572  7762 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:02:21.496610  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:21.502647  1634 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 8. (term mismatch)
I20260709 13:02:21.512861  1634 pending_rounds.cc:85] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 7
I20260709 13:02:21.513293  1634 pending_rounds.cc:107] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:02:21.513659  1634 raft_consensus.cc:2889] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:21.514189  1634 pending_rounds.cc:107] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:02:21.514554  1634 raft_consensus.cc:2889] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:21.474159  7897 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.436540  7892 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:21.527127  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.443133  7714 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.537930  1701 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:02:21.540953  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3013) took 2730 ms. Trace:
I20260709 13:02:21.538815  1708 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:21.462795  7842 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:02:21.546137  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:21.541554  1703 rpcz_store.cc:276] 0709 13:02:18.809454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.821661 (+ 12207us) service_pool.cc:224] Handling call
0709 13:02:18.821961 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.822170 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:18.822492 (+   322us) log.cc:844] Serialized 14 byte log entry
0709 13:02:20.407057 (+1584565us) spinlock_profiling.cc:243] Waited 6.16 ms on lock 0x7b1800371260. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda2b0d23c7 00007fda276e49a7 00007fda276f40bf 00007fda276f4038
0709 13:02:21.398891 (+991834us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.539334 (+140443us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:21.539405 (+    71us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:21.539476 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.539561 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.539671 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":582257,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":75,"raft.queue_time_us":120,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":31229,"spinlock_wait_cycles":14175488,"thread_start_us":1352407,"threads_started":3,"wal-append.queue_time_us":466582,"wal-append.run_cpu_time_us":1693,"wal-append.run_wall_time_us":1001717}
I20260709 13:02:21.548509  1634 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:21.551826  1707 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:21.553900  1632 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:21.555090  1704 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:02:21.555562  1704 pending_rounds.cc:85] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 9
I20260709 13:02:21.555822  1704 pending_rounds.cc:107] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:02:21.556155  1704 raft_consensus.cc:2889] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:21.562752  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:21.563500  1554 raft_consensus.cc:2376] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15. Re-sending same reply.
I20260709 13:02:21.496666  7336 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.573372  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:21.565148  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 175138us
I20260709 13:02:21.579267  1559 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:21.489259  7792 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:21.562286  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 157535us
I20260709 13:02:21.489735  7844 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.652422  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:21.510526  7614 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.540745  7897 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.653505  1707 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:21.677098  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:21.680130  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:21.685895  1560 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:21.662510  7903 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.705269  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:21.706195  1703 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:21.718696  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:21.720016  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:21.690318  7911 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.723045  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:21.724172  1708 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:02:21.727360  1601 connection.cc:256] RPC call timeout handler was delayed by -2.301091552999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
W20260709 13:02:21.727902  1601 connection.cc:256] RPC call timeout handler was delayed by -2.2749915530000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.215s for a response
W20260709 13:02:21.731215  1601 connection.cc:256] RPC call timeout handler was delayed by -2.2434915530000126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
I20260709 13:02:21.731516  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:21.731786  1601 connection.cc:256] RPC call timeout handler was delayed by -2.0824854859999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
W20260709 13:02:21.732362  1601 connection.cc:256] RPC call timeout handler was delayed by -2.0698854859999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 13:02:21.732791  1601 connection.cc:256] RPC call timeout handler was delayed by -2.0527854860000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
I20260709 13:02:21.708127  7765 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.738615  1601 connection.cc:256] RPC call timeout handler was delayed by -2.0185854860000063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 13:02:21.742707  1601 connection.cc:256] RPC call timeout handler was delayed by -1.917785485999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
W20260709 13:02:21.744824  1601 connection.cc:256] RPC call timeout handler was delayed by -1.904285485999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
I20260709 13:02:21.734061  7661 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.745764  1601 connection.cc:256] RPC call timeout handler was delayed by -1.8268854859999806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 13:02:21.749522  1601 connection.cc:256] RPC call timeout handler was delayed by -1.8097854859999529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
W20260709 13:02:21.750162  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.750377  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7935854859999836s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18730000000000002s for a response
W20260709 13:02:21.751720  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7431854859999589s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
W20260709 13:02:21.752249  1601 connection.cc:256] RPC call timeout handler was delayed by -1.6496362579999868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19140000000000001s for a response
W20260709 13:02:21.752704  1601 connection.cc:256] RPC call timeout handler was delayed by -1.6099854859999709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 13:02:21.753165  1601 connection.cc:256] RPC call timeout handler was delayed by -1.5285435879999909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
W20260709 13:02:21.753661  1601 connection.cc:256] RPC call timeout handler was delayed by -1.5267447009999842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
W20260709 13:02:21.754136  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4847915530000364s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3028s for a response
W20260709 13:02:21.754586  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4731854859999771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
I20260709 13:02:21.755530  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:21.758517  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.762750  2317 connection.cc:256] RPC call timeout handler was delayed by -1.9526901880000196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
W20260709 13:02:21.739467  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 155993us
W20260709 13:02:21.765161  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.766707  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4077435389999664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
W20260709 13:02:21.769650  1601 connection.cc:256] RPC call timeout handler was delayed by -1.3390854859999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2378s for a response
W20260709 13:02:21.770059  1601 connection.cc:256] RPC call timeout handler was delayed by -1.2754435389999799s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
I20260709 13:02:21.770339  1703 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:21.766921  1674 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:21.770913  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:21.738544  7893 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.773129  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:21.779196  1707 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.743103  7909 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.740525  7854 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:21.742538  7733 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.780786  2317 connection.cc:256] RPC call timeout handler was delayed by -1.4237257429999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
W20260709 13:02:21.789146  2317 connection.cc:256] RPC call timeout handler was delayed by -1.1828780379999557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
I20260709 13:02:21.718851  7762 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:02:21.747340  7796 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.746357  7718 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.784299  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:21.737694  7911 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.777848  7831 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.788030  7913 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:02:21.792034  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:21.718219  7903 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.832985  7831 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.835841  7831 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:21.749608  7765 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.836599  7831 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:21.784770  7784 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:21.860615  7784 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:21.847600  7921 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:21.860924  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2365) took 3168 ms. Trace:
I20260709 13:02:21.791270  7907 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.861330  1552 rpcz_store.cc:276] 0709 13:02:18.692882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.825201 (+2132319us) service_pool.cc:224] Handling call
0709 13:02:20.888210 (+ 63009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.935480 (+ 47270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.860677 (+925197us) spinlock_profiling.cc:243] Waited 46.6 ms on lock 0x7b54002521d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:21.860863 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107161856}
I20260709 13:02:21.862027  7907 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.862775  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:21.863871  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:21.866091  7714 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:21.867101  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:21.797678  7733 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.868373  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:21.869387  1628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1118) took 3534 ms (client timeout 1588 ms). Trace:
I20260709 13:02:21.869781  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:21.853394  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 939) took 2831 ms. Trace:
W20260709 13:02:21.869796  1628 rpcz_store.cc:269] 0709 13:02:18.334837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.335143 (+   306us) service_pool.cc:224] Handling call
0709 13:02:21.869180 (+3534037us) spinlock_profiling.cc:243] Waited 2.49 s on lock 0x7b54001b6a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:21.869299 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5721148672}
I20260709 13:02:21.791601  7909 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.847286  7914 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.874459  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:21.858672  7923 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.858870  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:21.799158  7858 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:21.859360  7916 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:02:21.859861  7901 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:02:21.859720  7864 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:02:21.860760  7919 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:02:21.765918  7661 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:21.846767  7920 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:21.831517  7796 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.837255  7718 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.799741  7892 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:21.875284  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:21.851195  7496 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:21.877604  1552 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:21.889637  7923 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.785214  7893 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:21.870446  1555 rpcz_store.cc:276] 0709 13:02:19.021703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.098232 (+2076529us) service_pool.cc:224] Handling call
0709 13:02:21.497577 (+399345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.497761 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.497790 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:21.497834 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:21.497875 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.497930 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.853205 (+355275us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b540005a850. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:21.853332 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917773696}
W20260709 13:02:21.955852  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.018726  1707 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:22.060063  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.032187  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:22.058022  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 266048us
I20260709 13:02:22.029297  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:22.072718  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:22.066478  1705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.006s	sys 0.010s
I20260709 13:02:22.075201  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:22.073000  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:21.930070  1703 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:22.073617  1634 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:22.080832  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.081462  2353 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:21.920866  1701 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
W20260709 13:02:21.961361  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.065817  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 294975us
W20260709 13:02:22.072286  1676 connection.cc:256] RPC call timeout handler was delayed by -1.0908262280000258s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
W20260709 13:02:22.085127  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.076099  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.085422  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:22.075276  1705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.007s	sys 0.011s
I20260709 13:02:22.077065  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:22.085058  1676 connection.cc:256] RPC call timeout handler was delayed by -1.0242262280000318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
I20260709 13:02:22.086793  1636 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:22.084081  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:22.079905  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 50729.6us
I20260709 13:02:22.072466  7496 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:21.865267  7907 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:21.860046  7751 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:22.093631  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:21.890729  7904 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:02:22.089399  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.092193  2353 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:21.874960  7682 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:22.081499  7922 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:02:22.114331  1634 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:21.890492  7897 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:21.869750  7831 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:22.095372  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.095258  7496 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:22.098309  7494 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:22.092633  1636 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:21.856359  7336 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:22.103474  1708 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:02:22.090072  1676 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.859s (SENT)
I20260709 13:02:22.162569  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:22.094436  1601 leader_election.cc:400] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17.
I20260709 13:02:22.162781  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3061) took 3487 ms. Trace:
W20260709 13:02:22.163475  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 72255.6us
I20260709 13:02:22.164805  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:22.105679  1628 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:22.017843  7733 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.175668  7935 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:21.891690  7914 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:22.176525  2348 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1076) took 3356 ms (client timeout 1785 ms). Trace:
I20260709 13:02:22.177996  7909 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:22.019380  7792 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:22.108172  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:22.102495  7879 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:22.108332  2345 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:22.083400  7856 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 13:02:22.109912  7896 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.003s
I20260709 13:02:22.104357  7820 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:22.110723  7765 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.089057  7870 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:22.112394  7796 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:22.035370  7894 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.946066  7614 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.115936  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.195386  1701 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 12.
I20260709 13:02:22.196373  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.021265  7844 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:22.130206  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:22.131528  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.198009  1707 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:22.091390  7920 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:22.117966  7682 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:22.167637  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 77591.6us
I20260709 13:02:22.178215  7914 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:22.171240  7932 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:02:22.174484  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:22.215214  1629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2921) took 2587 ms (client timeout 3159 ms). Trace:
W20260709 13:02:22.215725  1629 rpcz_store.cc:269] 0709 13:02:19.646652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.646861 (+   209us) service_pool.cc:224] Handling call
0709 13:02:22.174299 (+2527438us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b5800135698. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:02:22.215132 (+ 40833us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4033920}
I20260709 13:02:22.217007  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3197) took 1387 ms. Trace:
I20260709 13:02:22.217453  1635 rpcz_store.cc:276] 0709 13:02:20.829377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.111116 (+281739us) service_pool.cc:224] Handling call
0709 13:02:21.111452 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.144478 (+ 33026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.144513 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.247035 (+102522us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.453556 (+206521us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:21.453607 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.453717 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.216730 (+763013us) raft_consensus.cc:1685] finished
0709 13:02:22.216761 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.216935 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16105,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":111,"thread_start_us":136466,"threads_started":1,"wal-append.queue_time_us":386377}
I20260709 13:02:22.174965  1636 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:22.220587  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.221151  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.221776  7804 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:22.222061  2345 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:22.178118  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1276) took 2915 ms. Trace:
I20260709 13:02:22.223292  2349 rpcz_store.cc:276] 0709 13:02:19.262918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.263426 (+   508us) service_pool.cc:224] Handling call
0709 13:02:19.266919 (+  3493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.821129 (+1554210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.177821 (+1356692us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54006202d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:22.178049 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3568982912}
I20260709 13:02:22.212213  7902 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:22.224870  7902 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:22.193006  7733 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:22.210042  7937 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.227784  7902 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.228848  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2356) took 3550 ms. Trace:
I20260709 13:02:22.229250  1557 rpcz_store.cc:276] 0709 13:02:18.678019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.810165 (+2132146us) service_pool.cc:224] Handling call
0709 13:02:20.810455 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.920162 (+109707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.228628 (+1308466us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54001fd454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:22.228790 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250915200}
I20260709 13:02:22.230505  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:22.231217  7914 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:02:22.177114  2348 rpcz_store.cc:269] 0709 13:02:18.820028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.821540 (+  1512us) service_pool.cc:224] Handling call
0709 13:02:19.960764 (+1139224us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b5800003c98. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5eee 0000000000375939 00007fda2c6291e0 00007fda2c6f20c4 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:02:22.176250 (+2215486us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54004ed754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:22.176405 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825961984}
I20260709 13:02:22.093257  7751 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:22.123988  7831 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:22.198513  1554 raft_consensus.cc:2410] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:02:22.132839  7494 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:22.207235  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1282) took 1260 ms. Trace:
I20260709 13:02:22.170559  2352 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [15.13-15.13]
W20260709 13:02:22.210199  7927 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:02:22.246812  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.151430  7897 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:22.211490  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.212955  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:22.209158  7765 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:22.227478  7937 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:22.163412  1553 rpcz_store.cc:276] 0709 13:02:18.675398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.806872 (+2131474us) service_pool.cc:224] Handling call
0709 13:02:20.823373 (+ 16501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.471260 (+647887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.471310 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.471950 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.647074 (+175124us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:21.759090 (+112016us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.763035 (+  3945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.763115 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.162473 (+399358us) raft_consensus.cc:1685] finished
0709 13:02:22.162500 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.162689 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":966,"prepare.queue_time_us":199013,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":123748,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":2537,"thread_start_us":885420,"threads_started":3,"wal-append.queue_time_us":243936}
I20260709 13:02:22.248828  1705 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:22.250088  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.250873  1705 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:22.235049  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.190614  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:22.231617  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:22.238893  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.221269  1701 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:22.192947  7834 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:22.256126  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 65566us
I20260709 13:02:22.260067  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:22.260412  7682 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.207015  7870 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:22.261967  7494 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:22.262516  7662 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:22.263155  7494 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:22.264144  7870 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.265710  7733 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:22.265956  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.266304  7733 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:22.267246  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.267373  7733 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:22.245888  2344 rpcz_store.cc:276] 0709 13:02:20.946651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.954791 (+  8140us) service_pool.cc:224] Handling call
0709 13:02:21.056463 (+101672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.056839 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:21.056865 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.063319 (+  6454us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.430126 (+366807us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:21.430172 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.430253 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.207012 (+776759us) raft_consensus.cc:1685] finished
0709 13:02:22.207035 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.207177 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":431,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":8992,"thread_start_us":327734,"threads_started":1,"wal-append.queue_time_us":472331}
I20260709 13:02:22.268941  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:22.245798  7796 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:02:22.270085  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:22.270162  7879 consensus_queue.cc:1059] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:22.271155  7879 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:02:22.271590  7933 log.cc:927] Time spent T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:02:22.272534  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1277) took 3009 ms. Trace:
I20260709 13:02:22.272924  2347 rpcz_store.cc:276] 0709 13:02:19.263245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.263555 (+   310us) service_pool.cc:224] Handling call
0709 13:02:19.264062 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.604359 (+1340297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.272291 (+1667932us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54004f4a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:22.272472 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545193728}
I20260709 13:02:22.276077  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.276300  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.276996  2344 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:02:22.270087  2350 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:02:22.271533  7879 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:22.259291  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:22.287487  1705 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:22.254864  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:22.288269  1707 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:22.289449  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.289850  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.259253  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:22.279203  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.290599  1629 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:22.292511  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.293390  1629 raft_consensus.cc:2410] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 13:02:22.294025  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.296772  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 91546.3us
I20260709 13:02:22.297801  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:22.282234  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 194360us
I20260709 13:02:22.309569  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:22.310231  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51071.1us
I20260709 13:02:22.310388  2344 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:02:22.311685  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:22.313688  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:22.314973  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.314985  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.316097  2345 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:22.316082  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.317068  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.317703  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.318343  2351 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:22.323397  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:22.324954  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:22.315080  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:22.326071  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.326632  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.327021  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.326646  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:22.333841  2352 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:22.327803  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.330739  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.247020  7911 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:22.322305  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.323154  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:22.273437  7937 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.375126  2353 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:22.331032  1557 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:22.327059  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.346048  2344 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:22.273545  7661 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:22.210561  7894 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:22.383673  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.352123  2350 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:02:22.385771  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.345005  1707 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
W20260709 13:02:22.365087  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.387650  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.388598  1707 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:22.331403  7614 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:22.322192  7864 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:22.376518  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 944) took 3284 ms. Trace:
I20260709 13:02:22.316476  7804 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:02:22.275084  7765 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:22.331346  7932 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 13:02:22.377460  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.011s	sys 0.009s
I20260709 13:02:22.378155  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:22.389122  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64249.8us
I20260709 13:02:22.408293  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:22.409111  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.011s	sys 0.009s
I20260709 13:02:22.392400  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:22.409947  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 643052us
I20260709 13:02:22.410185  1701 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:22.410717  1674 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:22.330945  7898 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:02:22.275786  7834 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:22.397979  1557 rpcz_store.cc:276] 0709 13:02:19.092235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.327894 (+3235659us) service_pool.cc:224] Handling call
0709 13:02:22.328212 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.376185 (+ 47973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.376448 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:22.421950  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:22.394464  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:22.399333  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 118062us
I20260709 13:02:22.400147  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:22.349961  7718 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.350518  7846 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:22.403415  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.371122  1705 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:22.351089  7650 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:02:22.378249  7652 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:22.380980  7929 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:22.394101  1559 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:22.414201  2352 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:22.453454  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.396250  7614 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:22.403330  7855 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.018s	user 0.006s	sys 0.008s
I20260709 13:02:22.456951  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.422259  1557 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:22.373924  2345 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:22.430912  1707 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:22.373916  7842 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:02:22.403990  7864 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.448366  7846 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:22.441816  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.470189  1702 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
W20260709 13:02:22.443742  1631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3199) took 1613 ms (client timeout 2109 ms). Trace:
I20260709 13:02:22.334443  7751 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:22.402366  7893 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:22.378098  7920 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:22.402985  7938 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:22.451764  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.454500  1705 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:22.454640  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3068) took 1073 ms. Trace:
I20260709 13:02:22.460757  1552 raft_consensus.cc:2410] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:02:22.487854  1702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3023) took 3072 ms (client timeout 1778 ms). Trace:
I20260709 13:02:22.456346  7614 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:22.407016  7879 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:22.505259  7662 consensus_queue.cc:1175] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" responder_term: 15 status { last_received { term: 15 index: 13 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: [14.13-14.13]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:22.508898  1710 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:22.489439  7496 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:22.459071  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1070) took 3640 ms. Trace:
I20260709 13:02:22.404860  7896 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:02:22.417340  7923 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:22.448175  7895 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:22.447876  7911 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:22.449350  7718 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.465842  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3073) took 1083 ms. Trace:
I20260709 13:02:22.526993  1557 rpcz_store.cc:276] 0709 13:02:21.382522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.421349 (+1038827us) service_pool.cc:224] Handling call
0709 13:02:22.421802 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.465450 (+ 43648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.465779 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:22.436656  7869 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:22.468894  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 155320us
I20260709 13:02:22.322835  7844 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:02:22.337077  7897 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
W20260709 13:02:22.471172  1631 rpcz_store.cc:269] 0709 13:02:20.829972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.947680 (+117708us) service_pool.cc:224] Handling call
0709 13:02:22.443437 (+1495757us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400138dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:22.443637 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285957376}
I20260709 13:02:22.456131  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:22.538929  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 130722us
I20260709 13:02:22.454396  7929 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:02:22.453158  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 138667us
I20260709 13:02:22.464515  7682 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:22.486102  1706 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:22.541452  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:22.464708  7652 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:22.490372  1705 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:02:22.487541  1559 rpcz_store.cc:276] 0709 13:02:21.381075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.381886 (+1000811us) service_pool.cc:224] Handling call
0709 13:02:22.393492 (+ 11606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.452672 (+ 59180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.454567 (+  1895us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:22.488391  1702 rpcz_store.cc:269] 0709 13:02:19.398295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.399487 (+  1192us) service_pool.cc:224] Handling call
0709 13:02:22.441534 (+3042047us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b5800155498. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:02:22.470725 (+ 29191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12373888}
I20260709 13:02:22.547503  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:22.485368  1707 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
W20260709 13:02:22.502617  7940 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:02:22.477696  7943 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:22.505653  7879 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:22.418844  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.509527  7912 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.526404  7447 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:02:22.405006  7762 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:02:22.469427  7864 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:22.473780  7846 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:22.470347  7949 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:22.524535  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 130179us
I20260709 13:02:22.525060  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:22.494004  7751 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:22.492391  7938 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.513734  2346 rpcz_store.cc:276] 0709 13:02:18.818269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.822426 (+  4157us) service_pool.cc:224] Handling call
0709 13:02:18.822929 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.823354 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:18.823615 (+   261us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.429183 (+2605568us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b18004a8000. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287
0709 13:02:21.642170 (+212987us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.003616 (+361446us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.083156 (+ 79540us) log.cc:844] Serialized 564 byte log entry
0709 13:02:22.083397 (+   241us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:22.083434 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.083516 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.458797 (+375281us) raft_consensus.cc:1685] finished
0709 13:02:22.458829 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.459001 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44910,"prepare.queue_time_us":276082,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":122,"raft.queue_time_us":28622,"raft.run_cpu_time_us":3464,"raft.run_wall_time_us":39728,"spinlock_wait_cycles":8186240,"thread_start_us":1008462,"threads_started":4,"wal-append.queue_time_us":890291,"wal-append.run_cpu_time_us":1407,"wal-append.run_wall_time_us":1083246}
I20260709 13:02:22.533651  1602 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:22.328107  7931 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:02:22.570385  7957 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:22.583755  7931 consensus_queue.cc:1243] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer 95fc6a4cf5874b7eb91280dffcd3b3fb log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:02:22.584443  7931 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:22.584879  7931 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:22.585901  7931 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:22.523594  7494 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.559348  7950 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:22.587464  7931 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:22.587172  7494 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:22.519716  7895 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.589716  7895 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:22.570922  7796 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
W20260709 13:02:22.593015  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.549563  1705 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [13.11-15.13]
I20260709 13:02:22.556974  7943 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.594080  7765 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.594295  7943 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:22.594336  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:22.538107  7718 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:22.594900  1705 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 13:02:22.596874  7718 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.550346  1557 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:22.598172  7844 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:22.598986  7949 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:22.566895  7894 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:22.583209  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3033) took 1011 ms. Trace:
I20260709 13:02:22.600602  1704 rpcz_store.cc:276] 0709 13:02:21.572233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.572706 (+   473us) service_pool.cc:224] Handling call
0709 13:02:21.573097 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.955559 (+382462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.955606 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.081942 (+126336us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.213928 (+131986us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:22.213972 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.214033 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.582828 (+368795us) raft_consensus.cc:1685] finished
0709 13:02:22.582864 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.583114 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6437,"raft.queue_time_us":141143,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":7080,"thread_start_us":500069,"threads_started":2,"wal-append.queue_time_us":114068}
W20260709 13:02:22.540516  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 84510.7us
W20260709 13:02:22.554787  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 141723us
I20260709 13:02:22.601415  7949 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:22.603449  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:22.553586  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:22.574177  7652 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:22.601938  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 76986.3us
W20260709 13:02:22.604597  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51175.1us
I20260709 13:02:22.527133  7831 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:22.542467  7929 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:22.623451  7831 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.597275  1705 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:22.549657  1706 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:02:22.626543  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:22.627287  1555 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:22.603192  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:22.617463  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.628259  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:22.629680  2345 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:02:22.587893  7950 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:22.602108  7447 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:22.602653  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 69151.2us
W20260709 13:02:22.647516  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.648196  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.648804  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45706.3us
I20260709 13:02:22.611420  7842 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:22.650090  1674 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:22.603895  1701 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:22.651841  1632 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:22.612685  7897 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:22.602779  7494 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:22.614466  7762 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:22.614605  7903 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:22.561038  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:22.605309  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:22.609692  7844 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:22.586036  7912 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:22.601629  7938 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:22.603904  7949 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:22.542840  7820 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:22.615486  7895 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:22.603076  1704 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:22.706223  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:22.537823  7923 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.131s
I20260709 13:02:22.656960  7894 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:22.649606  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.760821  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.761577  1631 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:22.650579  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:22.762230  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.658102  2346 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:22.669735  1701 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:02:22.762976  1701 pending_rounds.cc:85] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 9
I20260709 13:02:22.762861  1627 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:22.763270  1701 pending_rounds.cc:107] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 14.10
W20260709 13:02:22.763454  1627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2924) took 3107 ms (client timeout 2052 ms). Trace:
I20260709 13:02:22.665416  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:22.764118  1627 rpcz_store.cc:269] 0709 13:02:19.655906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.656608 (+   702us) service_pool.cc:224] Handling call
0709 13:02:22.762071 (+3105463us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b5800135698. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:02:22.763377 (+  1306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2998656}
I20260709 13:02:22.699448  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.701216  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.703457  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:22.765537  1635 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:02:22.702008  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.709710  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:22.766197  1674 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:22.646762  7447 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:22.768661  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 65544.9us
I20260709 13:02:22.770407  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:22.771646  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:22.711185  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 105946us
I20260709 13:02:22.773805  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:22.667968  7796 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:22.635658  1705 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:22.699268  7762 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.741144  7821 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:22.703450  7949 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.657428  7950 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:22.751457  7855 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:22.753868  1709 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:22.667207  7659 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.641294  7953 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:22.763607  1701 raft_consensus.cc:2889] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:22.629251  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:22.778738  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.780350  1629 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:02:22.814710  1629 pending_rounds.cc:85] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 12
I20260709 13:02:22.815102  1629 pending_rounds.cc:107] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:02:22.815392  1629 raft_consensus.cc:2889] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:22.786309  1705 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:22.786996  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:22.788347  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2384) took 1126 ms. Trace:
I20260709 13:02:22.802800  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3025) took 3403 ms. Trace:
W20260709 13:02:22.811134  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.776674  7447 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:22.785342  7762 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:22.761374  7952 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:22.847031  1554 rpcz_store.cc:276] 0709 13:02:21.661526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.702707 (+ 41181us) service_pool.cc:224] Handling call
0709 13:02:22.788115 (+1085408us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5400311d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:22.788273 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1098495872}
I20260709 13:02:22.846381  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.785379  7820 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:22.850867  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:22.857683  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:22.786016  7765 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:22.807803  7733 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:22.806988  7855 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:22.860108  1704 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:22.858569  1555 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:22.793841  7907 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:22.793113  7844 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:22.801635  7949 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:22.859560  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:22.943475  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 84050.3us
I20260709 13:02:22.848430  1709 rpcz_store.cc:276] 0709 13:02:19.398905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.399651 (+   746us) service_pool.cc:224] Handling call
0709 13:02:22.753108 (+3353457us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5800155498. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:02:22.753452 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.802366 (+ 48914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.802711 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25750912}
I20260709 13:02:22.807402  7821 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:22.792279  7864 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:22.929738  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.930068  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.793089  7955 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.828776  1525 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:22.845675  7959 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:22.936717  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2373) took 4241 ms. Trace:
I20260709 13:02:22.860437  7820 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:22.883628  7733 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:22.950667  1631 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:22.819218  7659 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:22.941087  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 380088us
W20260709 13:02:22.971318  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 197674us
W20260709 13:02:22.972088  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.959784  1561 rpcz_store.cc:276] 0709 13:02:18.694955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.917742 (+2222787us) service_pool.cc:224] Handling call
0709 13:02:20.925250 (+  7508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.092790 (+167540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.936326 (+1843536us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54002706d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:22.936629 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383711104}
I20260709 13:02:22.973155  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:22.987995  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:22.987033  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.992269  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.993112  1631 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:22.995925  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.999893  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:22.997516  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:23.002475  1631 raft_consensus.cc:2376] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15. Re-sending same reply.
I20260709 13:02:23.004705  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:23.003237  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.007130  1631 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:22.937505  7907 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:23.010759  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:23.011680  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 183020us
I20260709 13:02:22.948575  7923 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:23.022008  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:23.024158  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 52577.6us
I20260709 13:02:23.024608  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:23.011294  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 239762us
I20260709 13:02:23.025082  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:23.025453  1561 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
W20260709 13:02:23.013952  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.026176  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:23.012247  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.027207  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:23.028661  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:22.958523  7844 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:23.016016  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:22.962592  7820 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:22.970237  7733 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:23.038599  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:23.047348  2346 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:23.012208  7920 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:23.033421  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:23.066136  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:23.053984  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.068046  1561 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:23.027304  7907 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:23.009771  7959 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:23.018770  7494 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:23.016124  7957 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:22.963562  7846 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.046135  7923 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:23.099630  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 85802.2us
I20260709 13:02:23.032898  7965 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:23.104696  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:23.085856  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.039001  7796 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.032737  7953 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:23.039249  7652 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:22.969937  7955 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.030046  7952 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:23.074622  7942 raft_consensus.cc:2764] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:02:23.113562  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 83462.4us
I20260709 13:02:23.125065  2318 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:23.032660  7960 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:02:23.100553  7923 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.113843  7893 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:23.114784  7953 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:23.151731  7923 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:23.194167  7958 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:02:23.196516  7846 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:23.090701  7821 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.116070  7879 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:02:23.204654  7864 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:23.204172  7959 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.131661  7895 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:23.146116  7968 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.100s
W20260709 13:02:23.197129  7963 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:02:23.112673  7972 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:23.184201  7953 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:23.232169  7962 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 13:02:23.243546  7966 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:23.229818  7972 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:23.251487  7972 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:23.056586  1558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2357) took 4378 ms (client timeout 2121 ms). Trace:
I20260709 13:02:23.293782  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:23.294342  1558 rpcz_store.cc:269] 0709 13:02:18.686378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.392839 (+706461us) service_pool.cc:224] Handling call
0709 13:02:23.056223 (+3663384us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54002cc254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:23.056465 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3878202240}
I20260709 13:02:23.296214  1554 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:23.054307  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:23.309087  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.036811  1527 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:23.312794  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 187595us
I20260709 13:02:23.314780  1554 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:23.316767  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.298730  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.113152  7494 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:23.301692  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:23.319962  7494 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:23.321126  7494 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:23.321960  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:23.323248  1634 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:02:23.129954  7732 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:23.318994  1561 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:23.326930  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.329030  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.330880  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.332772  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.344859  7659 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:23.345844  7714 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:23.302769  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.308074  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2343) took 1269 ms. Trace:
W20260709 13:02:23.308558  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 203718us
I20260709 13:02:23.350975  1702 rpcz_store.cc:276] 0709 13:02:22.038546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.571707 (+533161us) service_pool.cc:224] Handling call
0709 13:02:22.572277 (+   570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.572579 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.572617 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.573273 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.794334 (+221061us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:22.794409 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.794531 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.307144 (+512613us) raft_consensus.cc:1685] finished
0709 13:02:23.307197 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.307662 (+   465us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":220505,"threads_started":1,"wal-append.queue_time_us":456352}
I20260709 13:02:23.303962  1631 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:23.353430  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:23.354579  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:23.356096  1631 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:02:23.349941  1561 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:23.368480  7751 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:23.369989  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.375607  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.297466  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.152571  7765 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:23.117233  7796 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.384961  7895 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:23.325009  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:23.357416  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:23.417927  7893 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:23.388376  7792 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:23.425925  7792 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:23.429641  7650 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.430940  7650 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.150826  7844 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.434667  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.436954  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.445987  7765 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:23.447149  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:23.448765  1634 raft_consensus.cc:2410] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:02:23.449798  1631 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
W20260709 13:02:23.456324  7496 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.946s	user 0.016s	sys 0.008s
I20260709 13:02:23.458629  1553 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:02:23.460318  7796 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:23.467038  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2375) took 4771 ms. Trace:
I20260709 13:02:23.467767  1560 rpcz_store.cc:276] 0709 13:02:18.695436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.045931 (+2350495us) service_pool.cc:224] Handling call
0709 13:02:21.067311 (+ 21380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.686465 (+619154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.466164 (+1779699us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b5400207954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:23.466686 (+   522us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422060800}
W20260709 13:02:23.469875  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 433039us
I20260709 13:02:23.471544  1527 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:23.472012  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3078) took 2072 ms. Trace:
I20260709 13:02:23.472931  7496 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:23.388113  7903 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
W20260709 13:02:23.457281  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.473845  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 156503us
I20260709 13:02:23.446066  7980 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:02:23.472628  1557 rpcz_store.cc:276] 0709 13:02:21.399101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.538446 (+1139345us) service_pool.cc:224] Handling call
0709 13:02:22.538931 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.597652 (+ 58721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.471271 (+873619us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b54003205d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:23.471738 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25354624}
I20260709 13:02:23.428421  7792 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:23.487747  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.488699  7792 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:23.489176  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.499883  1560 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:23.503826  7978 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:02:23.515437  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.517628  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2388) took 1785 ms. Trace:
I20260709 13:02:23.521888  1560 rpcz_store.cc:276] 0709 13:02:21.728725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.469236 (+1740511us) service_pool.cc:224] Handling call
0709 13:02:23.474241 (+  5005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.501032 (+ 26791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.514192 (+ 13160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:23.519299  1557 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 13:02:23.473879  7494 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.523731  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2389) took 1791 ms. Trace:
I20260709 13:02:23.524154  1557 rpcz_store.cc:276] 0709 13:02:21.732462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.495615 (+1763153us) service_pool.cc:224] Handling call
0709 13:02:23.518836 (+ 23221us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:23.523095 (+  4259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.523652 (+   557us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:23.525347  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2383) took 1864 ms. Trace:
I20260709 13:02:23.525918  1557 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:02:23.527734  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.529546  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:23.530624  1627 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:23.534267  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3079) took 2132 ms. Trace:
I20260709 13:02:23.534817  1559 rpcz_store.cc:276] 0709 13:02:21.401924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.555206 (+1153282us) service_pool.cc:224] Handling call
0709 13:02:22.555558 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.555906 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:02:22.783003 (+227097us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.492544 (+709541us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b180048b860. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a
0709 13:02:23.500812 (+  8268us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.516338 (+ 15526us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:23.533769 (+ 17431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.533871 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.534051 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":455879,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":4511,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":11761,"spinlock_wait_cycles":557901696,"thread_start_us":495948,"threads_started":2,"wal-append.queue_time_us":456714}
I20260709 13:02:23.542124  1627 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:02:23.542686  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 476504us
I20260709 13:02:23.545243  7986 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:23.546232  1634 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:02:23.546487  7986 consensus_queue.cc:1243] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:02:23.547680  7983 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:23.548614  7983 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:23.550251  7983 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.552047  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1035) took 1431 ms. Trace:
I20260709 13:02:23.555142  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3041) took 1723 ms. Trace:
I20260709 13:02:23.525772  1552 rpcz_store.cc:276] 0709 13:02:21.661178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.275609 (+614431us) service_pool.cc:224] Handling call
0709 13:02:23.524876 (+1249267us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400207e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:23.525132 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313081216}
W20260709 13:02:23.556965  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3024) took 4158 ms (client timeout 1727 ms). Trace:
W20260709 13:02:23.558633  1632 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.006s	sys 0.005s
W20260709 13:02:23.561206  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 43946.6us
I20260709 13:02:23.561316  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:23.565026  1636 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:02:23.567276  7661 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:23.570060  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:23.572607  7943 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.977s	user 0.009s	sys 0.009s
W20260709 13:02:23.579430  7733 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.501s	user 0.018s	sys 0.000s
W20260709 13:02:23.588231  7938 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.878s	user 0.008s	sys 0.008s
W20260709 13:02:23.588950  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 469680us
I20260709 13:02:23.594578  7834 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:23.596222  7854 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:23.621909  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 150613us
W20260709 13:02:23.606122  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 61541.6us
I20260709 13:02:23.526561  7494 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:23.612399  1561 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:23.600636  7855 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:23.610152  1710 rpcz_store.cc:269] 0709 13:02:19.398597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.399571 (+   974us) service_pool.cc:224] Handling call
0709 13:02:21.385699 (+1986128us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b5800155498. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:02:23.556376 (+2170677us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002e7cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:23.556612 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2567008896}
W20260709 13:02:23.613540  1632 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.007s	sys 0.005s
I20260709 13:02:23.609745  1708 rpcz_store.cc:276] 0709 13:02:21.832132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.832345 (+   213us) service_pool.cc:224] Handling call
0709 13:02:21.832706 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.162436 (+329730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.554442 (+1392006us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400223654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:23.554895 (+   453us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621789568}
I20260709 13:02:23.617585  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:23.618036  7661 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:02:23.619025  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 49405.5us
I20260709 13:02:23.619026  7894 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:23.621690  7614 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.101s	user 0.011s	sys 0.014s
I20260709 13:02:23.609179  1706 rpcz_store.cc:276] 0709 13:02:22.119975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.626185 (+506210us) service_pool.cc:224] Handling call
0709 13:02:22.626496 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.759357 (+132861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.759396 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:22.759471 (+    75us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:22.759532 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.551405 (+791873us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54001865d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:23.551474 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.551777 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304958848}
I20260709 13:02:23.646512  7614 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.654388  1557 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:02:23.654932  7614 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:23.657688  7854 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:23.659622  7982 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.465s	user 0.004s	sys 0.000s
I20260709 13:02:23.660558  7854 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:23.662019  7987 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:23.663098  7987 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.665797  7978 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:23.667265  7650 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:23.668433  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:23.670001  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:23.670219  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:23.671415  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:23.672200  1709 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:23.674829  1706 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:02:23.669310  1707 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:23.679363  7983 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.682307  1636 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:23.685883  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:23.687541  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.688462  7903 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:23.688853  7762 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.825s	user 0.013s	sys 0.008s
I20260709 13:02:23.688987  7903 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.689378  7751 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:23.690037  7751 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:23.694512  7914 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.689555  7903 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:23.695392  7914 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.635437  7733 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.698154  1706 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:23.698526  2318 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:23.699502  7733 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:23.680972  7855 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:23.699944  7856 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:23.701670  7856 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:23.702126  7856 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:23.673058  7987 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:23.720368  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3080) took 2250 ms. Trace:
I20260709 13:02:23.726400  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1143) took 1061 ms. Trace:
I20260709 13:02:23.727298  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:23.726913  1630 rpcz_store.cc:276] 0709 13:02:22.660265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.660415 (+   150us) service_pool.cc:224] Handling call
0709 13:02:22.660894 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.963676 (+302782us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.963713 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.964339 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.293907 (+329568us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:23.294082 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.294266 (+   184us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.658464 (+364198us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:23.712421 (+ 53957us) raft_consensus.cc:1685] finished
0709 13:02:23.712457 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.721793 (+  9336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49613,"raft.queue_time_us":103462,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":13343,"spinlock_wait_cycles":45436288,"thread_start_us":473331,"threads_started":2,"wal-append.queue_time_us":176747}
W20260709 13:02:23.729655  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.730523  1634 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:23.730427  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 80505.4us
I20260709 13:02:23.636395  7943 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.729285  1675 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:23.668025  7650 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
W20260709 13:02:23.742498  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.747151  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:23.748708  7796 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:23.749374  1675 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:23.749523  7796 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.750288  7796 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:23.751442  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.752269  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.753990  1674 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:23.761899  1709 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:23.641515  7938 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:23.723623  1558 rpcz_store.cc:276] 0709 13:02:21.403811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.438684 (+2034873us) service_pool.cc:224] Handling call
0709 13:02:23.439338 (+   654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.439966 (+   628us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:23.457445 (+ 17479us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.653353 (+195908us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.653555 (+   202us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:23.653607 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.653691 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.653826 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15336,"prepare.queue_time_us":16759,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":83,"raft.queue_time_us":27711,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":158,"thread_start_us":169706,"threads_started":1}
I20260709 13:02:23.771283  1554 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:23.775305  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:23.753648  7614 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.710724  7762 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.758786  7714 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:23.785776  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.769703  7893 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:23.777786  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:23.782810  1560 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:02:23.799813  7447 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.877s	user 0.013s	sys 0.009s
I20260709 13:02:23.800849  7938 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:23.801044  7949 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.836s	user 0.017s	sys 0.001s
W20260709 13:02:23.802436  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44854.8us
I20260709 13:02:23.803510  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.792845  7972 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:23.805662  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:23.807063  7869 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:23.806705  7714 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:23.807909  7950 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.992s	user 0.015s	sys 0.004s
I20260709 13:02:23.808605  1557 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:02:23.793236  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 945) took 1196 ms. Trace:
I20260709 13:02:23.800515  7447 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.809264  1560 rpcz_store.cc:276] 0709 13:02:22.596464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.630056 (+1033592us) service_pool.cc:224] Handling call
0709 13:02:23.731647 (+101591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.789327 (+ 57680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.793154 (+  3827us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:23.808460  7911 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:23.788540  7894 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:23.823172  1601 leader_election.cc:400] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17.
I20260709 13:02:23.823690  1601 leader_election.cc:403] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:23.814172  7914 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:23.825671  7661 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17.
I20260709 13:02:23.826581  7661 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:23.827651  7661 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:02:23.827970  7931 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.240s	user 0.010s	sys 0.013s
W20260709 13:02:23.828778  7923 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.608s	user 0.006s	sys 0.005s
W20260709 13:02:23.835320  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 86245.3us
I20260709 13:02:23.801692  7949 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.836895  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:23.835019  7733 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.838442  7733 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:23.840549  7907 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.743s	user 0.012s	sys 0.002s
I20260709 13:02:23.841148  7907 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.842489  1705 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:02:23.844128  7907 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:23.844242  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:23.844660  1631 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:02:23.845098  1631 pending_rounds.cc:85] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 10
I20260709 13:02:23.845309  1631 pending_rounds.cc:107] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:02:23.845597  1631 raft_consensus.cc:2889] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:23.846621  7447 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:23.847210  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 110066us
I20260709 13:02:23.847714  1629 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:23.838837  7714 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:23.848380  7912 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:23.849047  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3208) took 1134 ms. Trace:
I20260709 13:02:23.849874  7714 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:23.846655  7943 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:23.850445  7714 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:23.796133  7614 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.848228  2318 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:23.849442  1629 rpcz_store.cc:276] 0709 13:02:22.714611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.531953 (+817342us) service_pool.cc:224] Handling call
0709 13:02:23.532421 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.848148 (+315727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.848741 (+   593us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54003b9cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:23.848954 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724262144}
W20260709 13:02:23.818462  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 140582us
I20260709 13:02:23.828037  7661 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:23.860773  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:23.861752  7447 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:23.861455  7714 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.829289  7923 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.866835  1708 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:02:23.866425  7614 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:23.868147  7923 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:23.868726  7953 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.012s
I20260709 13:02:23.868798  7662 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:23.869246  7447 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:23.869797  7447 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.869663  7953 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.870201  7447 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:23.870841  1627 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:23.871256  7821 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.006s	sys 0.016s
W20260709 13:02:23.871807  7821 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.006s	sys 0.016s
I20260709 13:02:23.872920  7953 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:23.877760  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:23.878728  2349 raft_consensus.cc:2376] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15. Re-sending same reply.
W20260709 13:02:23.879482  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.881603  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:23.882225  7945 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.124s	user 0.003s	sys 0.001s
I20260709 13:02:23.882615  2344 raft_consensus.cc:2410] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:23.885959  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:23.886911  2344 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:02:23.888765  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:23.838868  7949 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:23.889552  2349 raft_consensus.cc:2376] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20. Re-sending same reply.
I20260709 13:02:23.889839  7844 consensus_queue.cc:1059] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:23.893802  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.894110  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.890642  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3083) took 2118 ms. Trace:
I20260709 13:02:23.894730  1555 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:23.895042  1553 rpcz_store.cc:276] 0709 13:02:21.772517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.249467 (+476950us) service_pool.cc:224] Handling call
0709 13:02:23.890411 (+1640944us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002530d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:23.890579 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2753222016}
I20260709 13:02:23.895735  7897 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:23.891250  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.896166  7897 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:23.896543  1557 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:23.896802  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:23.892561  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.897284  7897 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:23.898279  1561 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:23.899055  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:23.900327  1560 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [13.10-15.11]
I20260709 13:02:23.898331  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:23.902433  1557 raft_consensus.cc:2368] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:23.905646  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 954) took 1124 ms. Trace:
I20260709 13:02:23.878773  7955 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:23.899068  1553 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:23.907249  1559 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:02:23.817456  7894 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.915184  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:23.906157  1554 rpcz_store.cc:276] 0709 13:02:22.781055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.316251 (+535196us) service_pool.cc:224] Handling call
0709 13:02:23.905349 (+589098us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b54001ce154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:23.905555 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1036574976}
W20260709 13:02:23.941376  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 216028us
I20260709 13:02:23.808444  7950 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.922156  7894 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:23.953428  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:23.938210  7895 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:23.958029  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:23.963066  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:23.943248  7955 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:23.968068  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:23.979192  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 99740.1us
I20260709 13:02:23.974723  1553 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:23.984455  1705 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
W20260709 13:02:23.990126  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 75055us
I20260709 13:02:23.974268  7923 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:23.995533  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:23.996826  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:23.968936  7955 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:23.995518  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3225) took 1188 ms. Trace:
W20260709 13:02:24.003708  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 58881.5us
W20260709 13:02:24.004020  7820 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.903s	user 0.011s	sys 0.005s
I20260709 13:02:24.004757  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.004630  7820 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:24.008797  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3094) took 1355 ms. Trace:
I20260709 13:02:24.009315  1559 rpcz_store.cc:276] 0709 13:02:22.646210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.537071 (+890861us) service_pool.cc:224] Handling call
0709 13:02:23.906914 (+369843us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.921689 (+ 14775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.001455 (+ 79766us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54001ce150. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:24.001660 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849486080}
I20260709 13:02:23.966946  7751 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.005717  7821 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.018788  1674 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:23.974443  7912 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 13:02:24.021873  7988 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 13:02:24.022280  7959 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.011s	sys 0.011s
W20260709 13:02:24.023064  7990 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:02:24.023998  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:24.025605  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 83861us
I20260709 13:02:24.027369  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:23.998351  7856 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:23.999970  1628 rpcz_store.cc:276] 0709 13:02:22.750671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.534572 (+783901us) service_pool.cc:224] Handling call
0709 13:02:23.535041 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.535439 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:23.635999 (+100560us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.938711 (+302712us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.938763 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:23.938810 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.938908 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.939046 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15352,"prepare.queue_time_us":152366,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":80,"raft.queue_time_us":7630,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":18234,"thread_start_us":373845,"threads_started":2}
W20260709 13:02:24.032130  7920 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.953s	user 0.017s	sys 0.000s
I20260709 13:02:24.027393  2346 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:24.011780  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.001154  7955 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.056043  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:24.058697  1710 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:24.053990  7959 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.012s	sys 0.011s
I20260709 13:02:24.052556  7765 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:24.029336  7447 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.077355  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:24.077823  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:24.077091  7949 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:24.078740  1705 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:24.084221  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:24.080440  7856 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:24.092413  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:24.093362  2345 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:24.084565  7920 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:24.097241  2352 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: [15.15-15.15]
I20260709 13:02:24.098897  7997 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:02:24.100898  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:24.078588  7751 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.105413  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:24.083226  7821 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.106521  7821 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.111322  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:24.112350  7762 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:24.076522  7765 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:24.113783  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 115380us
I20260709 13:02:24.114010  7950 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.114253  2345 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:24.116012  7907 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:24.116460  7907 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:24.116824  7907 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:24.117815  7870 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:24.118821  7820 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:24.119078  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:24.119848  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.083006  7914 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:02:24.113288  7765 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:24.061888  7903 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:24.097645  7920 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:02:24.122185  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.123596  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1146) took 1462 ms. Trace:
I20260709 13:02:24.132866  2348 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:02:24.097318  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:24.136896  1705 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:24.134202  1708 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:24.128435  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:24.127737  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:24.146863  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.147508  7949 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:24.148394  7949 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.148743  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:24.135142  1631 rpcz_store.cc:276] 0709 13:02:22.661264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.453381 (+792117us) service_pool.cc:224] Handling call
0709 13:02:23.600065 (+146684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.846106 (+246041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.123252 (+277146us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5400343854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:24.123520 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":561508096}
I20260709 13:02:24.147671  1633 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:24.161289  2346 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
W20260709 13:02:24.128803  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3040) took 2344 ms (client timeout 2273 ms). Trace:
I20260709 13:02:24.166917  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:24.124250  7792 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.004s
I20260709 13:02:24.169844  1553 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
W20260709 13:02:24.125824  7929 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.425s	user 0.009s	sys 0.011s
I20260709 13:02:24.172911  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.188030  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:24.165249  1703 rpcz_store.cc:269] 0709 13:02:21.783783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.783941 (+   158us) service_pool.cc:224] Handling call
0709 13:02:24.090449 (+2306508us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54002e86d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:24.128579 (+ 38130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236122880}
I20260709 13:02:24.186447  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:24.190055  1710 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.186604  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:24.162258  2346 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 11
I20260709 13:02:24.133371  7897 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:24.171766  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:24.194144  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.187973  1702 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:24.130973  7903 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:24.126955  7447 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:24.154340  7751 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:24.174975  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 238583us
I20260709 13:02:24.206003  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:24.211206  1631 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:24.204963  1627 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
W20260709 13:02:24.212757  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 43533.2us
I20260709 13:02:24.167092  7950 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.121433  7796 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:24.191002  2346 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 16.12
I20260709 13:02:24.218981  2346 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Aborting config change with OpId 16.12: Aborted: Op aborted by new leader
I20260709 13:02:24.174117  7821 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.220885  1628 raft_consensus.cc:2410] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 19 because replica has last-logged OpId of term: 18 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:02:24.188124  7983 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:02:24.230548  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 68732.5us
W20260709 13:02:24.231494  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 95117.1us
I20260709 13:02:24.235321  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:24.229159  2346 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:24.130548  7920 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.242278  1629 raft_consensus.cc:2410] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:02:24.205510  7929 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.245895  1706 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:24.246596  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.227839  7733 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.257936  1631 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:24.251606  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 216966us
I20260709 13:02:24.261168  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:24.263338  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:24.266808  1703 raft_consensus.cc:2393] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:24.265532  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:24.227113  7751 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.251726  7953 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:02:24.273466  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.276046  7869 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.276916  7869 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.278375  7842 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:24.278831  7842 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.279176  7842 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.276129  7662 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:24.279903  7831 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:02:24.280294  7662 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:24.280762  7662 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:24.284020  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:24.284919  1708 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:24.271034  7923 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:24.287528  7920 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.245065  7983 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:24.289031  7923 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:24.290366  7923 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.291657  7983 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:24.292992  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:24.293535  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.294342  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:24.294929  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:24.295584  1706 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.296269  7751 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.296890  7751 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:24.303004  1705 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:24.306850  1636 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:24.317830  7949 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.294975  1707 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:24.316215  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:24.316816  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:24.317166  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:24.324311  7796 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:24.317044  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:24.325747  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.272188  7953 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.294963  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:24.326578  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:24.327555  1676 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.880s (SENT)
I20260709 13:02:24.328359  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.328598  7751 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:24.332053  7844 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:24.334272  7714 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:24.334828  7714 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:24.336256  2318 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.336099  7714 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:24.336823  7914 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.337893  7714 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:24.316615  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:24.340411  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:24.328035  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:24.353376  1630 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:24.326488  7953 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:24.351198  1675 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.363700  1553 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:24.371169  1631 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:24.333370  7844 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:24.374702  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:24.376060  1631 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:24.380488  1633 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:24.381310  1632 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:02:24.382695  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.259788  7950 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:24.360826  7855 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.005s
I20260709 13:02:24.402055  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.402917  1628 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:24.317991  7894 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.410439  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:24.412477  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:24.357122  7869 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.416741  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.380048  7844 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.419993  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.449401  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:24.455343  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:24.464954  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:24.466305  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:24.466879  1634 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:24.467168  1635 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:24.466552  2350 raft_consensus.cc:2955] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:24.459496  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:24.476083  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:24.476806  1557 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.403424  7986 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:24.416220  7914 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:02:24.490465  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 155722us
W20260709 13:02:24.504508  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.406430  7938 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.077s
W20260709 13:02:24.549590  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 221930us
W20260709 13:02:24.549594  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.464186  7844 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:24.502521  2352 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:24.403568  7920 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:02:24.551276  7937 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:24.638499  7986 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:24.551713  7905 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:24.640463  7953 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:24.552155  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:24.552448  7907 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:24.554441  8006 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:24.555608  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:24.556221  7950 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:24.561709  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 225585us
W20260709 13:02:24.566295  1457 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 189) took 25924 ms (client timeout 15000 ms). Trace:
I20260709 13:02:24.567812  7895 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:24.475344  8011 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.166s
I20260709 13:02:24.575553  1710 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:24.577225  7894 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.578059  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.658676  1633 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:24.585071  1701 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:02:24.585997  7796 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:24.586935  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 189839us
I20260709 13:02:24.406910  7955 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.590812  1675 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.591055  7987 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:24.591665  7997 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:24.593819  7954 raft_consensus.cc:2955] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:24.469544  7661 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:24.600239  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 49572.3us
I20260709 13:02:24.602005  7903 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:24.590552  7923 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:24.676726  1628 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:24.617035  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:24.633816  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.637537  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.641036  7905 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.551846  7939 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:24.652128  2347 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
W20260709 13:02:24.649233  7765 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.520s	user 0.012s	sys 0.000s
I20260709 13:02:24.660027  1629 raft_consensus.cc:2368] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14.
I20260709 13:02:24.665436  7987 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:24.678479  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:24.690479  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.677521  1557 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:24.614545  7929 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.670204  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.681167  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.656605  7895 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:24.687485  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 70551.5us
I20260709 13:02:24.648897  7662 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.667985  7920 consensus_queue.cc:1243] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:02:24.693295  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 102651us
I20260709 13:02:24.705726  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:24.707075  7953 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:24.709950  7955 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.714542  7987 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.716500  7987 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:24.721587  1633 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.722890  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:24.724885  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46551.4us
I20260709 13:02:24.725116  1628 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:24.727234  7898 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:24.730742  7923 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:24.732522  7907 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:24.733026  7662 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.733585  7937 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:24.735085  7957 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.705765  7661 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:24.730492  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 61363.9us
I20260709 13:02:24.685369  7905 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:24.692915  7820 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:24.663671  7842 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:24.751034  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:24.699908  7986 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:24.683873  8012 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:24.764642  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:24.757777  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:24.757968  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:24.758500  1628 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:24.652783  1457 rpcz_store.cc:269] 0709 13:01:58.641894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.644129 (+  2235us) service_pool.cc:224] Handling call
0709 13:02:24.566004 (+25921875us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:14.362395 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:14.588754 (+226359us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:14.588813 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:02:14.588826 (+    13us) tablet.cc:662] Decoding operations
0709 13:02:14.802861 (+214035us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:14.802963 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:14.802997 (+    34us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:02:14.871798 (+ 68801us) tablet.cc:701] Row locks acquired
0709 13:02:14.871852 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:02:14.872082 (+   230us) write_op.cc:270] Start()
0709 13:02:14.872263 (+   181us) write_op.cc:276] Timestamp: P: 1783602134872007 usec, L: 0
0709 13:02:14.872289 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:02:14.873222 (+   933us) log.cc:844] Serialized 8574 byte log entry
0709 13:02:15.427856 (+554634us) op_driver.cc:464] REPLICATION: finished
0709 13:02:15.554178 (+126322us) write_op.cc:301] APPLY: starting
0709 13:02:15.573695 (+ 19517us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:16.171710 (+598015us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:16.171881 (+   171us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:16.793099 (+621218us) spinlock_profiling.cc:243] Waited 5.94 ms on lock 0x7b180008f6e0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:17.201527 (+408428us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:17.286650 (+ 85123us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=0,mrs_lookups=0
0709 13:02:17.286713 (+    63us) write_op.cc:312] APPLY: finished
0709 13:02:17.287261 (+   548us) log.cc:844] Serialized 188 byte log entry
0709 13:02:17.466968 (+179707us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:17.483763 (+ 16795us) write_op.cc:454] Released schema lock
0709 13:02:17.501246 (+ 17483us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:17.560056 (+ 58810us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:17.563892 (+  3836us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":576715,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":81835,"apply.run_cpu_time_us":72839,"apply.run_wall_time_us":2000099,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91907,"num_ops":21,"prepare.queue_time_us":391922,"prepare.run_cpu_time_us":14782,"prepare.run_wall_time_us":570331,"raft.queue_time_us":204358,"raft.run_cpu_time_us":5218,"raft.run_wall_time_us":49991,"replication_time_us":555337,"spinlock_wait_cycles":13663104,"thread_start_us":538863,"threads_started":5,"wal-append.queue_time_us":245412,"wal-append.run_cpu_time_us":31986,"wal-append.run_wall_time_us":2720901}]]}
I20260709 13:02:24.758877  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:24.762120  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 56525.5us
I20260709 13:02:24.763342  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:24.760093  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:24.749297  7920 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:24.775532  1706 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:24.776660  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:24.693167  7765 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:24.777406  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 140001us
W20260709 13:02:24.759269  7897 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.554s	user 0.013s	sys 0.000s
I20260709 13:02:24.786034  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:24.786315  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.762544  7894 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:24.764509  8012 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.791090  7929 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.801894  7929 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:24.795276  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.761089  7957 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.811975  1633 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:24.795929  7820 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:24.815593  1602 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:24.817881  7983 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:24.818521  7983 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.818998  7983 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.825106  7796 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:24.826336  7796 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:24.826711  7796 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:24.827669  7796 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.828444  7765 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:24.829135  7796 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:24.786038  7986 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:24.786077  7869 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:24.829715  2345 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:24.832214  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:24.833117  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:24.833838  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:24.838121  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.765784  7955 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:02:24.766129  7907 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:02:24.850514  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.855720  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:24.809017  7929 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:24.869365  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:24.860203  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:24.865188  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:24.865648  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:24.870015  1636 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:24.806689  8012 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:24.867194  1557 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:24.811245  7903 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:24.845558  7897 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.874078  1552 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:24.878788  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:24.880229  1674 leader_election.cc:400] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14.
I20260709 13:02:24.887121  1674 leader_election.cc:403] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:02:24.880220  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:24.888731  1556 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 13: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:24.890053  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.892587  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.865101  7820 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:24.898582  7820 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:24.918258  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 50006.2us
I20260709 13:02:24.860998  7920 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:24.925539  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 159108us
I20260709 13:02:24.927729  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.937773  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:24.937964  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:24.942975  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:24.862957  7938 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:02:24.947729  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:24.948513  1631 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:24.925197  7959 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:24.921077  7903 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.937443  7972 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:02:24.950728  7972 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:24.951076  7972 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:24.951756  7903 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:24.952508  8026 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:02:24.953769  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:24.955039  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:24.958928  1631 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:02:24.843284  7953 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 13:02:24.870023  7855 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:02:24.964778  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:24.966017  2347 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:24.952031  7972 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.962980  1675 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:24.967618  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 168637us
I20260709 13:02:24.932160  8011 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:24.943141  7714 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.004s
I20260709 13:02:24.878404  7662 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 13:02:24.977164  1634 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:24.910444  7897 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:24.980549  1554 raft_consensus.cc:2410] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:02:24.960803  7821 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:24.985239  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:25.002586  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:25.010667  1634 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:24.974382  7914 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:24.976302  7997 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:24.978865  7959 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:25.012068  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:24.978530  7831 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:25.011555  7895 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:24.978276  7938 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:25.021308  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:25.026400  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.966838  7949 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 13:02:25.033084  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 268025us
I20260709 13:02:25.033102  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:24.983202  7447 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:02:25.047109  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:25.017421  7914 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:25.018790  7907 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:25.038563  1629 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:02:25.019071  7870 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:25.055044  1634 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:02:25.031373  7842 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:25.033828  7972 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:25.037237  7831 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:25.047058  7762 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:24.985868  7986 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.036329  7854 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:25.086246  1632 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:25.047341  7821 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:25.095494  1629 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
W20260709 13:02:25.074776  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 62839.6us
I20260709 13:02:25.050918  7869 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:25.089329  7855 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:25.045329  7938 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.111497  1629 pending_rounds.cc:85] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 12
I20260709 13:02:25.121611  1629 pending_rounds.cc:107] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:02:25.121994  1629 raft_consensus.cc:2889] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:25.043404  7844 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.128495  1676 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:25.045981  7920 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.118693  7959 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:25.143647  7957 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:25.155400  7821 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:25.135262  7914 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.188599  7914 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:25.189579  8024 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:02:25.180531  7897 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:25.175514  7831 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:25.192932  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.193269  8033 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:02:25.196046  8026 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:02:25.159297  7905 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.178952  7895 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:25.214728  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.218245  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 94677.4us
W20260709 13:02:25.222074  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 262631us
W20260709 13:02:25.232932  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 187760us
W20260709 13:02:25.231957  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 267226us
I20260709 13:02:25.239516  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:25.245641  8034 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:02:25.247772  1706 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:02:25.251283  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.176455  8039 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:02:25.249091  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:25.253365  1560 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:25.210801  7997 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.259421  1552 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:25.209892  7855 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:25.222827  7907 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:25.245649  8028 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:02:25.266780  7796 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.274063  7938 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:25.274619  7762 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:25.275975  1559 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:25.276333  7987 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.559s	user 0.016s	sys 0.000s
W20260709 13:02:25.276911  7939 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.004s
I20260709 13:02:25.259712  7959 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.279426  7905 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:25.304790  7894 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:02:25.282454  8031 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:02:25.283918  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:25.306627  2352 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:25.242856  7821 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:25.307698  7821 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.310276  7821 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:25.284519  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:25.288316  1703 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:25.268236  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:25.311265  7855 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:25.312606  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44499.8us
I20260709 13:02:25.314381  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:25.315140  7950 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.293825  7796 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.261283  1707 raft_consensus.cc:2955] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:02:25.297170  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 57645.2us
I20260709 13:02:25.300513  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:25.317078  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:25.317672  1710 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:25.317811  1552 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:25.303750  7662 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 13:02:25.333133  7661 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.002s
I20260709 13:02:25.334195  1633 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:02:25.334592  1633 pending_rounds.cc:85] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:02:25.334828  1633 pending_rounds.cc:107] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:02:25.335075  1633 raft_consensus.cc:2889] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:25.337289  1455 catalog_manager.cc:5697] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 9 to 12, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 12 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:25.339766  1455 catalog_manager.cc:5697] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:25.346977  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:25.349156  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:25.349911  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:25.353850  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:25.354839  8032 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 13:02:25.341274  1455 catalog_manager.cc:5697] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 11 to 14, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:25.315754  7939 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:25.369988  2318 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:25.373594  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 48989.3us
I20260709 13:02:25.365859  7949 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:25.369922  8053 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
W20260709 13:02:25.383901  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 67237.4us
I20260709 13:02:25.366155  7987 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:02:25.386032  1552 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
W20260709 13:02:25.368772  7662 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:25.396126  1675 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:25.407524  1710 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:25.412497  7796 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:25.413321  7923 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:25.414215  7923 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:25.296372  7997 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.326772  7950 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.271788  7907 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.366410  7751 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:25.374130  7920 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:25.434458  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:25.322914  8039 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:02:25.374413  8050 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:02:25.391580  7844 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:25.432209  8028 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:02:25.424502  7987 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:02:25.494671  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 60259us
W20260709 13:02:25.495061  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.423472  7950 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:25.420783  7997 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:25.415652  7923 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.506628  1455 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 8 to 13, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:25.511030  1455 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 8 to 10, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 10 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:25.518937  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:25.524117  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 128127us
I20260709 13:02:25.498198  7949 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:25.530241  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 160366us
I20260709 13:02:25.502362  7987 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:25.427201  7920 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.539870  1455 catalog_manager.cc:5697] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 5 to 13. New cstate: current_term: 13 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:25.517823  7447 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEARNER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:25.541407  1455 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:25.547988  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:25.548933  1703 raft_consensus.cc:2410] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:02:25.562322  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:25.563730  1710 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:25.577051  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:25.578616  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:25.580907  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:25.585865  8062 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.586903  8062 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.588543  7987 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.589308  7950 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:25.590111  7987 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:25.590197  7950 raft_consensus.cc:2764] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:02:25.591050  7950 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:25.595741  8057 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 13:02:25.596314  8057 raft_consensus.cc:1068] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:25.531813  7949 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.597894  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:25.598702  8056 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:02:25.599943  7949 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:25.600703  7949 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:25.602525  1601 leader_election.cc:400] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:25.602823  1601 leader_election.cc:403] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:02:25.603156  7662 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:25.594481  1455 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 6 to 12, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 12 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:25.614661  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:25.616379  1558 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:25.619339  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:25.622309  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46547us
I20260709 13:02:25.622645  1710 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:25.620286  1455 catalog_manager.cc:5697] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 7 to 9, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 9 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:25.604638  8055 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:25.628079  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:25.608767  7949 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16.
I20260709 13:02:25.632507  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:25.626724  7972 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.000s
I20260709 13:02:25.634891  1455 catalog_manager.cc:5697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:25.636987  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:25.637738  1710 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:25.631559  7662 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.008s
W20260709 13:02:25.643437  7869 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.000s
I20260709 13:02:25.643972  7869 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:25.644819  1455 catalog_manager.cc:5697] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 9 to 10, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 10 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:25.646693  8057 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:25.646661  1455 catalog_manager.cc:5697] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:25.649710  8057 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:25.650979  8057 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.651516  2318 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:25.652974  7907 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:25.653786  8070 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:25.656046  7893 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:25.656473  7893 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:25.657918  7912 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:25.658452  7912 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:25.638019  7950 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:25.661142  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1051) took 2967 ms. Trace:
I20260709 13:02:25.661684  8039 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:02:25.662146  8060 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:02:25.630501  8055 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.661715  1701 rpcz_store.cc:276] 0709 13:02:22.694086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.846901 (+152815us) service_pool.cc:224] Handling call
0709 13:02:24.114456 (+1267555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.663168 (+548712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.659586 (+996418us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54002e86d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:25.660655 (+  1069us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002e86d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:25.661010 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3712652288}
W20260709 13:02:25.668430  1676 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.640s (SENT)
I20260709 13:02:25.669047  1676 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:25.674456  1455 catalog_manager.cc:5697] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:25.680482  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:25.698546  1710 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:25.665750  8055 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:25.665961  8062 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:25.666640  7893 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.705538  1455 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:25.703457  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:25.709365  1701 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:25.712260  8021 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.916s	user 0.005s	sys 0.001s
I20260709 13:02:25.712431  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:25.713199  1710 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:25.714284  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3061) took 1899 ms. Trace:
I20260709 13:02:25.714387  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:25.715718  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:25.716465  1710 raft_consensus.cc:606] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:25.716789  1710 raft_consensus.cc:614] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:25.698655  7820 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:25.738974  1710 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [13.10-15.11]
I20260709 13:02:25.741235  1710 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:02:25.714982  1705 rpcz_store.cc:276] 0709 13:02:23.814969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.304012 (+489043us) service_pool.cc:224] Handling call
0709 13:02:24.304362 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.304932 (+   570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.304961 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.342269 (+ 37308us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.619819 (+277550us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:24.650357 (+ 30538us) log.cc:844] Serialized 14 byte log entry
0709 13:02:24.653273 (+  2916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.653366 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.896684 (+243318us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b18002a7fc0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:25.611716 (+715032us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b24001df648. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda244739c4 00007fda24471acb 00007fda2462cdc9 00007fda2462ca55 00007fda2462c09d 00007fda2725e10e 00007fda2727588c 00007fda27274bb6 00007fda272747d8 00007fda2727e221
0709 13:02:25.713960 (+102244us) raft_consensus.cc:1685] finished
0709 13:02:25.713998 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.714191 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2594,"prepare.queue_time_us":30032,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":11435,"raft.run_cpu_time_us":438,"raft.run_wall_time_us":12664,"spinlock_wait_cycles":51819008,"thread_start_us":158365,"threads_started":1,"wal-append.queue_time_us":307654}
I20260709 13:02:25.750984  1455 catalog_manager.cc:5697] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:25.743335  7893 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:25.757326  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 88942.8us
I20260709 13:02:25.759439  1455 catalog_manager.cc:5697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 9 to 10, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 10 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:25.761173  1455 catalog_manager.cc:5697] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:25.755790  7920 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:25.762205  7983 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.943s	user 0.009s	sys 0.009s
I20260709 13:02:25.796989  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 2980) took 1059 ms. Trace:
I20260709 13:02:25.799916  1635 rpcz_store.cc:276] 0709 13:02:24.737518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.737637 (+   119us) service_pool.cc:224] Handling call
0709 13:02:25.796658 (+1059021us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54002db9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:25.796882 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422839040}
I20260709 13:02:25.792889  7983 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.804068  1701 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
W20260709 13:02:25.789273  8012 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.853s	user 0.012s	sys 0.002s
W20260709 13:02:25.812225  8071 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 13:02:25.813030  8011 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.833s	user 0.014s	sys 0.000s
I20260709 13:02:25.813562  8011 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14.
I20260709 13:02:25.818414  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:25.819420  1554 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:25.819665  7895 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:02:25.824076  7914 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.635s	user 0.006s	sys 0.005s
I20260709 13:02:25.824440  7796 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:25.825199  7796 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.827863  7796 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:25.828853  7949 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:25.830394  7650 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:25.830827  7650 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:25.831758  7650 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.833662  8057 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.834255  8057 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.836935  7869 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:25.837528  7869 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.839936  7869 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:25.844812  1633 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:25.898945  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45458.9us
I20260709 13:02:25.899363  1703 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:25.899772  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:25.900184  2351 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:25.899430  1527 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:25.833412  7650 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:25.904011  7844 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:25.904524  7844 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.904881  7844 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:25.859215  8038 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:25.906584  8038 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:25.907518  8038 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:25.908888  8038 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.898268  7949 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.909900  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:25.910717  1634 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:25.912024  7949 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:25.917739  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 54526.5us
I20260709 13:02:25.923990  1634 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:25.859596  7855 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.954849  2351 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:25.958287  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:25.924196  8030 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.037s	user 0.004s	sys 0.017s
I20260709 13:02:25.956388  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:25.953464  1676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.641s (SENT) [suppressed 22 similar messages]
W20260709 13:02:25.965837  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.964818  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:25.967093  1553 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:25.970183  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:25.917644  8057 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:25.920254  7870 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:25.974316  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:25.898245  8069 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:02:25.975921  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:25.973084  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:25.975052  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:25.993422  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:25.996224  1552 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:02:25.852353  8012 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.998026  2344 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
W20260709 13:02:26.001093  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 101427us
I20260709 13:02:26.006081  1561 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:02:26.001956  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:25.926673  7949 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:26.061641  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:26.004137  1555 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:02:25.963964  7855 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:25.961437  7959 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:02:25.991173  7869 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:26.013720  7907 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:02:26.016043  8012 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:26.022105  1629 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:26.022482  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:26.028278  7844 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.039361  7943 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.007923  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:26.052537  7762 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:26.061631  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:26.062542  1701 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:26.067368  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:26.102365  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:25.971642  7950 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.004724  7896 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:26.089388  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:26.106137  1634 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:26.092447  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.106510  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:26.107256  1706 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:26.073324  8069 consensus_queue.cc:1243] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:02:26.109997  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 102198us
I20260709 13:02:26.107002  1629 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:26.118101  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:26.118355  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.085297  7869 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:26.138720  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:26.138931  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:26.139740  1701 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:26.128901  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.096514  8011 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:26.150688  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:26.151211  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:26.000928  7870 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.082221  7949 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.120419  7447 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:26.094238  8055 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:26.127660  7796 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.127215  7914 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.142899  7869 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:26.222692  1707 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:26.108804  7943 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:26.120306  7950 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.236411  1701 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:26.150784  7959 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:26.226343  8062 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.275249  1632 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:26.271358  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2984) took 1527 ms. Trace:
I20260709 13:02:26.126888  7896 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.231457  7957 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:02:26.161376  8011 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.285055  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:26.239235  8070 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
W20260709 13:02:26.286103  8077 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.055s	user 0.003s	sys 0.000s
I20260709 13:02:26.242388  7796 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:26.275662  7983 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:26.297856  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 144003us
I20260709 13:02:26.302091  1704 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:02:26.314499  1557 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:02:26.253614  7914 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.290560  7957 consensus_queue.cc:1243] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:02:26.330252  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:26.316247  7949 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.337420  1702 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:26.277856  1628 rpcz_store.cc:276] 0709 13:02:24.743954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.776416 (+ 32462us) service_pool.cc:224] Handling call
0709 13:02:24.776742 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.785690 (+  8948us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:24.830305 (+ 44615us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.354094 (+523789us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:26.042831 (+688737us) spinlock_profiling.cc:243] Waited 8.29 ms on lock 0x7b18000ca0c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a
0709 13:02:26.043855 (+  1024us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.270506 (+226651us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.270947 (+   441us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:26.271028 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.271131 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.271283 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40761,"prepare.queue_time_us":45307,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":3019,"raft.queue_time_us":149764,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":87432,"spinlock_wait_cycles":72893824,"thread_start_us":275154,"threads_started":2,"wal-append.queue_time_us":172698}
I20260709 13:02:26.282503  8062 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.339486  7959 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:02:26.349556  8029 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.226s	user 0.009s	sys 0.033s
I20260709 13:02:26.342903  8065 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:02:26.344482  8027 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.462s	user 0.004s	sys 0.013s
I20260709 13:02:26.345274  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:26.358446  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2978) took 1621 ms. Trace:
I20260709 13:02:26.316493  7896 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.359165  1627 rpcz_store.cc:276] 0709 13:02:24.736684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.749168 (+ 12484us) service_pool.cc:224] Handling call
0709 13:02:24.765282 (+ 16114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.765665 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.765696 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.789840 (+ 24144us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.966056 (+176216us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:24.966107 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.966186 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.348212 (+1382026us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:26.350964 (+  2752us) raft_consensus.cc:1685] finished
0709 13:02:26.350987 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.358338 (+  7351us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5794,"raft.queue_time_us":158,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":11566,"spinlock_wait_cycles":11675520,"thread_start_us":145105,"threads_started":1,"wal-append.queue_time_us":228442}
W20260709 13:02:26.369685  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 276126us
I20260709 13:02:26.374557  8062 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.375346  8062 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:26.379112  7914 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.381064  7796 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:26.382053  8055 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:26.382315  7950 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.390089  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:26.362892  7855 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:26.386026  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:26.398258  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:26.368825  7844 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.392373  7765 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:26.400224  7762 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:26.402259  8011 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.405939  7870 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:26.406087  7943 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:26.390620  7896 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:26.451099  7765 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.452584  7762 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:26.425885  1634 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:26.453317  7762 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:26.416112  8023 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.527s	user 0.004s	sys 0.015s
W20260709 13:02:26.437047  1558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3109) took 1587 ms (client timeout 2041 ms). Trace:
W20260709 13:02:26.454649  1558 rpcz_store.cc:269] 0709 13:02:24.849894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.850072 (+   178us) service_pool.cc:224] Handling call
0709 13:02:26.436824 (+1586752us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b5400337cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:26.436970 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1245225856}
I20260709 13:02:26.456476  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3236) took 2090 ms. Trace:
I20260709 13:02:26.444625  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1106) took 2326 ms. Trace:
I20260709 13:02:26.457710  2345 rpcz_store.cc:276] 0709 13:02:24.117957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.134406 (+ 16449us) service_pool.cc:224] Handling call
0709 13:02:24.144171 (+  9765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.851147 (+706976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.444145 (+1592998us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b540002bf54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:26.444481 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1576070912}
I20260709 13:02:26.418030  7855 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:02:26.451181  7896 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.418730  7912 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:26.466645  8039 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:26.467608  7896 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.467334  8039 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.470310  8039 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:26.457046  1630 rpcz_store.cc:276] 0709 13:02:24.365845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.368300 (+  2455us) service_pool.cc:224] Handling call
0709 13:02:24.368798 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.403463 (+ 34665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.403521 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.497413 (+ 93892us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.756585 (+259172us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:24.756804 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.756894 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.415458 (+1658564us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:26.456042 (+ 40584us) raft_consensus.cc:1685] finished
0709 13:02:26.456076 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.456393 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15070,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":10926,"spinlock_wait_cycles":59901312,"thread_start_us":162788,"threads_started":1,"wal-append.queue_time_us":213526}
I20260709 13:02:26.480474  7765 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:26.450579  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.483335  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.485802  1555 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:26.489152  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:26.489938  1561 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:26.495918  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:26.505764  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 433665us
I20260709 13:02:26.507721  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:26.512867  1561 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:02:26.514923  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:26.519888  1558 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [16.9-17.10]
I20260709 13:02:26.539115  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:26.539647  1555 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 13:02:26.538112  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 192895us
I20260709 13:02:26.543385  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:26.547741  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:26.548192  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:26.552896  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.549343  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:26.555619  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:26.555902  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.556631  1561 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:26.558157  7796 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:26.541399  8093 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:26.565814  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:26.566614  1707 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:26.567812  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:26.568692  1710 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:26.559486  8065 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:26.569437  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:26.545184  7844 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:26.543831  7943 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:02:26.549800  8011 raft_consensus.cc:2764] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:02:26.559185  7914 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:26.560935  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:26.561679  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.577740  2352 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:26.561754  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:26.563087  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:26.586232  1552 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:26.586544  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1317) took 1061 ms. Trace:
I20260709 13:02:26.582808  2352 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:26.563902  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.564592  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.564824  1709 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:26.565351  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:26.565708  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:26.590054  1701 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:26.594763  1561 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:26.597213  1558 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:26.545794  7869 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.592357  7765 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:26.603320  7869 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.592011  2352 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.604948  2352 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:26.610085  7762 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:26.613060  7762 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:26.583207  7914 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:26.598291  8011 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:26.614396  8011 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.614420  7914 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.616178  7914 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:26.616897  8011 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.591902  8062 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:26.618355  8062 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.620600  8062 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.622000  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.623592  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.587007  2347 rpcz_store.cc:276] 0709 13:02:25.524458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.524686 (+   228us) service_pool.cc:224] Handling call
0709 13:02:25.536057 (+ 11371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.536521 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:25.577539 (+ 41018us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.433249 (+855710us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b18002caa00. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:02:26.585604 (+152355us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.585668 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:26.585868 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.585963 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.586138 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5171,"prepare.queue_time_us":31324,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":30635,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":207,"spinlock_wait_cycles":62935424,"thread_start_us":121661,"threads_started":1,"wal-append.queue_time_us":979706}
W20260709 13:02:26.625430  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.603083  1552 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:26.628576  1555 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:26.631361  1701 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:26.588398  8093 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:26.634079  8099 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:26.637624  8093 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:26.642822  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 82480.6us
I20260709 13:02:26.638418  8099 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:26.649976  8090 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:02:26.650939  7831 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:26.651108  8099 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:26.648124  7972 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:02:26.662987  7765 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:26.664458  7898 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:26.664922  7898 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:26.666075  8090 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.665804  7898 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.666949  7898 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:26.666618  8090 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.647925  7943 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.670261  7943 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.672437  7943 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:26.673106  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:26.652431  7972 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:26.678292  7972 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:26.679327  7972 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.680205  1558 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:26.680629  7972 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:26.680716  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.681459  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.682184  1558 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:26.683076  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.653597  7831 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:26.684470  1558 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:26.685683  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.686270  1561 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:26.687753  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:26.689227  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:26.689124  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:26.686380  1559 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:26.685886  1675 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:26.646107  8100 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.704068  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:26.715660  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.707976  1675 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:26.716421  1553 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:26.708798  7894 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:26.713105  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:26.713212  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:26.713903  1702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "169e9f6ed9684060b36e93c778925461"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:26.718277  1702 raft_consensus.cc:606] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:26.718814  1702 raft_consensus.cc:614] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:26.714598  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:26.717612  2345 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:26.718035  2347 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:26.720824  2350 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:26.722990  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:26.724954  2344 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 17: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:26.729528  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:02:26.730778  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.707067  7765 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:26.735671  1703 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:26.721962  7893 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:26.701718  8092 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:26.735671  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:26.726985  8094 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:02:26.732149  7923 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.110s	user 0.008s	sys 0.021s
I20260709 13:02:26.741583  1526 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:26.756137  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:26.755172  7923 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.009s	sys 0.021s
W20260709 13:02:26.740446  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 56761.3us
I20260709 13:02:26.728911  7447 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:26.753782  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:26.767499  7923 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:26.769373  7923 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:26.769819  7923 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:26.771986  8099 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:26.772541  8099 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:26.773518  8099 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:26.775381  7821 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:02:26.775666  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.775815  7821 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:26.776712  7821 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.741936  7943 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:02:26.777798  7821 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:26.780362  8065 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:26.733457  7870 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:02:26.780773  7949 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.785418  8069 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:26.750551  7831 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.785902  8069 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:26.785980  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.787022  1627 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:26.788044  1634 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:02:26.788821  2349 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:02:26.790047  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.763960  7893 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:26.798699  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.799902  1631 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
W20260709 13:02:26.801857  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.803489  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.731681  8046 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:02:26.724193  8100 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:26.809805  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.814065  1628 raft_consensus.cc:2410] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:26.813630  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.824575  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:26.825874  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.824060  1630 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:02:26.831840  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 108957us
I20260709 13:02:26.786839  8052 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:26.826570  1627 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:26.825410  1633 pending_rounds.cc:85] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 12
I20260709 13:02:26.837348  1633 pending_rounds.cc:107] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:02:26.837661  1633 raft_consensus.cc:2889] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.837890  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.838073  1559 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:02:26.838917  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 109481us
I20260709 13:02:26.839645  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:26.841282  1675 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:26.841758  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.842429  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:26.843372  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:26.822681  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:26.844364  1631 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:26.796505  7929 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:26.846429  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:26.847268  1553 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:26.764084  8025 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:02:26.843030  1632 raft_consensus.cc:2376] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19. Re-sending same reply.
I20260709 13:02:26.807561  8055 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:26.852747  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.824918  8046 consensus_queue.cc:1243] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:02:26.851966  1706 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:26.866920  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.781356  7949 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:26.865442  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.866797  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.863838  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:26.851778  7938 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:26.851939  7762 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:26.848146  7869 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:26.851513  8090 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:26.877959  1631 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:26.880198  1630 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:26.880427  1553 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:26.881331  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.879935  1674 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:26.898065  7957 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:26.897013  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:26.786921  8069 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:26.875288  8055 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:26.867785  8110 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:26.916647  8110 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.917141  8110 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:26.907116  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.910732  7893 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:26.912024  8109 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:26.912145  8100 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:26.847934  8052 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:26.900485  7869 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:26.947505  7869 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:26.900478  7896 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 13:02:26.950618  8101 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.173s	user 0.003s	sys 0.000s
I20260709 13:02:26.956867  1706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:26.915215  8055 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:26.908345  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 58556.7us
I20260709 13:02:26.886240  7938 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:02:26.926302  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:26.935055  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.981988  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:26.938030  1631 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: []
W20260709 13:02:26.944478  8103 log.cc:927] Time spent T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:26.925349  8025 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:26.908383  8040 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:26.989599  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:26.940030  8069 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:26.919488  7894 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:26.924980  8090 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:26.961844  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:26.985674  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 144458us
I20260709 13:02:26.987285  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:26.987920  1632 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:26.910300  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 70736.7us
I20260709 13:02:26.990763  2348 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:26.903462  7914 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:02:27.012455  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.940824  8109 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:27.016837  8109 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.018430  8109 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:26.994417  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:26.907796  7987 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
W20260709 13:02:27.026453  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.995636  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:26.996673  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 150341us
I20260709 13:02:27.027827  2347 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:26.997121  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.021777  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:27.029155  1631 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:26.918874  7856 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:02:26.956754  8114 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:26.985100  7938 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:27.006867  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:27.002763  8090 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.984524  7907 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.991770  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:27.049279  2344 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:27.015524  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:26.990612  8025 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:27.000334  8055 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:26.989820  8040 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:27.032797  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:26.999167  7894 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.038185  1704 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:02:27.069310  1704 pending_rounds.cc:85] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 12
W20260709 13:02:27.009400  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47683.1us
I20260709 13:02:27.040659  1632 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:26.932502  7447 consensus_queue.cc:1175] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d62ce354d09a4be59c29094425d09bc1" responder_term: 16 status { last_received { term: 16 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 16 index: 12 } }
I20260709 13:02:27.042553  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:27.070658  1707 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:27.019903  8112 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:27.031541  8098 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:27.052790  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.069664  1704 pending_rounds.cc:107] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:02:27.075129  1704 raft_consensus.cc:2889] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:27.076031  1704 pending_rounds.cc:107] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:02:27.076319  1704 raft_consensus.cc:2889] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:27.076924  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44224.9us
I20260709 13:02:27.045291  7870 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:27.048838  8090 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:27.081724  8099 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:27.082443  1631 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:27.052207  7939 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.056826  8025 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:27.084868  1527 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:27.025143  7914 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.074050  7894 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:27.102138  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:27.076105  8112 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:27.051223  8115 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:02:27.079686  8098 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.105568  1704 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:27.113224  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.058336  7907 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.115897  1631 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:27.091897  8099 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:27.127565  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:27.104521  8118 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:27.126885  8079 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.005s	sys 0.005s
I20260709 13:02:27.124065  8112 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:27.146054  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.150105  1631 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:27.146253  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:27.093999  7870 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.125430  7949 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:27.122104  8098 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:27.174533  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.184959  1707 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:02:27.140552  8099 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:27.147184  8079 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.005s	sys 0.005s
I20260709 13:02:27.100620  7939 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.140172  8052 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:27.248205  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 2990) took 1228 ms. Trace:
I20260709 13:02:27.265187  7986 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.273927  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.274636  7939 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:27.274895  1631 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:27.274456  7986 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:27.275247  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.276875  7986 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:27.291502  1559 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:27.197929  8114 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:27.262427  7870 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:27.295361  7907 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:27.139778  8040 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.296939  8040 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:27.301059  7914 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:27.286394  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.303320  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.305011  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.287387  1629 rpcz_store.cc:276] 0709 13:02:26.019380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.148184 (+128804us) service_pool.cc:224] Handling call
0709 13:02:26.148594 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.148883 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:26.149231 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.918505 (+769274us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b180043ce40. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:02:27.247785 (+329280us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.247848 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:27.247915 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.247988 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.248145 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62404,"prepare.queue_time_us":131,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":60,"raft.queue_time_us":50845,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":18783,"spinlock_wait_cycles":10296192,"thread_start_us":278161,"threads_started":1}
I20260709 13:02:27.250840  8117 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:27.312343  7938 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.320206  2347 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:27.305040  1631 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:27.328661  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:27.311353  1558 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
W20260709 13:02:27.309311  7898 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.005s
W20260709 13:02:27.261049  8125 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:02:27.336014  1558 pending_rounds.cc:85] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 8
I20260709 13:02:27.341081  1558 pending_rounds.cc:107] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:02:27.341554  1558 raft_consensus.cc:2889] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:27.339934  1629 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:27.249270  8012 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:27.261245  8087 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.349980  1631 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:27.358817  1554 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
W20260709 13:02:27.364300  7923 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.001s
I20260709 13:02:27.365425  7923 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.363596  1629 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:02:27.407660  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.415750  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.291109  8116 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:27.383021  1631 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:27.338955  8114 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.419256  1629 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:27.422119  1556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.015s	sys 0.018s
W20260709 13:02:27.464689  1556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.016s	sys 0.032s
W20260709 13:02:27.465305  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 319110us
W20260709 13:02:27.438977  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.466428  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:27.467499  1635 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:27.470605  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:27.471117  1704 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:27.441296  1631 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:27.354645  8117 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:27.364498  8119 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:27.488436  8011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.011s	sys 0.045s
I20260709 13:02:27.422808  8087 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.527555  1708 raft_consensus.cc:1217] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 10.10->[11.11-15.13]   Dedup: 11.11->[14.12-15.13]
I20260709 13:02:27.531260  1708 pending_rounds.cc:85] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 11
I20260709 13:02:27.531589  1708 pending_rounds.cc:107] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:02:27.531884  1708 raft_consensus.cc:2889] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:27.495482  7907 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:27.548044  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:27.410597  7914 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.115s
W20260709 13:02:27.530339  8011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.011s	sys 0.045s
I20260709 13:02:27.560432  2345 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEARNER]: Advancing to term 16
I20260709 13:02:27.499779  8114 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.518836  8116 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.556890  8012 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:27.583709  8116 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:27.585107  8025 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:27.586337  8124 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 13:02:27.587394  8122 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:02:27.587782  8025 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:27.552943  7907 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.592620  7907 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:27.593784  2345 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:02:27.595889  2345 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 9
I20260709 13:02:27.596865  2345 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:02:27.597944  2345 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:27.599704  8114 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:27.566412  7662 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:27.599931  2345 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:02:27.601104  2345 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:27.601186  8112 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.555231  8131 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:27.605670  8112 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:27.581218  7972 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.896s	user 0.009s	sys 0.009s
I20260709 13:02:27.607249  7923 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:27.560621  8052 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.608187  8052 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.610008  8098 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.606051  7938 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:27.612032  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:27.617033  1632 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.533s	user 0.012s	sys 0.000s
I20260709 13:02:27.616214  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:27.617841  2345 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:27.618619  1632 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:02:27.592864  8099 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.525641  8119 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:27.626521  8090 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.524s	user 0.009s	sys 0.004s
W20260709 13:02:27.620584  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 150048us
I20260709 13:02:27.628528  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:27.641089  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:27.644145  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.648630  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:27.653026  1553 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:27.638324  8123 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:02:27.660614  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 112670us
I20260709 13:02:27.661532  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:27.663717  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.665270  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.666378  1674 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:27.669698  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:27.671161  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:27.673027  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:27.648268  8134 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:27.657099  8112 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:27.627170  8090 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.682118  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.663641  8011 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:27.678037  8110 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.760s	user 0.008s	sys 0.008s
I20260709 13:02:27.689767  8087 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:27.690583  8109 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.672s	user 0.013s	sys 0.000s
W20260709 13:02:27.691028  8055 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.622s	user 0.012s	sys 0.000s
I20260709 13:02:27.691900  8090 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:27.693099  8119 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:27.673875  8098 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:27.663761  7914 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:02:27.694247  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:27.695184  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:27.695703  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.698246  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:27.700707  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.643344  8117 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.701709  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:27.674971  7949 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:27.724485  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.746955  1633 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:27.741794  1707 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:27.706907  8099 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:27.704566  8135 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:27.708873  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:27.755867  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 74434.3us
I20260709 13:02:27.758750  1634 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:27.748275  7949 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:27.750674  8055 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:27.723467  8110 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.709050  8011 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.772452  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:27.750087  8052 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:27.755604  7662 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:27.753700  8087 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:27.786022  1707 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:27.760511  8131 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:27.767138  7949 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:27.761405  8142 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:02:27.785097  8069 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.783s	user 0.007s	sys 0.008s
I20260709 13:02:27.808099  8150 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:27.809449  8150 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.813581  8150 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:27.815508  8110 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:27.771502  8055 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:27.841769  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1120) took 1156 ms. Trace:
W20260709 13:02:27.845081  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 172160us
I20260709 13:02:27.844620  2349 rpcz_store.cc:276] 0709 13:02:26.685765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.705686 (+ 19921us) service_pool.cc:224] Handling call
0709 13:02:26.723684 (+ 17998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.797105 (+ 73421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.834891 (+1037786us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b5400540e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:27.841678 (+  6787us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102321024}
W20260709 13:02:27.848791  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.848444  1707 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:02:27.850978  1707 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:27.826604  8055 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:27.826917  8150 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.835444  8131 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.863607  8152 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:27.864609  8152 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.879004  8038 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:27.867792  8152 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:27.880735  8152 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:02:27.882897  8040 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.585s	user 0.012s	sys 0.003s
I20260709 13:02:27.892125  2345 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
W20260709 13:02:27.891458  7894 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.769s	user 0.011s	sys 0.003s
I20260709 13:02:27.888514  7662 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:27.893797  1705 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:27.892922  8131 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.847376  8110 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:02:27.895435  7662 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:27.881700  8150 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:27.909411  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.910218  8011 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:27.908430  8150 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:27.923321  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.924216  1632 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:27.925266  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.926316  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:27.927642  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:27.926412  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:27.929556  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.929730  1632 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:02:27.929868  1707 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:02:27.930222  1707 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 11
I20260709 13:02:27.930612  1707 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 16.12
I20260709 13:02:27.930946  1707 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Aborting config change with OpId 16.12: Aborted: Op aborted by new leader
I20260709 13:02:27.931463  1707 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:27.932446  2318 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.608s (SENT)
I20260709 13:02:27.932567  8065 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:27.932983  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:27.934851  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.936026  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:27.823226  8142 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.937403  1704 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:27.939421  8142 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:27.940496  8131 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:27.940521  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:27.941442  1552 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:27.943917  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:27.944473  1556 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:27.945168  1635 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:27.947751  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:27.948246  1632 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:27.954038  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:27.955932  8148 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.128s	user 0.000s	sys 0.003s
W20260709 13:02:27.956617  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.957625  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.961057  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:27.963434  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:27.959739  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:27.930387  7831 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:27.970297  1556 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:27.972172  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:27.973356  1635 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:27.973906  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:27.973898  1635 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:27.974807  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:27.975991  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:27.976682  1559 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:27.893090  7894 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:27.885353  8152 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:27.979974  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:27.975462  1561 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:27.981003  1558 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:02:27.984436  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 135037us
I20260709 13:02:27.985302  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:27.975564  1635 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:27.991190  1635 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:27.993137  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:27.994107  1561 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:27.996918  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:27.969730  8051 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:27.971253  8110 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:27.996433  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:27.983786  7997 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:28.016863  1552 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:28.018078  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:28.024909  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:28.031522  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:28.033623  1635 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:28.007728  8142 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.042094  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 973) took 2079 ms (client timeout 1608 ms). Trace:
I20260709 13:02:27.999879  8149 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:02:28.042583  1557 rpcz_store.cc:269] 0709 13:02:25.962333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.965430 (+  3097us) service_pool.cc:224] Handling call
0709 13:02:28.041710 (+2076280us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b5400389ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:28.041994 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706105600}
I20260709 13:02:28.009310  8110 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:28.034126  8094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.267s	user 0.007s	sys 0.012s
I20260709 13:02:28.016201  7997 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:28.036671  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:28.047968  1561 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:02:28.045179  7661 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:02:28.050920  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 118495us
W20260709 13:02:28.113092  8094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.280s	user 0.007s	sys 0.013s
I20260709 13:02:28.048868  8110 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:28.029141  8157 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:28.136874  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.137977  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:28.155824  1557 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 21: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:28.047418  8153 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:28.116513  8149 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:28.138759  8094 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:28.144047  7894 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:28.049118  8142 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:28.184935  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:28.149124  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:28.189234  1557 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [13.10-15.11]
I20260709 13:02:28.049841  1636 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:02:28.116741  8025 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:02:28.152266  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 115659us
I20260709 13:02:28.119460  7949 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:28.047897  8112 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.128659  7661 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:28.199018  1675 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:28.190742  1636 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:02:28.199905  1636 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 13.10
W20260709 13:02:27.967289  8114 consensus_queue.cc:1175] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba0548b7d2a94e8598475a46c279fe0e" responder_term: 15 status { last_received { term: 14 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:02:28.159725  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 1.13133e+06us
I20260709 13:02:28.174320  8156 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:28.158718  8040 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.129241  7650 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:28.202641  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:28.200253  1636 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:28.224190  1636 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:02:28.224664  1636 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:28.193820  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:28.176482  8041 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.174135  8157 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:28.174253  8069 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:28.216553  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:28.282366  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 65957.5us
I20260709 13:02:28.282853  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:28.283335  1675 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:28.284163  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 54746.1us
I20260709 13:02:28.044919  8152 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:28.226612  7821 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:28.227614  8155 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:28.229333  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:28.229712  8047 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:28.230122  8098 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:28.231076  8150 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:28.314551  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:28.234553  8087 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:28.240530  7912 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:28.242762  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1074) took 1188 ms. Trace:
I20260709 13:02:28.412101  1705 rpcz_store.cc:276] 0709 13:02:27.054792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.054997 (+   205us) service_pool.cc:224] Handling call
0709 13:02:27.055437 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.897565 (+842128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.242354 (+344789us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b54002e95d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:28.242667 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927518976}
I20260709 13:02:28.248077  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1117) took 1879 ms. Trace:
W20260709 13:02:28.254882  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 981) took 1415 ms (client timeout 1619 ms). Trace:
W20260709 13:02:28.415849  1554 rpcz_store.cc:269] 0709 13:02:26.839947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.841888 (+  1941us) service_pool.cc:224] Handling call
0709 13:02:28.254581 (+1412693us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b5400312254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:28.254773 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1111265152}
I20260709 13:02:28.262893  1702 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:28.267112  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:28.275243  8132 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.681s	user 0.003s	sys 0.001s
I20260709 13:02:28.277858  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:28.291353  7914 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:28.285832  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:28.287063  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:28.287040  2317 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:28.424244  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 137696us
I20260709 13:02:28.414191  2352 rpcz_store.cc:276] 0709 13:02:26.371194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.383143 (+ 11949us) service_pool.cc:224] Handling call
0709 13:02:26.383563 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.632541 (+248978us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.632923 (+   382us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.632978 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:26.633014 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.861211 (+1228197us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b18004cbac0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419
0709 13:02:27.947194 (+ 85983us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b18004cbac0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:28.247672 (+300478us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b540055ffd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:28.247741 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.247988 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":441,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":996,"spinlock_wait_cycles":436156416}
I20260709 13:02:28.289988  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:28.205652  7661 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:28.197242  8161 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:28.295900  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:28.293517  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:28.312143  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:28.432935  1636 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:28.313163  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:28.433887  1630 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:28.331064  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:28.434990  2346 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:28.301069  7894 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:02:28.333009  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 103761us
I20260709 13:02:28.334542  2351 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEARNER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:28.333091  2318 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:28.355865  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:28.378537  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:28.419586  8055 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.564s	user 0.006s	sys 0.007s
I20260709 13:02:28.340286  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:28.292019  8092 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:28.339717  8052 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:28.448722  1555 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:28.417928  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1070) took 1413 ms. Trace:
I20260709 13:02:28.449571  1702 rpcz_store.cc:276] 0709 13:02:27.004979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.005133 (+   154us) service_pool.cc:224] Handling call
0709 13:02:27.015923 (+ 10790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.262644 (+1246721us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b54001cbea0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda27651dc0 00007fda276dc1c0 00007fda276dd231 00007fda276d9d8f 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c
0709 13:02:28.417460 (+154816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.417841 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57646336}
W20260709 13:02:28.417965  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.450482  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:28.451448  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.193774  8131 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.129s
I20260709 13:02:28.289063  8156 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:02:28.342314  8158 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:28.200650  8025 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.296090  7903 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:28.336715  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:28.314621  8065 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.204062  8112 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.280097  8040 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.293709  8051 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.421687  8087 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:28.444187  1554 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:28.339587  8155 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:28.452337  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:28.469400  1559 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:28.411307  8137 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.454737  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 177008us
I20260709 13:02:28.486205  1636 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:28.343871  7949 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:28.428977  8152 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:28.454710  7894 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:28.287425  7650 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:28.287575  8041 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:28.476282  8160 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:02:28.277423  7997 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.539340  2347 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 17: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:28.440877  8149 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.462188  8057 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:28.356040  8094 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:28.534118  8126 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.988s	user 0.003s	sys 0.000s
W20260709 13:02:28.453183  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 167440us
I20260709 13:02:28.461277  2346 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:28.486199  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:28.560904  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:28.505553  2349 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:28.518529  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.497676  8168 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:28.570385  7894 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.571408  8168 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:28.476964  8159 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:02:28.573915  8112 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:28.438700  7914 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.519362  8065 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:28.578397  7914 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:28.542026  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:28.503655  7929 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Replica ba0548b7d2a94e8598475a46c279fe0e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
I20260709 13:02:28.579962  7903 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:28.543354  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 91140.5us
I20260709 13:02:28.461611  8055 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.586333  7903 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.587688  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:28.528954  8150 raft_consensus.cc:2764] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:02:28.588967  7903 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:28.479898  8142 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:28.590425  8065 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:28.590857  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:28.590648  8142 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:28.591686  8150 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:28.513623  8052 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.595031  8146 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.795s	user 0.002s	sys 0.000s
I20260709 13:02:28.596027  2353 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:28.593348  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:28.592778  8052 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:28.558684  2348 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEARNER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:28.518746  1553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.004s	sys 0.010s
W20260709 13:02:28.618557  1553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.005s	sys 0.010s
I20260709 13:02:28.564978  8041 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:28.531740  8137 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.567319  7650 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:28.568424  7662 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.666s	user 0.007s	sys 0.007s
I20260709 13:02:28.569425  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:28.528144  8051 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.471822  8156 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:28.629379  8091 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.661s	user 0.012s	sys 0.024s
I20260709 13:02:28.571218  8155 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:28.575994  8161 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:02:28.529592  1631 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.002s	sys 0.015s
I20260709 13:02:28.588492  8055 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:28.652643  8110 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:28.597229  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:28.293309  8069 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.484326  8025 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:28.634024  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:28.637378  2352 raft_consensus.cc:2410] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:02:28.565023  8040 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:28.477097  8157 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:28.640790  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 215489us
I20260709 13:02:28.643761  2349 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:28.645071  1555 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:28.630030  7855 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:28.647380  1631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.003s	sys 0.015s
W20260709 13:02:28.646649  8091 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.700s	user 0.012s	sys 0.024s
I20260709 13:02:28.653563  8171 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:02:28.641326  7662 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.655352  1706 raft_consensus.cc:606] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:28.502202  8047 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:28.637181  7923 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.639663  8041 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:28.640013  8150 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:28.665850  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.667843  8052 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:28.657886  7831 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.685s	user 0.012s	sys 0.001s
I20260709 13:02:28.669126  8040 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:28.656139  8161 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:28.695091  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 61168.2us
I20260709 13:02:28.655018  8137 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:28.696754  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 364805us
I20260709 13:02:28.701972  8025 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.766091  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 89309.4us
I20260709 13:02:28.770522  8052 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.771353  8052 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.719477  8040 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:28.705874  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 176829us
W20260709 13:02:28.715116  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.695982  7986 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:28.659521  8168 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:28.731442  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:28.784814  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 53517.5us
I20260709 13:02:28.786377  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:28.706032  8041 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:28.689356  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 183020us
I20260709 13:02:28.729540  8090 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.710952  7923 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:28.731400  8150 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:28.798445  1602 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:28.801721  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:28.799288  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:28.734766  7831 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:02:28.782665  8040 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:28.779578  8052 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:28.805930  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:28.813620  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:28.813740  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:28.794708  8041 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:28.818288  8153 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:02:28.831297  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1080) took 1556 ms. Trace:
I20260709 13:02:28.832185  8051 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:28.779892  8025 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.837707  8052 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:28.839596  8025 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:28.842394  8173 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:02:28.844018  8144 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.043s	user 0.011s	sys 0.000s
W20260709 13:02:28.844518  8169 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:02:28.822652  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:28.823350  8177 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.818288  7796 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:28.826535  1632 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:02:28.792860  7661 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:28.804569  8090 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.793049  8152 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:28.793972  7986 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:28.830258  8069 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:28.794478  8142 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:28.792598  8181 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:28.834030  8150 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.834702  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2386) took 1289 ms. Trace:
I20260709 13:02:28.877202  1710 rpcz_store.cc:276] 0709 13:02:27.545312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.675871 (+130559us) service_pool.cc:224] Handling call
0709 13:02:28.834432 (+1158561us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002e7cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:28.834593 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2410342656}
I20260709 13:02:28.818599  8038 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:28.885221  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:28.886196  1633 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:28.811707  8091 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:02:28.812192  7929 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:28.815865  8171 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:02:28.845922  1629 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:02:28.851508  1632 pending_rounds.cc:85] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:02:28.853894  7662 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:28.853737  7796 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.856335  7661 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:28.857758  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:28.858953  8175 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:28.831729  1708 rpcz_store.cc:276] 0709 13:02:27.274415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.499993 (+225578us) service_pool.cc:224] Handling call
0709 13:02:27.523284 (+ 23291us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:27.543081 (+ 19797us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:27.574498 (+ 31417us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.890598 (+316100us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b18004a9860. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:28.265171 (+374573us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b18004a9860. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:02:28.636271 (+371100us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:28.639385 (+  3114us) log.cc:844] Serialized 52 byte log entry
0709 13:02:28.639610 (+   225us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:28.639924 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.640174 (+   250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.640279 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.831049 (+190770us) raft_consensus.cc:1685] finished
0709 13:02:28.831076 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.831235 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25769,"prepare.queue_time_us":25967,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":58483,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":3760,"spinlock_wait_cycles":23213440,"thread_start_us":351594,"threads_started":1,"wal-append.queue_time_us":1229110}
I20260709 13:02:28.865574  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1521) took 1834 ms. Trace:
I20260709 13:02:28.868269  7923 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:28.869273  8110 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:28.846160  8135 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.930158  2350 rpcz_store.cc:276] 0709 13:02:27.031655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.033332 (+  1677us) service_pool.cc:224] Handling call
0709 13:02:27.050860 (+ 17528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.285647 (+234787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.285697 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.285758 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:27.285815 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.865198 (+1579383us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b5400643a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:28.865265 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.865456 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530784512}
I20260709 13:02:28.851651  8161 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:28.874320  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:28.884464  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 151984us
W20260709 13:02:28.888984  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:28.934461  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.934340  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:28.889721  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3268) took 1040 ms. Trace:
W20260709 13:02:28.890456  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 121290us
I20260709 13:02:28.901942  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:28.902508  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:28.946095  1633 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:28.946183  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44724.5us
I20260709 13:02:28.916942  1632 pending_rounds.cc:107] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:02:28.947392  1632 raft_consensus.cc:2889] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:28.820765  8087 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.950774  7796 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:28.930824  8135 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:28.871594  8177 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.956228  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1079) took 1682 ms. Trace:
I20260709 13:02:28.927999  8092 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:28.957957  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:28.957466  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 140440us
I20260709 13:02:28.963704  8116 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.961187  7923 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:28.966596  7447 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:02:28.971701  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:28.992785  1676 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:28.951695  8087 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:28.953357  7914 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.968876  7831 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:02:28.965852  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:28.932312  7912 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.960038  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 179592us
I20260709 13:02:28.935395  1631 rpcz_store.cc:276] 0709 13:02:27.848891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.849096 (+   205us) service_pool.cc:224] Handling call
0709 13:02:27.849523 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.966897 (+117374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.968377 (+  1480us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.981074 (+ 12697us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.682468 (+701394us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:28.682518 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.682598 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.889447 (+206849us) raft_consensus.cc:1685] finished
0709 13:02:28.889503 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.889659 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140,"raft.queue_time_us":30666,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"thread_start_us":781739,"threads_started":2,"wal-append.queue_time_us":644944}
I20260709 13:02:28.980063  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:28.960215  8047 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:28.953893  8171 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:28.960409  1709 rpcz_store.cc:276] 0709 13:02:27.273971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.307088 (+ 33117us) service_pool.cc:224] Handling call
0709 13:02:27.348503 (+ 41415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:27.540789 (+192286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.540831 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:27.542072 (+  1241us) log.cc:844] Serialized 52 byte log entry
0709 13:02:27.722652 (+180580us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:27.747402 (+ 24750us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.748592 (+  1190us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.748887 (+   295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.748995 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.953300 (+204305us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b18002cfda0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:27.954236 (+   936us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b18002cfda0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:28.843334 (+889098us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:28.955967 (+112633us) raft_consensus.cc:1685] finished
0709 13:02:28.955990 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.956157 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1043,"prepare.queue_time_us":47423,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":10282,"raft.queue_time_us":175706,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":2901,"spinlock_wait_cycles":23712768,"thread_start_us":328195,"threads_started":2,"wal-append.queue_time_us":211513}
I20260709 13:02:28.987833  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3147) took 1022 ms. Trace:
I20260709 13:02:28.952111  8115 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.989018  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:28.960529  7650 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:28.951977  8110 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.978129  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:29.056747  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 78762us
I20260709 13:02:29.057224  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:29.004215  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.059391  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.005393  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.006413  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.007534  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:29.002840  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 86612.9us
I20260709 13:02:28.997251  1633 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:29.042161  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:28.892457  8181 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.986047  7923 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.044526  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:29.077275  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.078032  1709 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:29.079157  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:29.079839  1709 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:29.046504  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.052757  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.054268  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.082078  1555 raft_consensus.cc:2410] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:29.052206  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:29.082593  1554 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:02:29.007597  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.072329  1561 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:29.072618  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.073098  1701 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:29.074086  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:29.084527  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.085394  1710 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:29.070741  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:29.086409  1704 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:29.074805  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.057945  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.087883  1703 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:29.091075  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.002014  8025 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:29.092325  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.091392  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:29.097319  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:29.047024  7957 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.069890  7893 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:29.043322  8171 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:29.106029  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.102104  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:28.995077  7894 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:29.111349  1705 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:29.103829  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:28.995566  8094 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:29.091351  7923 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:29.089777  8087 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:29.107584  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 142152us
I20260709 13:02:29.113631  7893 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:29.071630  7650 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.106578  7929 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:29.096308  8092 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.992038  8116 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.102305  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:29.042932  7912 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.121078  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:29.113237  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:29.110574  2318 connection.cc:256] RPC call timeout handler was delayed by -1.0082739669999796s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
I20260709 13:02:29.174356  1701 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:29.123629  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:29.103495  8155 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:29.000717  7914 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:28.990965  7986 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.109082  8171 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:29.062574  8115 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.073274  8110 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:29.143841  7929 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.158154  7650 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:29.160697  8181 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:29.162432  8177 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:29.128268  8194 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:29.119401  8189 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:29.181888  7796 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:29.167086  8025 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:29.184377  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 60858.2us
W20260709 13:02:29.186149  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 625299us
I20260709 13:02:29.202441  8157 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:29.121215  7447 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.194031  1710 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:29.272545  1558 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:29.273324  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.274143  1710 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:29.274469  1701 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:29.275102  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.276320  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.276851  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.102782  7957 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.205524  1709 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:29.157568  7662 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:29.219688  1708 raft_consensus.cc:2410] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:02:29.146891  8047 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.203162  7831 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:29.226226  8194 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:29.249665  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:29.252276  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:29.186156  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:29.316124  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 130100us
I20260709 13:02:29.263579  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:29.129434  7894 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:29.218228  7929 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:29.227176  8190 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.294s	user 0.000s	sys 0.002s
I20260709 13:02:29.278847  1701 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:29.276953  1710 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:02:29.155872  7893 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.284389  8090 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:29.284675  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:29.267753  8177 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:29.288846  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.288430  8157 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:29.289737  8185 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
W20260709 13:02:29.290731  8193 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:02:29.176381  8041 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:29.173763  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 71547.2us
I20260709 13:02:29.222141  8189 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:02:29.245468  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 139920us
I20260709 13:02:29.326476  8091 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:29.179391  8092 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.308409  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:29.307056  1704 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:02:29.321240  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:29.324448  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 61067.1us
I20260709 13:02:29.314178  1627 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
W20260709 13:02:29.302903  8186 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.347s	user 0.000s	sys 0.002s
W20260709 13:02:29.385076  8197 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
W20260709 13:02:29.386682  8187 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:02:29.386090  8194 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.394671  1675 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:29.328375  8152 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:29.303448  8040 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.443223  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:29.278191  8025 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:29.372799  7831 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:29.532847  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 138266us
I20260709 13:02:29.423694  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:29.305519  7447 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.323441  8203 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.097s
W20260709 13:02:29.555267  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 131706us
W20260709 13:02:29.560077  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.556143  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:29.407557  8189 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:29.500551  8090 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:29.532130  7662 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.531986  8094 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:29.402638  8177 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.556272  7796 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:29.639114  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3146) took 1673 ms. Trace:
I20260709 13:02:29.540325  8051 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:29.619643  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 62882.7us
W20260709 13:02:29.620540  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 312236us
W20260709 13:02:29.621132  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 178049us
I20260709 13:02:29.648805  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:29.650056  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:29.621863  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:29.650701  1630 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:29.628635  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 307482us
I20260709 13:02:29.659507  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:29.636843  8099 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:29.567597  8091 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.639722  1556 rpcz_store.cc:276] 0709 13:02:27.965565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.998347 (+ 32782us) service_pool.cc:224] Handling call
0709 13:02:27.998776 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.002136 (+  3360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.002198 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.584759 (+582561us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b2c00000658. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:02:28.585340 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.782587 (+197247us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:28.782632 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.782723 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.597791 (+815068us) raft_consensus.cc:1685] finished
0709 13:02:29.597822 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.639019 (+ 41197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":384,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":4194,"spinlock_wait_cycles":11419904,"thread_start_us":141794,"threads_started":1,"wal-append.queue_time_us":307381}
W20260709 13:02:29.628051  1554 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260709 13:02:29.703951  1554 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:29.654856  8051 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:29.694900  8025 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:29.667577  8207 raft_consensus.cc:2764] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:02:29.669016  8215 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:29.689061  8208 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:29.690542  8195 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:29.691638  8152 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:29.666230  8194 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:29.693387  7447 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:29.694273  2347 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:29.694195  8106 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:02:29.689344  8156 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.034s	user 0.007s	sys 0.015s
I20260709 13:02:29.680222  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:29.638655  7796 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.706166  8091 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.584633  7894 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.664819  8142 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:29.679677  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:29.767159  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 87584.4us
I20260709 13:02:29.767959  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:29.604323  7662 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:29.609599  8090 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.695817  7650 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:29.639114  8094 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.696327  8117 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:29.643359  8153 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.006s	sys 0.020s
I20260709 13:02:29.640755  8211 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 13:02:29.695040  8155 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.738394  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:29.739305  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:29.788362  2351 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:29.738649  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:29.703250  8110 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:29.742951  2317 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:29.718629  8051 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:29.751667  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:29.730967  8215 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:29.722641  7914 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:29.752782  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:29.753316  8115 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:29.808321  2344 raft_consensus.cc:2410] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:29.753607  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:29.744623  7912 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:29.734958  8156 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:29.756731  7855 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:29.735178  8152 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:29.758112  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:29.822468  1632 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
W20260709 13:02:29.773154  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 125958us
I20260709 13:02:29.773898  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:29.775851  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 127839us
W20260709 13:02:29.824071  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 50266.8us
I20260709 13:02:29.787134  7650 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:29.735482  7447 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.799031  7957 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:29.783375  8214 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:29.742817  7796 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:29.793721  8153 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.007s	sys 0.020s
I20260709 13:02:29.766721  8099 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:29.774781  8195 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:29.696645  7831 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.772397  8191 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:29.774396  8207 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:29.816341  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 185160us
I20260709 13:02:29.757427  8087 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
W20260709 13:02:29.881462  8206 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:02:29.831187  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:29.831446  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:29.832116  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.832844  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.910940  1704 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:29.780575  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 224498us
I20260709 13:02:29.853873  1709 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 13:02:29.731597  8208 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.862265  1527 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:29.824429  7855 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:29.796558  8117 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.826349  8116 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:29.800932  8155 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:29.853824  7986 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:29.855995  8157 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:29.898123  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:29.827252  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 84425.8us
I20260709 13:02:29.901816  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.895555  8177 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:29.903805  7937 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:29.904615  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.905449  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.910076  1710 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:29.871124  8092 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:29.912634  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.873649  8214 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:29.915095  1702 raft_consensus.cc:2368] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15.
W20260709 13:02:29.914309  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:29.913961  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 89222.8us
I20260709 13:02:29.935555  1709 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:29.925846  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.927218  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:29.933095  1556 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:29.934788  1557 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:29.875185  7447 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.939095  1555 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:29.940232  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.914158  8195 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:29.921052  7893 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.967154  1710 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
W20260709 13:02:29.968701  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 106587us
I20260709 13:02:29.969029  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.970034  1704 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:02:29.966378  1675 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:29.947695  8171 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.724s	user 0.021s	sys 0.000s
I20260709 13:02:29.973063  8171 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.976605  8171 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:29.944615  8214 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:29.978251  8226 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:02:29.979835  8177 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:29.935168  7957 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:02:29.981525  7957 consensus_queue.cc:1243] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:02:29.983601  8204 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.698s	user 0.002s	sys 0.000s
I20260709 13:02:29.944766  7986 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:29.986544  7986 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:29.989631  8213 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:29.990149  8213 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:29.990568  8213 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:29.990865  8208 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:29.936502  8215 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:29.992995  8183 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.991s	user 0.002s	sys 0.000s
I20260709 13:02:29.941469  7650 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:29.993904  8090 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:29.973282  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.986562  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.934295  7937 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:30.001111  1702 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:30.002831  1705 raft_consensus.cc:2410] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:30.003695  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:29.905576  8207 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:29.901206  8191 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:29.995687  8099 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:30.021924  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:30.042187  1558 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:29.965876  8116 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:30.044385  7662 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:30.050421  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:30.050807  8205 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:30.051283  1561 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:30.053644  8203 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:30.054284  8203 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:30.055976  8189 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:30.060037  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:30.051429  8205 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.004629  8181 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:30.066617  2352 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
I20260709 13:02:30.067965  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:30.068418  1702 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:30.034596  8230 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:02:30.044342  7923 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.910s	user 0.017s	sys 0.014s
I20260709 13:02:30.073415  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:30.075349  1561 raft_consensus.cc:2410] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:29.937399  8152 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:30.099326  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:30.103353  1555 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:30.112421  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:29.996980  7893 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:30.113265  1554 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:30.065270  8205 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:30.059441  8094 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:30.062927  8091 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:30.125936  8207 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:30.128517  7997 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:30.101673  7923 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:30.131618  7923 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:30.133373  1707 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:30.135365  8191 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:30.136276  8218 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.275s	user 0.001s	sys 0.001s
I20260709 13:02:30.073659  8142 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:30.114024  8195 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:30.139166  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 1.14028e+06us
I20260709 13:02:30.140985  8117 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:30.150600  8189 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:30.050839  8116 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:30.074388  8230 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:30.164168  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 231735us
W20260709 13:02:30.161680  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 233346us
I20260709 13:02:30.177397  1554 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:30.166251  8150 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:30.180891  8150 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:30.120328  7937 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:30.182734  7937 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:30.183422  8150 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:30.166677  8117 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:30.184527  8117 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:30.170182  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:30.121352  7914 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.230927  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:30.231519  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:30.234052  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:30.235129  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:30.237192  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.239781  1553 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:30.541042  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:30.784835  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2389) took 1799 ms. Trace:
I20260709 13:02:30.300146  1554 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:30.911406  1708 rpcz_store.cc:276] 0709 13:02:28.985530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.995422 (+  9892us) service_pool.cc:224] Handling call
0709 13:02:30.784420 (+1788998us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540016ce54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:30.784725 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336754560}
W20260709 13:02:30.398739  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 257797us
I20260709 13:02:30.778343  2350 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:30.913394  1557 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:02:30.325014  1555 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:02:30.399091  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 279533us
I20260709 13:02:30.337141  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:30.917613  2345 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:02:30.910969  2346 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:30.312295  7986 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:30.631765  8142 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:30.919123  7986 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:30.558607  8242 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.233s	user 0.000s	sys 0.000s
I20260709 13:02:30.921777  1559 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
W20260709 13:02:30.707033  8051 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.856s	user 0.007s	sys 0.006s
I20260709 13:02:30.923161  8051 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:30.187443  8115 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:30.312649  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:30.915875  2348 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
W20260709 13:02:30.927731  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.241392  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44101.3us
W20260709 13:02:30.756970  7997 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.578s	user 0.013s	sys 0.000s
W20260709 13:02:30.780262  8205 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.000s
I20260709 13:02:30.930873  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:30.931476  1557 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:30.918220  2344 raft_consensus.cc:2376] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16. Re-sending same reply.
I20260709 13:02:30.911242  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:30.947067  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43592 (request call id 45) took 3976 ms. Trace:
I20260709 13:02:30.953557  1706 rpcz_store.cc:276] 0709 13:02:26.956300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.956673 (+   373us) service_pool.cc:224] Handling call
0709 13:02:30.932633 (+3975960us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400224f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d4853 00007fda2c6a260a 00007fda25c287d4 00007fda25c28762 00007fda25c28691 00007fda25c2860c 00007fda25c278c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:30.932707 (+    74us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3903314816}
W20260709 13:02:30.947562  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.948318  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:02:30.379158  8243 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:02:30.958169  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:30.952924  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:30.689987  8231 log.cc:927] Time spent T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.570s	user 0.001s	sys 0.000s
I20260709 13:02:30.226874  8181 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:30.972405  1526 leader_election.cc:400] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15.
I20260709 13:02:30.241869  8236 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:30.978909  1702 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.025s	sys 0.014s
I20260709 13:02:30.978482  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:30.925468  8115 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:30.545956  7447 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:30.374356  8116 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:30.973137  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3107) took 1140 ms. Trace:
I20260709 13:02:30.381141  8203 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:30.341415  8157 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:30.226490  8194 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:02:31.003697  1702 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.025s	sys 0.014s
I20260709 13:02:31.004122  2344 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:30.967631  8246 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:30.940629  7914 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:30.971475  8205 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.003337  1526 leader_election.cc:403] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:30.299633  8150 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:31.002526  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:31.009063  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:31.013586  2351 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.001s	sys 0.022s
I20260709 13:02:31.016980  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:30.967947  8215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.004s	sys 0.030s
I20260709 13:02:31.022851  1708 rpcz_store.cc:276] 0709 13:02:29.832233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.940499 (+1108266us) service_pool.cc:224] Handling call
0709 13:02:30.972420 (+ 31921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.972675 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.972709 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.972759 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:30.972803 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.972855 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.973038 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.185398  7796 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:31.023689  1676 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.027s	sys 0.019s
I20260709 13:02:30.350884  8225 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.656s
I20260709 13:02:30.240236  7650 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.767s
I20260709 13:02:30.956804  8110 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.948436  8038 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:31.183717  2345 raft_consensus.cc:2410] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 13:02:31.185515  2351 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.003s	sys 0.023s
W20260709 13:02:31.183843  1676 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.027s	sys 0.019s
W20260709 13:02:31.187464  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 872508us
W20260709 13:02:31.188009  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.188346  1676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.886s (SENT) [suppressed 4 similar messages]
I20260709 13:02:29.044029  1552 rpcz_store.cc:276] 0709 13:02:27.965822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.019087 (+ 53265us) service_pool.cc:224] Handling call
0709 13:02:28.035277 (+ 16190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.035504 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:28.365883 (+330379us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.369152 (+  3269us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.836998 (+467846us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b1800093d60. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda27290d70 00007fda27284293 00007fda27279cea 00007fda2727a23a 00007fda276e4a2a 00007fda276f40bf 00007fda276f4038
0709 13:02:28.865413 (+ 28415us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b1800093d60. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:28.987405 (+121992us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.987475 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:28.987539 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.987623 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.987767 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":206404,"prepare.queue_time_us":332561,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":106,"raft.queue_time_us":204542,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":44488,"spinlock_wait_cycles":45034752,"thread_start_us":316447,"threads_started":1}
I20260709 13:02:31.186221  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1339) took 1282 ms. Trace:
I20260709 13:02:31.206377  1709 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:31.188530  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.207674  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1086) took 1258 ms. Trace:
I20260709 13:02:31.208189  1709 rpcz_store.cc:276] 0709 13:02:29.949057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.144098 (+1195041us) service_pool.cc:224] Handling call
0709 13:02:31.206084 (+ 61986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.207293 (+  1209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.207591 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71424}
W20260709 13:02:31.217228  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 208299us
I20260709 13:02:31.001783  7997 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:31.190088  2353 rpcz_store.cc:276] 0709 13:02:29.904312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.904545 (+   233us) service_pool.cc:224] Handling call
0709 13:02:29.904881 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.038473 (+133592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.038535 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.040013 (+  1478us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.300711 (+260698us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:30.300771 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.300873 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.185913 (+885040us) raft_consensus.cc:1685] finished
0709 13:02:31.185950 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.186141 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":63221,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":106,"thread_start_us":320925,"threads_started":2,"wal-append.queue_time_us":1019684}
I20260709 13:02:31.226496  1709 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:31.234265  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 267877us
I20260709 13:02:31.234717  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1089) took 1284 ms. Trace:
I20260709 13:02:31.235785  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:31.235376  1708 rpcz_store.cc:276] 0709 13:02:29.950025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.184397 (+1234372us) service_pool.cc:224] Handling call
0709 13:02:31.234153 (+ 49756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.234379 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.234403 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:31.234447 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:31.234494 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.234553 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.234658 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.242923  1561 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:02:31.245385  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1090) took 1279 ms. Trace:
I20260709 13:02:31.246222  1709 rpcz_store.cc:276] 0709 13:02:29.953468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.225813 (+1272345us) service_pool.cc:224] Handling call
0709 13:02:31.226239 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.231999 (+  5760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.232309 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:31.227159  8214 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.249s	user 0.010s	sys 0.004s
I20260709 13:02:31.156049  8087 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:31.184841  8194 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.029404  8155 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:31.159394  8069 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:31.263913  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1140) took 1482 ms. Trace:
I20260709 13:02:31.026516  8153 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:31.067533  8182 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:31.289276  8182 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.289523  8134 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:31.289846  8182 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:31.290270  7914 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:31.290180  8134 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.290957  7914 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.292474  8051 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:31.292999  8134 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:31.216580  7986 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:31.295439  8156 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:31.240047  8215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.005s	sys 0.042s
I20260709 13:02:31.069020  7893 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:31.061829  8152 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.240882  7997 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.297870  8152 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.300042  7997 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:31.300328  8152 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:31.223464  7447 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:31.249451  8222 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15.
I20260709 13:02:31.227766  8211 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:31.076964  8254 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:31.226958  8092 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:31.277576  2347 rpcz_store.cc:276] 0709 13:02:29.781962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.782168 (+   206us) service_pool.cc:224] Handling call
0709 13:02:29.804272 (+ 22104us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.804636 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.804683 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.805358 (+   675us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.909342 (+103984us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:30.168475 (+259133us) log.cc:844] Serialized 14 byte log entry
0709 13:02:30.169125 (+   650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.169255 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.142451 (+973196us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b180049f900. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:31.235120 (+ 92669us) raft_consensus.cc:1685] finished
0709 13:02:31.263593 (+ 28473us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.263825 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":152399,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":70,"raft.queue_time_us":257,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":133,"spinlock_wait_cycles":9122688,"thread_start_us":361656,"threads_started":2,"wal-append.queue_time_us":252940}
I20260709 13:02:31.234539  8025 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.282759  8069 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:31.288342  8153 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.227588  8137 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:31.293267  8252 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.259364  8246 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:31.291469  7914 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.258915  8117 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.221882  8157 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.323622  7796 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:31.327425  8115 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:31.325259  8250 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.540s	user 0.001s	sys 0.003s
W20260709 13:02:31.304193  7929 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.925s	user 0.007s	sys 0.011s
I20260709 13:02:31.330539  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:31.330920  8188 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 2.224s	user 0.004s	sys 0.005s
I20260709 13:02:31.330291  8142 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.331347  8254 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.221216  8038 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.221060  8150 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.227425  8110 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:31.302963  7937 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.120s	user 0.006s	sys 0.012s
I20260709 13:02:31.284273  8194 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:31.248425  8225 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:02:31.019958  8236 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:31.353621  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.453383  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.392976  8025 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.401779  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3104) took 1570 ms. Trace:
I20260709 13:02:31.397604  8252 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.397763  8137 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.332952  8092 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:31.250499  8181 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:31.408603  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 460333us
I20260709 13:02:31.471740  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:31.472203  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:31.348958  7855 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.396s	user 0.008s	sys 0.005s
W20260709 13:02:31.352136  8213 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.361s	user 0.015s	sys 0.001s
I20260709 13:02:31.279029  8214 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.434465  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:31.482100  2347 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:31.437328  8195 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:31.441272  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:31.489144  1633 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 14: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:31.344594  7447 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.444326  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:31.345682  8211 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:31.406760  8149 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:31.423727  8152 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.408588  7986 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:31.457562  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:31.431663  8215 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.469662  2349 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:31.465265  1703 rpcz_store.cc:276] 0709 13:02:29.831214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.143815 (+312601us) service_pool.cc:224] Handling call
0709 13:02:30.144295 (+   480us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:30.166356 (+ 22061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.166406 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:30.180685 (+ 14279us) log.cc:844] Serialized 52 byte log entry
0709 13:02:30.370522 (+189837us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:31.389391 (+1018869us) log.cc:844] Serialized 14 byte log entry
0709 13:02:31.401316 (+ 11925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.401409 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.401455 (+    46us) raft_consensus.cc:1685] finished
0709 13:02:31.401469 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.401707 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20482,"raft.queue_time_us":47642,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":249,"thread_start_us":653491,"threads_started":2,"wal-append.queue_time_us":893038}
I20260709 13:02:31.285413  8087 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.433157  8156 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:31.472927  1632 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:31.472522  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:31.433298  8191 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:31.437551  8155 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:31.471760  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:31.472622  1631 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:31.472563  1628 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:31.349541  8247 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.006s	sys 0.040s
I20260709 13:02:31.437788  8254 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:31.449002  8245 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:31.490010  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 55704.3us
I20260709 13:02:31.490314  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:31.543504  1633 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:31.492614  2345 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:31.544318  2345 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:31.436981  7929 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:31.515159  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:31.509275  7650 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:02:31.538045  8247 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.007s	sys 0.040s
I20260709 13:02:31.443097  8069 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:31.525146  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2399) took 2485 ms (client timeout 2169 ms). Trace:
I20260709 13:02:31.537624  8248 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:31.527060  8025 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:31.527184  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2416) took 4964 ms. Trace:
I20260709 13:02:31.530107  8055 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:31.530611  8092 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.484670  7855 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.510113  8222 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:02:31.497541  8211 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:31.532811  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:31.499579  8195 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.549000  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:31.506464  8051 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.520939  7447 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:31.521590  8156 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.526021  1706 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
W20260709 13:02:31.566705  1707 rpcz_store.cc:269] 0709 13:02:29.039830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.046178 (+  6348us) service_pool.cc:224] Handling call
0709 13:02:31.524893 (+2478715us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002a67d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:31.525066 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2418728704}
I20260709 13:02:31.532356  8153 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:31.572973  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:31.593088  1632 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:31.583148  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 50442.6us
I20260709 13:02:31.557125  7650 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:31.587268  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.248s	user 0.010s	sys 0.035s
I20260709 13:02:31.509179  8152 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.557276  8205 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.506856  8149 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.586831  1628 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:31.521099  8230 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.612097  8230 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:31.612275  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:31.613026  1628 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:31.556195  2345 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.495002  8213 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.614005  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:31.614207  2345 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:31.615272  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:31.616192  2350 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:31.534590  7986 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.618427  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:31.619254  1635 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:31.619656  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:31.491909  8115 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:31.572469  1560 rpcz_store.cc:276] 0709 13:02:26.579614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.579889 (+   275us) service_pool.cc:224] Handling call
0709 13:02:28.043645 (+1463756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.043962 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:28.313283 (+269321us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.958573 (+645290us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.958651 (+    78us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:28.958713 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.958788 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.526989 (+2568201us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400389ad0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:31.527113 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":357240,"prepare.queue_time_us":297169,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":338833,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":11533,"spinlock_wait_cycles":3365418752,"thread_start_us":491252,"threads_started":2,"wal-append.queue_time_us":452538,"wal-append.run_cpu_time_us":2179,"wal-append.run_wall_time_us":1158529}
I20260709 13:02:31.594259  1675 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:31.534914  8215 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:31.597846  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.259s	user 0.010s	sys 0.035s
I20260709 13:02:31.538407  8191 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.543722  8155 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.604637  8153 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.585001  8069 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:31.646497  1628 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:31.587291  8025 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:31.636720  7821 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:31.638896  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.651343  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3166) took 1090 ms. Trace:
I20260709 13:02:31.652467  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:31.653378  1556 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:02:31.654529  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 102785us
I20260709 13:02:31.654904  8098 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.655520  8098 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.657346  8152 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.649159  7912 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:02:31.658624  8195 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:31.659461  8150 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:31.660192  8155 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.649545  7821 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.661403  8191 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:31.646224  8115 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.550186  8245 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.666600  8087 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.667241  8245 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.667686  8137 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.668133  8099 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.668694  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:31.621876  8092 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:31.607411  7447 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:31.661494  7821 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:31.645251  1635 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:31.651816  1559 rpcz_store.cc:276] 0709 13:02:30.560490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.922708 (+362218us) service_pool.cc:224] Handling call
0709 13:02:30.923125 (+   417us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:30.923612 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.923646 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:30.924329 (+   683us) log.cc:844] Serialized 52 byte log entry
0709 13:02:31.353233 (+428904us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:31.650528 (+297295us) log.cc:844] Serialized 14 byte log entry
0709 13:02:31.650895 (+   367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.650981 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.651025 (+    44us) raft_consensus.cc:1685] finished
0709 13:02:31.651041 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.651275 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":300112,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":9634,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":271,"thread_start_us":689587,"threads_started":2,"wal-append.queue_time_us":595715}
I20260709 13:02:31.702016  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:31.685926  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:31.687064  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:31.689029  1634 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:02:31.704537  1553 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:31.676998  1556 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:31.703980  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:31.706589  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:31.672884  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:31.698858  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:31.705907  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:31.709036  1559 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:31.668815  8099 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.668859  8222 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:31.596629  8051 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.649887  8025 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.714488  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:31.696192  8177 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:31.715478  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1087) took 1766 ms. Trace:
I20260709 13:02:31.715730  7914 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.670405  8153 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.696607  7447 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:31.727315  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:31.721133  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:31.728042  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:31.728271  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:31.709767  2318 connection.cc:256] RPC call timeout handler was delayed by -1.3067327649999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 13:02:31.720821  1632 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:31.723709  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1531) took 1509 ms. Trace:
I20260709 13:02:31.730150  2349 rpcz_store.cc:276] 0709 13:02:30.214315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.223230 (+  8915us) service_pool.cc:224] Handling call
0709 13:02:31.723518 (+1500288us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54004be1d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:31.723636 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1247701248}
W20260709 13:02:31.722090  2352 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1135) took 1984 ms (client timeout 1581 ms). Trace:
W20260709 13:02:31.734277  2352 rpcz_store.cc:269] 0709 13:02:29.737380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.737618 (+   238us) service_pool.cc:224] Handling call
0709 13:02:31.721890 (+1984272us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b540061fb54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:31.721992 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":592094592}
I20260709 13:02:31.728816  1705 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 11.
I20260709 13:02:31.735379  2351 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:02:31.709398  8038 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.717267  1704 rpcz_store.cc:276] 0709 13:02:29.949346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.154967 (+1205621us) service_pool.cc:224] Handling call
0709 13:02:31.206084 (+ 51117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.217021 (+ 10937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.217061 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:31.217838 (+   777us) log.cc:844] Serialized 26 byte log entry
0709 13:02:31.313973 (+ 96135us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b18002ba800. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:02:31.674548 (+360575us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:31.674611 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.674917 (+   306us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.715119 (+ 40202us) raft_consensus.cc:1685] finished
0709 13:02:31.715145 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.715400 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10338,"raft.queue_time_us":18388,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":16407,"spinlock_wait_cycles":9016192,"thread_start_us":360249,"threads_started":1,"wal-append.queue_time_us":435468}
I20260709 13:02:31.703318  8115 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.759397  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:31.717238  8177 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:31.716238  8025 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:31.763319  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:31.794596  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 245670us
W20260709 13:02:31.793268  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 230692us
I20260709 13:02:31.801059  8266 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.801822  8236 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.801738  8266 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.804214  8266 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.746773  8195 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:31.805814  8268 raft_consensus.cc:670] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } [suppressed 4 similar messages]
I20260709 13:02:31.805976  8195 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:31.806881  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:31.807549  2352 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:31.808303  8195 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:31.808485  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:31.809340  2345 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:31.809938  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:31.800071  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:31.721097  8110 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.722102  8153 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.731197  7650 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.715982  8262 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:31.723014  8201 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:02:31.861841  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.893834  1559 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:02:31.900189  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:31.798416  2344 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:31.839483  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:31.838596  2352 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:31.849855  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:31.841833  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:31.931324  1704 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:31.838972  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:31.861747  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 87303.6us
I20260709 13:02:31.901418  2348 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 11.
W20260709 13:02:31.934091  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.839102  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:31.732697  7447 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:31.885110  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 84988us
I20260709 13:02:31.885751  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:31.972332  2346 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:31.811901  2351 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:02:31.884585  2353 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:31.761698  8284 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.724810  8161 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:31.936050  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:31.842815  8182 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.552s	user 0.013s	sys 0.000s
I20260709 13:02:31.976862  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:31.978111  1552 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:31.854377  8135 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:31.815049  8177 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:31.986343  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 50402.6us
W20260709 13:02:31.986704  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.854516  7937 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.890597  8150 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.855377  8215 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.811115  7986 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:32.078765  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 116578us
I20260709 13:02:31.890020  8055 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:32.009299  7821 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.995661  8284 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.011390  8057 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:31.890159  8252 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:31.890949  8201 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:32.111402  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.026950  8099 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:31.997686  8153 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:02:31.726069  8222 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:32.034957  8194 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.001s
W20260709 13:02:32.035781  8279 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:02:32.017084  8110 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.984508  8270 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:32.040524  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.041877  8040 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.042335  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.129403  2348 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:32.130232  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.043720  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:32.000929  8195 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:02:32.125227  8110 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:32.134353  1602 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:54748 dropped due to backpressure: service queue is full with 50 items
I20260709 13:02:32.046911  8157 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:31.999907  8182 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.997949  8152 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:02:32.144894  8157 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:32.054169  8267 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.145331  8157 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:32.145817  8267 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.146314  8157 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.147096  8201 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:32.055927  1552 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:32.148257  1552 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:32.148379  8201 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:32.149027  1552 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:32.135821  8270 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:32.150173  1552 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:32.054405  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:32.057721  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.018231  7650 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:32.045316  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.155658  2349 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:32.156641  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.145183  8110 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:32.157717  8092 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.158259  8092 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:32.094622  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.095086  8150 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:32.078166  8227 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.057179  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.046908  8225 raft_consensus.cc:3055] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:31.994832  2346 raft_consensus.cc:2410] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:31.842008  8038 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:32.101112  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.105926  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.032825  8211 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:31.963359  8246 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:32.099071  8155 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.933266  8262 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:32.118763  1710 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:32.160565  8092 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.131515  1558 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:02:32.137131  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 93517.5us
I20260709 13:02:32.139328  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:32.141006  8254 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.000s
I20260709 13:02:32.141259  7821 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:32.126708  8194 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.134727  1602 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:54748 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3301)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3053)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3306)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3302)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3303)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3297)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3057)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3284)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3052)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3305)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3298)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3294)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3299)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3289)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3295)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3055)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3296)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3307)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1210)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3032)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3033)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3034)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3035)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3037)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3038)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3039)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3043)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3044)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3045)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3046)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3047)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3048)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3049)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3050)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3051)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3293)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1216)
I20260709 13:02:32.144156  8182 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:32.147720  8157 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:32.171379  8225 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:32.185124  8069 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.000s
I20260709 13:02:32.152920  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.206367  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.175767  8246 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:32.208057  2345 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:32.208068  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.208946  1560 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:32.162016  2353 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:32.127606  8222 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:32.170248  1552 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:32.211153  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:32.172389  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.173535  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.154296  8230 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:32.180426  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.077217  8025 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:32.221937  1701 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:32.200042  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:32.152405  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:32.201635  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 103793us
W20260709 13:02:32.153126  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.004571  8245 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:32.207517  1559 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:32.173632  8227 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.215286  2346 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:32.107175  7937 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.224519  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.183434  8038 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.226491  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:32.186735  8155 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.228417  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.210155  8246 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:32.225735  8245 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.196523  8254 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.214445  8230 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.231982  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:32.239197  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.238240  2318 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:32.239940  1602 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:54748 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3301)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3053)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3306)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3302)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3303)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3297)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3057)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3284)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3052)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3305)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3298)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3294)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3299)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3289)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3295)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3296)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3307)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1210)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3032)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3033)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3034)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3035)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3037)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3038)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3039)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3043)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3044)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3045)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3046)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3047)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3048)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3049)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3050)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3051)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3293)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1216)
I20260709 13:02:32.243909  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.245100  1559 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:32.245712  1552 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:32.245971  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.245177  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:32.243033  8245 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:32.249403  8284 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.250447  8281 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:32.250916  8281 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.251245  8281 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:32.229743  8069 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:32.253767  8178 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:32.254376  8178 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:32.254513  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1539) took 1250 ms. Trace:
I20260709 13:02:32.242499  8040 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:32.255316  8178 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:32.233078  8288 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:02:32.262353  1630 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:32.255021  2350 rpcz_store.cc:276] 0709 13:02:31.004578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.004731 (+   153us) service_pool.cc:224] Handling call
0709 13:02:31.183407 (+178676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.616652 (+433245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.254230 (+637578us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54007701d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:32.254442 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":994811136}
I20260709 13:02:32.263616  2347 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:32.265280  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:32.256268  8040 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.268777  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.233493  8150 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.274704  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.279381  1709 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:32.270330  1627 raft_consensus.cc:2376] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15. Re-sending same reply.
I20260709 13:02:32.281606  2349 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:32.284081  1675 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:32.287284  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.290499  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:32.290544  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:32.285943  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:32.292630  1559 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:32.256867  8230 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:32.288051  1627 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:32.313935  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.298434  7821 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:32.299259  8106 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:32.305560  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.283376  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:32.280094  8040 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:32.296160  8161 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:32.282389  8284 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 13:02:32.321722  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.322621  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.341950  1705 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:32.324716  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.306174  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:32.343556  1560 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:32.335328  1630 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:32.345049  1627 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:32.351667  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.338372  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:32.358098  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:32.358249  1676 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:32.361311  1559 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:32.361745  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.362402  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.327229  8106 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:32.364583  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:32.338954  8150 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:32.233731  8225 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.357366  1709 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:32.334592  8161 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.372872  1676 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:32.376777  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.381443  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.382429  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.364154  8211 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:32.338653  8252 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:02:32.347016  8152 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:02:32.363174  7894 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:32.627521  8114 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:32.645051  1559 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
W20260709 13:02:32.403487  8253 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.645s	user 0.000s	sys 0.016s
I20260709 13:02:32.638001  8110 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:32.630471  8213 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:32.414710  1636 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:32.633065  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1211) took 1181 ms. Trace:
I20260709 13:02:32.500486  8194 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:32.457155  8055 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:32.632004  8149 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.399866  8177 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.613224  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.429919  8195 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.415817  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.381537  1676 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:32.373966  8150 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.499975  8025 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:32.412925  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.524797  8227 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:32.527190  8038 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:32.380031  8191 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:32.527542  7937 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.392925  7796 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:32.375797  8094 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:32.627781  7821 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:32.619359  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2438) took 2386 ms. Trace:
I20260709 13:02:32.645427  7894 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:32.658742  7894 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:32.659777  8270 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.645370  8114 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.374109  8106 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:32.606511  8258 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.167s	user 0.001s	sys 0.002s
I20260709 13:02:32.409283  8040 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.571969  1602 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:54746 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3284)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3052)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3305)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3298)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3294)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3059)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3312)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3299)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3309)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3289)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3311)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3310)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3295)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3308)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3296)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3058)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3307)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1210)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3032)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3033)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3034)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3035)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3037)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3038)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3039)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3043)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3044)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3045)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3046)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3047)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3048)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3049)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3050)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3051)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3293)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1216)
I20260709 13:02:32.661435  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.662156  8114 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:32.662714  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.663028  8040 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:32.663622  1556 raft_consensus.cc:2410] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 21 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:02:32.611194  8257 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.395s	user 0.000s	sys 0.002s
I20260709 13:02:32.529495  8117 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
W20260709 13:02:32.500674  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.656240  8094 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.484064  8225 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:32.646486  8110 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:32.653069  8025 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:32.646199  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.657936  8252 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:32.610131  8051 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.527645  7855 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:32.664844  1556 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:02:32.647397  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3102) took 2816 ms (client timeout 1560 ms). Trace:
I20260709 13:02:32.590626  8182 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:32.707383  8025 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:02:32.710666  1701 rpcz_store.cc:269] 0709 13:02:29.830629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.830858 (+   229us) service_pool.cc:224] Handling call
0709 13:02:32.647228 (+2816370us) spinlock_profiling.cc:243] Waited 2.29 s on lock 0x7b540014c8d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:32.647325 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5277061376}
I20260709 13:02:32.649907  1628 rpcz_store.cc:276] 0709 13:02:31.451351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.614996 (+163645us) service_pool.cc:224] Handling call
0709 13:02:32.632784 (+1017788us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b54002db4d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:32.632982 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69963648}
I20260709 13:02:32.371778  8284 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:32.638144  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.650658  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1200) took 1216 ms. Trace:
W20260709 13:02:32.703835  8117 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:32.655633  1627 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:32.651278  2352 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:32.674963  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1209) took 1224 ms. Trace:
W20260709 13:02:32.652508  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 271046us
I20260709 13:02:32.720510  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:32.656119  1561 rpcz_store.cc:276] 0709 13:02:30.232719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.233620 (+   901us) service_pool.cc:224] Handling call
0709 13:02:30.325697 (+ 92077us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.243535 (+917838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.618940 (+1375405us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b540005a854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:32.619260 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2108596096}
I20260709 13:02:32.721755  1627 raft_consensus.cc:2410] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:32.722317  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:32.716332  1636 rpcz_store.cc:276] 0709 13:02:31.434053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.436156 (+  2103us) service_pool.cc:224] Handling call
0709 13:02:32.414262 (+978106us) spinlock_profiling.cc:243] Waited 6.49 ms on lock 0x7b5800135698. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:02:32.414501 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.650388 (+235887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.650600 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14921600}
I20260709 13:02:32.642697  7929 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:32.697624  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.720364  1632 rpcz_store.cc:276] 0709 13:02:31.450697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.620553 (+169856us) service_pool.cc:224] Handling call
0709 13:02:31.620885 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.729504 (+108619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.674430 (+944926us) spinlock_profiling.cc:243] Waited 99.7 ms on lock 0x7b54001b6a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:32.674712 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229392128}
I20260709 13:02:32.678215  2344 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:32.656072  7821 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:32.712393  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.730856  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.731200  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:32.731755  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.648044  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1206) took 1198 ms. Trace:
I20260709 13:02:32.655767  7796 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.714224  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1203) took 1271 ms. Trace:
I20260709 13:02:32.736119  1635 rpcz_store.cc:276] 0709 13:02:31.443021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.444038 (+  1017us) service_pool.cc:224] Handling call
0709 13:02:32.713336 (+1269298us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540010dbd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:32.714149 (+   813us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282807168}
I20260709 13:02:32.737708  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:32.738111  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.738337  1635 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:32.438468  8155 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:32.640782  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3103) took 2809 ms. Trace:
I20260709 13:02:32.739818  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.715955  8025 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:32.635481  8214 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:32.654592  8191 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:32.652045  1553 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:32.379654  8161 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:32.652420  8150 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:32.652379  8177 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:32.647428  8213 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:32.444792  8254 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:32.500674  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.576828  8041 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:32.418584  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.766717  1555 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:32.741335  8252 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:32.761217  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 388466us
I20260709 13:02:32.732568  1634 rpcz_store.cc:276] 0709 13:02:31.449390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.449559 (+   169us) service_pool.cc:224] Handling call
0709 13:02:31.449938 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.704832 (+254894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.647775 (+942943us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54001b7454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:32.647983 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548976000}
I20260709 13:02:32.768062  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:32.714545  1559 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:02:32.714525  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:32.742650  1635 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:32.770061  1628 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:32.708673  8094 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:32.771016  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.771694  1635 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:32.772554  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:32.739256  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3052) took 1030 ms. Trace:
I20260709 13:02:32.773070  1635 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:32.748803  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.752302  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.474752  8245 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:32.741391  1702 rpcz_store.cc:276] 0709 13:02:29.830919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.072480 (+241561us) service_pool.cc:224] Handling call
0709 13:02:30.112523 (+ 40043us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.243486 (+130963us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:31.183335 (+939849us) log.cc:844] Serialized 14 byte log entry
0709 13:02:31.434704 (+251369us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:31.434892 (+   188us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:31.434957 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.041859 (+606902us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b18000f8ce0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1
0709 13:02:32.043489 (+  1630us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b18000f8ce0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:32.639865 (+596376us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b18000f8ce0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419
0709 13:02:32.640423 (+   558us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400247054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:32.640487 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.640691 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1165726,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":1050,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":7003,"spinlock_wait_cycles":361407616,"thread_start_us":1189298,"threads_started":2,"wal-append.queue_time_us":474147,"wal-append.run_cpu_time_us":1212,"wal-append.run_wall_time_us":1001241}
I20260709 13:02:32.773164  1627 rpcz_store.cc:276] 0709 13:02:31.709324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.730912 (+1021588us) service_pool.cc:224] Handling call
0709 13:02:32.739186 (+  8274us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:32.717523  8182 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:02:32.744055  7447 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.736s	user 0.011s	sys 0.004s
I20260709 13:02:32.741220  7796 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:32.651582  8195 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:32.768162  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.650576  8055 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.748688  8191 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:32.786130  1628 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:32.810902  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:32.776456  7650 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.613s	user 0.004s	sys 0.009s
I20260709 13:02:32.795284  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:32.815472  2318 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:32.816579  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.770025  8252 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.817250  1632 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:32.790010  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.813927  1634 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:32.819056  1627 raft_consensus.cc:2410] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:32.819510  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:32.821061  1635 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:32.824091  1560 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:32.824460  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.825119  1634 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:32.826042  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.827627  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.784327  8041 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:32.828939  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.805447  8284 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:32.835289  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.832942  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.836102  2349 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:32.834595  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.837358  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.838579  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.839857  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.841853  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.843497  1635 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:32.843672  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.844321  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3295) took 1003 ms. Trace:
I20260709 13:02:32.844710  2345 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:32.844789  1635 rpcz_store.cc:276] 0709 13:02:31.841165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.839629 (+998464us) service_pool.cc:224] Handling call
0709 13:02:32.844258 (+  4629us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.846866  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.836367  1627 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
W20260709 13:02:32.871695  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 217018us
I20260709 13:02:32.850442  1635 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:32.852407  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:32.853199  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.854964  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:32.855242  1701 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:32.923282  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 68431.3us
I20260709 13:02:32.857010  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.924270  1628 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:32.924549  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:32.925079  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3291) took 1085 ms. Trace:
I20260709 13:02:32.892921  7894 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:32.849833  1632 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:32.894606  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.860862  8254 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.831041  8194 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:32.870648  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:32.946512  1559 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:32.823426  8252 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:32.805256  7447 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.865384  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:32.766752  8213 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.915815  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.964617  1708 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:32.865510  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.887451  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:32.965847  2349 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:32.833119  8041 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:32.841005  8284 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.916035  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.806149  8168 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:02:32.829748  7937 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:32.901018  8281 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.649s	user 0.015s	sys 0.000s
I20260709 13:02:32.922492  1554 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:32.922704  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.857029  1710 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:02:32.865597  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.976923  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.977567  1636 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:32.919734  8110 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:32.858518  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.980765  1709 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:32.980988  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.981787  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.925733  1628 rpcz_store.cc:276] 0709 13:02:31.840057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.828808 (+988751us) service_pool.cc:224] Handling call
0709 13:02:32.924929 (+ 96121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.929253  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:32.982789  1701 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:32.925312  8295 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:32.990510  8055 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:32.991816  1628 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:02:32.992120  8246 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.002s
I20260709 13:02:32.992689  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1210) took 1541 ms. Trace:
I20260709 13:02:32.994339  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.994992  1709 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:32.938831  8177 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:32.995836  7447 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.941457  8267 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.942629  8156 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.944936  8040 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:32.963608  8254 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.973253  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.998373  1708 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:32.999298  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:32.999578  8254 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.999884  1708 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:33.000408  8230 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.002s
I20260709 13:02:33.000850  8230 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.995878  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.925120  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:33.048072  1709 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:32.981938  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.932078  1707 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:33.049273  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:33.049789  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:33.051231  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.985000  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:32.975529  7937 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:33.055887  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 129287us
I20260709 13:02:33.056456  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:32.987848  8296 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:02:32.983588  8281 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:32.988340  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:32.969532  7821 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:32.997613  8040 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:32.947870  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3289) took 1150 ms. Trace:
I20260709 13:02:33.067255  1632 rpcz_store.cc:276] 0709 13:02:31.797263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.817959 (+1020696us) service_pool.cc:224] Handling call
0709 13:02:32.947783 (+129824us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.002086  8245 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:32.993428  1628 rpcz_store.cc:276] 0709 13:02:31.451007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.991038 (+1540031us) service_pool.cc:224] Handling call
0709 13:02:32.991403 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.992371 (+   968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.992627 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.005957  1701 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:33.069162  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:32.969674  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46232.2us
I20260709 13:02:33.007021  8213 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:33.008158  1708 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
I20260709 13:02:33.074605  1560 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:33.077164  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:33.009963  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3307) took 1165 ms. Trace:
I20260709 13:02:33.078464  1635 rpcz_store.cc:276] 0709 13:02:31.844596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.980708 (+1136112us) service_pool.cc:224] Handling call
0709 13:02:33.009859 (+ 29151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.978255  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1215) took 1126 ms. Trace:
I20260709 13:02:33.079452  1636 rpcz_store.cc:276] 0709 13:02:31.851592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.837204 (+985612us) service_pool.cc:224] Handling call
0709 13:02:32.978174 (+140970us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.012718  8106 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:33.015712  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:33.080732  1556 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:33.016350  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 1.19467e+06us
I20260709 13:02:33.070785  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:33.082829  1674 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:33.018517  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:32.992640  8246 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:33.090776  1635 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:33.091614  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3035) took 1638 ms. Trace:
W20260709 13:02:33.026517  1630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3054) took 1305 ms (client timeout 1570 ms). Trace:
W20260709 13:02:33.093667  1630 rpcz_store.cc:269] 0709 13:02:31.720599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.054065 (+333466us) service_pool.cc:224] Handling call
0709 13:02:33.026236 (+972171us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54001b6054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:33.026417 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253757184}
I20260709 13:02:33.048439  1627 raft_consensus.cc:2410] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:02:33.057440  1705 raft_consensus.cc:2368] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15.
I20260709 13:02:32.983973  8090 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:33.060271  2348 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
W20260709 13:02:33.061105  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 78457.7us
I20260709 13:02:32.974177  8284 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:33.062111  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:33.064191  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:33.096694  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3037) took 1635 ms. Trace:
I20260709 13:02:33.093178  1635 rpcz_store.cc:276] 0709 13:02:31.460710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.090210 (+1629500us) service_pool.cc:224] Handling call
0709 13:02:33.090566 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.091305 (+   739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.091550 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.098198  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:33.099429  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3038) took 1637 ms. Trace:
I20260709 13:02:33.099551  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:33.069914  1707 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:33.099833  1635 rpcz_store.cc:276] 0709 13:02:31.461687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.098513 (+1636826us) service_pool.cc:224] Handling call
0709 13:02:33.098891 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.099092 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.099116 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.099158 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:33.099197 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.099251 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.099373 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.100977  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:33.097221  1630 rpcz_store.cc:276] 0709 13:02:31.461380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.094549 (+1633169us) service_pool.cc:224] Handling call
0709 13:02:33.094975 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.095174 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.096356 (+  1182us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.096416 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:33.096467 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.096529 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.096639 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.973968  8194 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:33.102317  1635 raft_consensus.cc:1217] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Deduplicated request from leader. Original: 8.5->[10.6-16.9]   Dedup: 12.7->[14.8-16.9]
I20260709 13:02:33.102851  1635 pending_rounds.cc:85] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 7
I20260709 13:02:33.103088  1635 pending_rounds.cc:107] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:02:33.103333  1635 raft_consensus.cc:2889] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:33.103406  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:33.070397  1559 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:33.072140  8040 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:33.074605  8161 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:33.022899  1633 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:33.112201  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:33.112490  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3032) took 1660 ms. Trace:
I20260709 13:02:33.112875  1708 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:33.112927  1633 rpcz_store.cc:276] 0709 13:02:31.451972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.022363 (+1570391us) service_pool.cc:224] Handling call
0709 13:02:33.022695 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.112180 (+ 89485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.112410 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.114683  1633 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:33.115302  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3044) took 1449 ms. Trace:
I20260709 13:02:33.115819  1633 rpcz_store.cc:276] 0709 13:02:31.666384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.114155 (+1447771us) service_pool.cc:224] Handling call
0709 13:02:33.114508 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.115053 (+   545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.115238 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.092214  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:33.116849  1526 connection.cc:256] RPC call timeout handler was delayed by -1.2410810279999396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
I20260709 13:02:33.092878  1631 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 22: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:33.117563  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3304) took 1273 ms. Trace:
I20260709 13:02:33.117885  1631 rpcz_store.cc:276] 0709 13:02:31.843784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.018218 (+1174434us) service_pool.cc:224] Handling call
0709 13:02:33.117495 (+ 99277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.095912  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3296) took 1254 ms. Trace:
I20260709 13:02:33.104827  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:33.118690  1627 rpcz_store.cc:276] 0709 13:02:31.841463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.924863 (+1083400us) service_pool.cc:224] Handling call
0709 13:02:33.095844 (+170981us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.119903  1627 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:33.108978  1632 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:33.120793  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3033) took 1668 ms. Trace:
I20260709 13:02:33.121320  1632 rpcz_store.cc:276] 0709 13:02:31.452257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.090045 (+1637788us) service_pool.cc:224] Handling call
0709 13:02:33.108789 (+ 18744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.120443 (+ 11654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.120639 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.101644  1707 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:33.123270  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:33.123953  1710 raft_consensus.cc:2376] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16. Re-sending same reply.
I20260709 13:02:33.124224  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:33.123313  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3045) took 1456 ms. Trace:
I20260709 13:02:33.057987  7937 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:33.127712  2346 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:33.128504  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3047) took 1461 ms. Trace:
I20260709 13:02:33.119792  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:33.133898  1559 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:33.126598  7821 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:33.205912  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:33.206812  1559 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
W20260709 13:02:33.209141  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 117029us
W20260709 13:02:33.215826  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.217198  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 97485.3us
I20260709 13:02:33.217980  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:33.218832  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:33.220026  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:33.125460  1633 rpcz_store.cc:276] 0709 13:02:31.666675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.116639 (+1449964us) service_pool.cc:224] Handling call
0709 13:02:33.122818 (+  6179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.123009 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.123033 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.123075 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:33.123106 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.123155 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.123260 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.222074  1633 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:33.222791  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3049) took 1555 ms. Trace:
I20260709 13:02:33.223151  1633 rpcz_store.cc:276] 0709 13:02:31.667848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.221502 (+1553654us) service_pool.cc:224] Handling call
0709 13:02:33.221879 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.222462 (+   583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.222730 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.161243  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:33.152061  1556 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
W20260709 13:02:33.226436  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.227226  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3036) took 1766 ms. Trace:
I20260709 13:02:33.227844  1636 rpcz_store.cc:276] 0709 13:02:31.461131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.092547 (+1631416us) service_pool.cc:224] Handling call
0709 13:02:33.226519 (+133972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.226784 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.226823 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.226888 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:33.226936 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.227010 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.227145 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:33.229624  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 126338us
I20260709 13:02:33.230408  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:33.228641  1627 rpcz_store.cc:276] 0709 13:02:31.667283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.119400 (+1452117us) service_pool.cc:224] Handling call
0709 13:02:33.119714 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.128219 (+  8505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.128445 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.231532  1628 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:33.232196  1633 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:33.232690  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3034) took 1780 ms. Trace:
I20260709 13:02:33.232936  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3050) took 1540 ms. Trace:
I20260709 13:02:33.235314  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:33.236176  1705 raft_consensus.cc:2376] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17. Re-sending same reply.
I20260709 13:02:33.234637  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:33.237335  1631 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:33.233584  1633 rpcz_store.cc:276] 0709 13:02:31.692364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.223951 (+1531587us) service_pool.cc:224] Handling call
0709 13:02:33.224236 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.232672 (+  8436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.232878 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.237715  1556 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:33.138829  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:33.233239  1628 rpcz_store.cc:276] 0709 13:02:31.452548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.091183 (+1638635us) service_pool.cc:224] Handling call
0709 13:02:33.231264 (+140081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.232339 (+  1075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.232617 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:33.152503  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.241087  1557 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:33.241622  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3043) took 1575 ms. Trace:
I20260709 13:02:33.242132  1630 rpcz_store.cc:276] 0709 13:02:31.666013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.126886 (+1460873us) service_pool.cc:224] Handling call
0709 13:02:33.240897 (+114011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.241095 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.241120 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.241166 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:33.241212 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.241355 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.241558 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.235728  1708 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
W20260709 13:02:33.236924  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.247810  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.250355  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3046) took 1583 ms. Trace:
W20260709 13:02:33.251112  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.251533  1631 rpcz_store.cc:276] 0709 13:02:31.666973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.118521 (+1451548us) service_pool.cc:224] Handling call
0709 13:02:33.212610 (+ 94089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.247754 (+ 35144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.249841 (+  2087us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.239171  1701 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:33.230274  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:33.259318  1709 raft_consensus.cc:2368] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16.
I20260709 13:02:33.256866  1560 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:33.263573  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:33.075711  8156 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.189s
W20260709 13:02:33.266530  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.271190  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:33.112892  8307 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:33.127979  8155 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:33.282079  1556 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:33.211598  8142 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:33.226156  8227 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.106976  8041 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:33.243391  7821 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:33.216030  8299 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:33.276283  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:33.213668  8110 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:33.128337  7957 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.213886  8152 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:33.139505  8055 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.176s
I20260709 13:02:33.111665  8090 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:33.256279  8149 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:33.240482  8117 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:33.402997  1636 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:02:33.630347  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3051) took 1928 ms. Trace:
I20260709 13:02:33.630880  1636 rpcz_store.cc:276] 0709 13:02:31.702235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.275094 (+1572859us) service_pool.cc:224] Handling call
0709 13:02:33.402676 (+127582us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.629874 (+227198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.630250 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.240738  8230 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:33.317836  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:33.229423  7855 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.648696  1705 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:33.599463  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 172524us
I20260709 13:02:33.284829  8195 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:33.274906  8245 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:33.286409  8307 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.599032  8041 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:33.313736  8137 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:33.267097  7650 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.313s
I20260709 13:02:33.599377  7957 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:33.404448  8299 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:33.707180  2351 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.004s	sys 0.022s
I20260709 13:02:33.712831  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3285) took 1916 ms. Trace:
I20260709 13:02:33.603958  8152 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:33.685634  8307 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:33.700721  8298 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:33.694619  8137 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:33.702364  8106 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:33.745823  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:33.747040  1705 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
W20260709 13:02:33.716568  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.689718  1674 leader_election.cc:336] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.940s (SENT)
I20260709 13:02:33.758097  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:33.759229  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.760708  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 71044.4us
W20260709 13:02:33.763141  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.733582  1676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.075s	sys 0.137s
W20260709 13:02:33.763805  1676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.075s	sys 0.137s
W20260709 13:02:33.764200  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 544149us
I20260709 13:02:33.405938  8227 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:33.730109  1636 rpcz_store.cc:276] 0709 13:02:31.796108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.653506 (+1857398us) service_pool.cc:224] Handling call
0709 13:02:33.698065 (+ 44559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.698422 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.698465 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.698534 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:33.698587 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.712531 (+ 13944us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.712729 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.775672  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:33.779417  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.000s	sys 0.042s
W20260709 13:02:33.844333  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.000s	sys 0.042s
I20260709 13:02:33.694793  8117 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.734373  8245 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:02:33.784108  1635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.024s	sys 0.020s
W20260709 13:02:33.859120  1635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.025s	sys 0.020s
I20260709 13:02:33.734745  8299 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:33.734769  8152 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:33.792539  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1544) took 1905 ms. Trace:
I20260709 13:02:33.737996  8137 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:33.738368  8298 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.673810  7855 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:33.797430  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:33.785511  8225 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.078s	user 0.009s	sys 0.004s
I20260709 13:02:33.639679  8194 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:33.778532  8106 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:33.811492  1704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.089s	sys 0.106s
W20260709 13:02:33.944278  1704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.089s	sys 0.107s
I20260709 13:02:33.830292  2345 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:33.834158  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:33.838907  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.946489  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.840901  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:33.844692  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 574762us
I20260709 13:02:33.845682  1636 raft_consensus.cc:2376] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17. Re-sending same reply.
W20260709 13:02:33.849020  8311 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:02:33.853909  8161 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:33.847875  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.863013  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:33.962924  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 422359us
I20260709 13:02:33.963145  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:33.967558  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.864501  2346 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:33.867404  2344 rpcz_store.cc:276] 0709 13:02:31.886782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.964956 (+ 78174us) service_pool.cc:224] Handling call
0709 13:02:32.667429 (+702473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.727718 (+ 60289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.790994 (+1063276us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b540048ab50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:33.792449 (+  1455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1521532288}
I20260709 13:02:33.753654  8090 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:33.316123  8142 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:33.888620  1629 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:02:33.890031  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.000s	sys 0.021s
W20260709 13:02:33.859836  7796 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.045s	user 0.014s	sys 0.000s
I20260709 13:02:33.843194  8281 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:33.968218  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.953567  1552 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:33.970440  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:33.971649  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.000s	sys 0.021s
I20260709 13:02:33.985266  1556 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:33.971755  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:33.877763  8270 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:33.973635  1674 leader_election.cc:336] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.118s (SENT)
I20260709 13:02:33.988557  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:33.975128  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:33.970292  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:33.989905  2353 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:33.990330  2345 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:33.991601  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:33.991693  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:33.992966  2345 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 13:02:33.986261  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 715014us
I20260709 13:02:33.994252  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:33.986143  2346 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:33.988637  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:33.997526  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:33.879514  8298 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:34.002542  8304 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.364s	user 0.000s	sys 0.001s
I20260709 13:02:34.002980  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3287) took 2206 ms. Trace:
I20260709 13:02:34.003558  1636 rpcz_store.cc:276] 0709 13:02:31.796656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.969929 (+2173273us) service_pool.cc:224] Handling call
0709 13:02:34.002236 (+ 32307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.002515 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.002554 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.002617 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:34.002678 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.002752 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.002900 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.005340  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1167) took 1183 ms. Trace:
I20260709 13:02:34.005945  2351 rpcz_store.cc:276] 0709 13:02:32.821565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.821720 (+   155us) service_pool.cc:224] Handling call
0709 13:02:32.822110 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:32.822568 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.822613 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:32.856100 (+ 33487us) log.cc:844] Serialized 52 byte log entry
0709 13:02:33.068160 (+212060us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:33.708098 (+639938us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.708520 (+   422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.708630 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.005007 (+296377us) raft_consensus.cc:1685] finished
0709 13:02:34.005037 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.005255 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":816509,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":6350,"raft.queue_time_us":693,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":21620,"thread_start_us":734519,"threads_started":2,"wal-append.queue_time_us":378376}
I20260709 13:02:34.008915  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:33.996491  8090 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.012081  8161 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:34.013008  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:34.044463  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:33.988152  8246 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:34.024858  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3039) took 2560 ms. Trace:
I20260709 13:02:33.987756  8315 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:34.026692  8150 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.263s	user 0.013s	sys 0.000s
I20260709 13:02:33.996165  2349 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 13.
I20260709 13:02:34.036676  2351 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:34.038383  1636 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:33.895552  8245 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:34.041126  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:34.056056  2346 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:34.056483  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:34.056926  1602 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:34.057219  2346 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:34.058454  2344 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:02:34.043078  1676 leader_election.cc:336] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.752s (SENT)
W20260709 13:02:34.043331  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 69770.2us
I20260709 13:02:33.975095  8142 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:33.988636  8156 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.030s	sys 0.054s
I20260709 13:02:34.028977  7855 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.046277  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:34.026573  7929 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:33.866835  8117 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:34.066496  7957 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:33.988195  7796 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.202519  7796 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:34.072300  7661 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:02:34.204049  7661 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:34.204520  7661 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:34.139844  7929 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 22 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:34.206946  7929 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:34.143714  8310 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:02:34.141937  8307 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:34.073048  8156 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.030s	sys 0.055s
I20260709 13:02:34.210263  7661 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.211647  8307 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:34.166122  8245 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:34.211781  7661 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:34.058588  8150 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:34.224233  8150 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:34.175993  8252 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.201s	user 0.008s	sys 0.003s
I20260709 13:02:34.236621  8252 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.240000  8252 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:34.074635  8317 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.114s
I20260709 13:02:34.241554  8317 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:34.242062  8317 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:34.243206  8317 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.245009  8317 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:34.183925  8191 consensus_meta.cc:287] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.362s	user 0.008s	sys 0.005s
I20260709 13:02:34.246057  8191 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.247704  2353 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 12: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:34.262257  8191 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:34.272111  8094 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.466s	user 0.006s	sys 0.008s
W20260709 13:02:34.274801  8025 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.513s	user 0.006s	sys 0.009s
W20260709 13:02:34.301594  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.323372  8281 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:34.327523  8155 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.011s	user 0.008s	sys 0.004s
I20260709 13:02:34.261157  8117 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:34.344200  7937 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.079s	user 0.013s	sys 0.000s
I20260709 13:02:34.376057  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1545) took 2484 ms. Trace:
I20260709 13:02:34.382027  2352 rpcz_store.cc:276] 0709 13:02:31.891437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.931550 (+ 40113us) service_pool.cc:224] Handling call
0709 13:02:32.058603 (+127053us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.720016 (+661413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.375675 (+1655659us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b540005bc54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:34.375944 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1362487552}
I20260709 13:02:34.384114  8155 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:34.386291  8098 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:34.388020  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1166) took 1567 ms. Trace:
W20260709 13:02:34.387509  8284 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.176s	user 0.015s	sys 0.000s
I20260709 13:02:34.368141  8094 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.371048  8025 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.297299  8152 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:34.372184  8281 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.390554  8137 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.516546  8040 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.520483  8040 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:34.458606  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 481360us
I20260709 13:02:34.523665  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:34.461166  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:34.524773  8040 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.461688  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:34.527158  8117 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:02:34.466056  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 408928us
I20260709 13:02:34.530776  8094 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:34.468058  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 473715us
I20260709 13:02:34.539609  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:34.054934  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1213) took 2204 ms. Trace:
I20260709 13:02:34.541404  1636 rpcz_store.cc:276] 0709 13:02:31.850265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.004926 (+2154661us) service_pool.cc:224] Handling call
0709 13:02:34.005396 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.054609 (+ 49213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.054811 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:34.544562  7821 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.962s	user 0.016s	sys 0.009s
I20260709 13:02:34.545181  7821 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:34.048957  1635 rpcz_store.cc:276] 0709 13:02:31.464628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.100956 (+1636328us) service_pool.cc:224] Handling call
0709 13:02:33.102077 (+  1121us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:02:33.126276 (+ 24199us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:33.126305 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:33.126906 (+   601us) log.cc:844] Serialized 52 byte log entry
0709 13:02:33.859724 (+732818us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:33.859787 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.859910 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.024474 (+164564us) raft_consensus.cc:1685] finished
0709 13:02:34.024495 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.024768 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26625,"raft.queue_time_us":34523,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":20742,"thread_start_us":732387,"threads_started":1,"wal-append.queue_time_us":837817}
I20260709 13:02:34.469246  1632 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:34.545418  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1216) took 2545 ms. Trace:
W20260709 13:02:34.550851  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 1003) took 2847 ms (client timeout 1884 ms). Trace:
I20260709 13:02:34.479167  1703 raft_consensus.cc:2368] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15.
W20260709 13:02:34.479856  1704 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.009s	sys 0.027s
W20260709 13:02:34.551532  1554 rpcz_store.cc:269] 0709 13:02:31.703634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.715794 (+ 12160us) service_pool.cc:224] Handling call
0709 13:02:34.550327 (+2834533us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400313654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:34.550594 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2513625984}
W20260709 13:02:34.552130  1704 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.010s	sys 0.027s
W20260709 13:02:34.548731  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.553560  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2411) took 1665 ms. Trace:
I20260709 13:02:34.555094  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2447) took 2902 ms. Trace:
I20260709 13:02:34.480571  1705 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:34.555516  1558 rpcz_store.cc:276] 0709 13:02:31.693359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.693583 (+   224us) service_pool.cc:224] Handling call
0709 13:02:31.694064 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.188457 (+494393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.554540 (+2366083us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54001fd454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:34.554859 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1005148928}
I20260709 13:02:34.557518  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2435) took 1613 ms. Trace:
W20260709 13:02:34.482016  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 420541us
I20260709 13:02:34.559751  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:34.554062  1704 rpcz_store.cc:276] 0709 13:02:32.888095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.890100 (+  2005us) service_pool.cc:224] Handling call
0709 13:02:32.895692 (+  5592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.896037 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:33.254405 (+358368us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.027848 (+773443us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b180040d1a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:34.552733 (+524885us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.552844 (+   111us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:34.553035 (+   191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.553175 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.553380 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":508618,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":25241,"raft.queue_time_us":42630,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":145,"spinlock_wait_cycles":33106688,"thread_start_us":1636519,"threads_started":3,"wal-append.queue_time_us":756753}
I20260709 13:02:34.562166  1553 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: [13.12-14.13]
I20260709 13:02:34.563902  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:34.564590  1554 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 13:02:34.550845  1636 rpcz_store.cc:276] 0709 13:02:31.999936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.543443 (+2543507us) service_pool.cc:224] Handling call
0709 13:02:34.544233 (+   790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.544842 (+   609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.544877 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.544972 (+    95us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:34.545038 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.545122 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.545282 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:34.497093  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.578025  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:34.578720  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.579609  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.579718  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3115) took 1616 ms (client timeout 1699 ms). Trace:
I20260709 13:02:34.580255  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:34.580525  1709 rpcz_store.cc:269] 0709 13:02:32.963452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.048968 (+ 85516us) service_pool.cc:224] Handling call
0709 13:02:34.578979 (+1530011us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400223b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:34.579445 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480372096}
I20260709 13:02:34.581274  1553 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:34.512885  7929 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:34.588706  7929 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.589366  7929 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.388512  2348 rpcz_store.cc:276] 0709 13:02:32.820369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.820497 (+   128us) service_pool.cc:224] Handling call
0709 13:02:32.820811 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.095978 (+275167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.387735 (+1291757us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54004ed754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:34.387957 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549821312}
I20260709 13:02:34.514045  1702 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:34.594184  8117 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:34.596487  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2406) took 1742 ms. Trace:
I20260709 13:02:34.596967  1702 rpcz_store.cc:276] 0709 13:02:32.854030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.854206 (+   176us) service_pool.cc:224] Handling call
0709 13:02:32.865194 (+ 10988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.594293 (+1729099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.596118 (+  1825us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400336b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:34.596391 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3790383488}
I20260709 13:02:34.598174  8098 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:34.598663  8098 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:33.976891  1526 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:34.518980  8109 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.600518  8109 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:34.602016  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 625300us
I20260709 13:02:34.603952  8109 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:34.605233  8098 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.605968  7957 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:34.606818  8098 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:34.607187  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.620275  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3048) took 2952 ms. Trace:
W20260709 13:02:34.623206  1552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2462) took 2524 ms (client timeout 2216 ms). Trace:
W20260709 13:02:34.623879  1552 rpcz_store.cc:269] 0709 13:02:32.098111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.246388 (+148277us) service_pool.cc:224] Handling call
0709 13:02:34.622717 (+2376329us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400304dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:34.622940 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3858696448}
I20260709 13:02:34.627291  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:34.628273  1555 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:34.629467  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:34.634732  1555 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
W20260709 13:02:34.630828  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3184) took 2396 ms (client timeout 2901 ms). Trace:
W20260709 13:02:34.636374  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.636294  1557 rpcz_store.cc:269] 0709 13:02:32.233804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.637714 (+403910us) service_pool.cc:224] Handling call
0709 13:02:34.630330 (+1992616us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b5400143054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:34.630561 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1206900480}
I20260709 13:02:34.621057  1632 rpcz_store.cc:276] 0709 13:02:31.667564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.122060 (+1454496us) service_pool.cc:224] Handling call
0709 13:02:33.250031 (+127971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.546798 (+1296767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.619762 (+ 72964us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003b8dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:34.620149 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2801956608}
I20260709 13:02:34.625258  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:34.640719  1552 raft_consensus.cc:2376] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18. Re-sending same reply.
W20260709 13:02:34.642069  1703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2412) took 1751 ms (client timeout 1990 ms). Trace:
W20260709 13:02:34.642572  1703 rpcz_store.cc:269] 0709 13:02:32.890615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.899208 (+  8593us) service_pool.cc:224] Handling call
0709 13:02:34.641744 (+1742536us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54000cbcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:34.642001 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3466956672}
I20260709 13:02:34.644227  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:34.645139  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3126) took 1560 ms. Trace:
W20260709 13:02:34.645591  2317 connection.cc:256] RPC call timeout handler was delayed by -2.0727825500000563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 13:02:34.646436  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.646958  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.514490  8025 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:34.530723  1602 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:34.050832  1629 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:34.470463  1634 raft_consensus.cc:2368] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16.
I20260709 13:02:34.557976  1705 rpcz_store.cc:276] 0709 13:02:32.944010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.977127 (+1033117us) service_pool.cc:224] Handling call
0709 13:02:34.557089 (+579962us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b540016ce54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:34.557351 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999509760}
W20260709 13:02:34.658262  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 52848.9us
I20260709 13:02:34.564953  1704 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
W20260709 13:02:34.487300  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.548215  1601 leader_election.cc:400] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15.
I20260709 13:02:34.661091  1601 leader_election.cc:403] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:34.568449  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3108) took 4735 ms. Trace:
I20260709 13:02:34.661972  1706 rpcz_store.cc:276] 0709 13:02:29.832503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.955024 (+1122521us) service_pool.cc:224] Handling call
0709 13:02:31.525735 (+570711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.591657 (+ 65922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.567927 (+2976270us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54002a67d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:34.568222 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204210560}
W20260709 13:02:34.662588  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 114746us
I20260709 13:02:34.663734  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:34.568838  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:34.665055  1561 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:34.571198  7821 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:34.666316  7821 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.666802  7821 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.668332  8152 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.670818  8195 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:34.671248  8195 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:34.671578  2347 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1163) took 1853 ms (client timeout 1752 ms). Trace:
W20260709 13:02:34.672035  2347 rpcz_store.cc:269] 0709 13:02:32.818295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.818429 (+   134us) service_pool.cc:224] Handling call
0709 13:02:34.671155 (+1852726us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54006bd354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:34.671334 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4215527296}
I20260709 13:02:34.672427  8195 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:34.573454  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:34.575556  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:34.577041  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:34.550153  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:34.577698  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:34.584969  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.508737  7796 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.586426  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:34.587982  2346 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:34.384812  7937 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.598546  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:34.519287  8137 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:34.647686  2317 connection.cc:256] RPC call timeout handler was delayed by -1.4497650880000492s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1778s for a response
I20260709 13:02:34.416829  8281 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.401299  8284 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:34.694736  8325 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
W20260709 13:02:34.697119  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 166702us
I20260709 13:02:34.632441  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:34.698645  1709 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:34.699453  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2433) took 1756 ms. Trace:
I20260709 13:02:34.699745  1602 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:34.699810  1709 rpcz_store.cc:276] 0709 13:02:32.943381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.632057 (+1688676us) service_pool.cc:224] Handling call
0709 13:02:34.699373 (+ 67316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.653749  8214 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:34.700898  8214 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:34.417299  8298 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.645839  7929 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.658015  1553 raft_consensus.cc:2410] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:02:34.681548  1701 raft_consensus.cc:2368] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18.
W20260709 13:02:34.683619  2317 connection.cc:256] RPC call timeout handler was delayed by -1.2378928539999947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
I20260709 13:02:34.686293  8117 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:34.687147  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3112) took 1741 ms. Trace:
I20260709 13:02:34.710220  1702 rpcz_store.cc:276] 0709 13:02:32.946337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.598151 (+1651814us) service_pool.cc:224] Handling call
0709 13:02:34.687057 (+ 88906us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.687960  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2419) took 1773 ms. Trace:
I20260709 13:02:34.710007  8090 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.711416  1707 rpcz_store.cc:276] 0709 13:02:32.914311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.574947 (+1660636us) service_pool.cc:224] Handling call
0709 13:02:34.687877 (+112930us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.712329  8090 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.715092  1554 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:34.715440  7937 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.717514  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:34.701839  7796 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:34.718930  2347 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:34.719061  8284 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.688937  8245 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:02:34.720669  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:34.720932  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3293) took 2880 ms. Trace:
I20260709 13:02:34.721692  2353 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:34.721331  1629 rpcz_store.cc:276] 0709 13:02:31.840657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.000814 (+2160157us) service_pool.cc:224] Handling call
0709 13:02:34.042579 (+ 41765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.709088 (+666509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.720696 (+ 11608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.723505  8098 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:34.724486  8098 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:34.716534  8090 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.649091  1703 rpcz_store.cc:276] 0709 13:02:33.085098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.643731 (+1558633us) service_pool.cc:224] Handling call
0709 13:02:34.645063 (+  1332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.697367  8270 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:34.702703  8323 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:34.705956  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:34.706663  8055 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:34.710041  8117 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:34.725670  8211 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:34.726771  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3130) took 1606 ms. Trace:
I20260709 13:02:34.701453  8214 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:34.728008  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:34.737841  1705 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:34.738400  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3129) took 1651 ms. Trace:
I20260709 13:02:34.738811  1705 rpcz_store.cc:276] 0709 13:02:33.086458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.727614 (+1641156us) service_pool.cc:224] Handling call
0709 13:02:34.738293 (+ 10679us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.740041  1705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "d62ce354d09a4be59c29094425d09bc1"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:34.740928  1705 raft_consensus.cc:606] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Received request to transfer leadership to TS d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:34.703850  8298 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:34.728437  8270 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.732174  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:34.690244  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:34.744112  1561 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:34.704733  1708 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:34.731889  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:34.746389  2348 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:34.747135  7796 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:34.747781  8152 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:34.749073  1674 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:34.731187  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:34.741370  1705 raft_consensus.cc:614] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:34.734277  1701 rpcz_store.cc:276] 0709 13:02:33.120590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.577168 (+1456578us) service_pool.cc:224] Handling call
0709 13:02:34.726686 (+149518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.733888  8211 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:34.733767  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:34.734079  7957 raft_consensus.cc:3055] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:34.684517  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2422) took 1750 ms. Trace:
I20260709 13:02:34.766547  1704 rpcz_store.cc:276] 0709 13:02:32.930008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.563307 (+1633299us) service_pool.cc:224] Handling call
0709 13:02:34.680350 (+117043us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:34.754050  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 204120us
I20260709 13:02:34.736037  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:34.732458  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:34.676476  8227 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.702317  1602 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:34.768360  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43592 (request call id 46) took 1698 ms. Trace:
I20260709 13:02:34.771977  1705 rpcz_store.cc:276] 0709 13:02:33.066384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.739726 (+1673342us) service_pool.cc:224] Handling call
0709 13:02:34.765076 (+ 25350us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.770309  8098 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:02:34.733119  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:34.765687  8055 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.778282  2352 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:34.778651  8055 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:34.785665  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2424) took 1846 ms. Trace:
I20260709 13:02:34.786083  7796 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:34.786244  1701 rpcz_store.cc:276] 0709 13:02:32.939004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.784396 (+1845392us) service_pool.cc:224] Handling call
0709 13:02:34.784906 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.785198 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.785221 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.785280 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:34.785345 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.785409 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.785578 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:34.789223  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.768114  7957 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:34.786701  7796 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.790694  7796 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.790949  7957 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.792644  7957 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:34.795269  1628 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:02:34.788820  1701 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:34.741791  8195 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:34.732779  8117 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.792665  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:34.801260  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 67749.4us
I20260709 13:02:34.803201  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:34.804352  1636 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:34.805898  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:34.806205  8262 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:34.809962  8142 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.725s	user 0.020s	sys 0.005s
I20260709 13:02:34.810436  8149 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.810988  8149 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.811231  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:34.812692  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:34.815057  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:34.815692  8299 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.922s	user 0.011s	sys 0.006s
I20260709 13:02:34.815882  1631 raft_consensus.cc:2376] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20. Re-sending same reply.
I20260709 13:02:34.816376  8299 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.810587  8142 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:34.818905  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:34.819301  1706 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:02:34.820031  2317 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:34.820492  1634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3060) took 2104 ms (client timeout 1699 ms). Trace:
W20260709 13:02:34.851904  1634 rpcz_store.cc:269] 0709 13:02:32.716392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.825827 (+109435us) service_pool.cc:224] Handling call
0709 13:02:34.820168 (+1994341us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540010ccd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:34.820382 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3778408832}
I20260709 13:02:34.853358  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:34.854379  1634 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:34.855315  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:34.821218  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:34.803516  1632 raft_consensus.cc:2368] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17.
I20260709 13:02:34.825567  1707 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:34.826056  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:34.826264  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.857151  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 67743.4us
I20260709 13:02:34.827605  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:34.828347  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:34.830832  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:34.832155  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3128) took 1746 ms. Trace:
I20260709 13:02:34.834889  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:34.795550  8214 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:34.839579  7929 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:34.844116  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:34.846202  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:34.848865  1561 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:34.802723  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2427) took 1856 ms. Trace:
I20260709 13:02:34.864316  1701 rpcz_store.cc:276] 0709 13:02:32.940058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.787633 (+1847575us) service_pool.cc:224] Handling call
0709 13:02:34.788315 (+   682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.796527 (+  8212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.796729 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.799350  2348 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:34.859102  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:34.858695  1708 rpcz_store.cc:276] 0709 13:02:33.086184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.572845 (+1486661us) service_pool.cc:224] Handling call
0709 13:02:34.831831 (+258986us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b54002e77d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:34.831992 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62596480}
I20260709 13:02:34.867327  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2430) took 1919 ms. Trace:
I20260709 13:02:34.867209  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:34.867949  1707 rpcz_store.cc:276] 0709 13:02:32.940944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.824544 (+1883600us) service_pool.cc:224] Handling call
0709 13:02:34.825008 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.857406 (+ 32398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.859855 (+  2449us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.868674  2344 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.866786  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2429) took 1919 ms. Trace:
I20260709 13:02:34.870950  1631 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:34.869913  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:34.871080  1706 rpcz_store.cc:276] 0709 13:02:32.940670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.818387 (+1877717us) service_pool.cc:224] Handling call
0709 13:02:34.818832 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.851552 (+ 32720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.859764 (+  8212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.876674  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:34.877494  1632 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:34.878222  8214 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.878502  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:34.879202  1628 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:34.879323  1632 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:34.876675  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:34.880251  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:34.880438  1631 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:34.881045  1632 raft_consensus.cc:2368] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17.
I20260709 13:02:34.881827  1703 raft_consensus.cc:2368] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16.
I20260709 13:02:34.883360  1703 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:34.883414  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 181214us
I20260709 13:02:34.884121  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3111) took 1938 ms. Trace:
I20260709 13:02:34.884505  1602 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:34.884483  1703 rpcz_store.cc:276] 0709 13:02:32.945895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.882758 (+1936863us) service_pool.cc:224] Handling call
0709 13:02:34.883146 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.883850 (+   704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.884032 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.885807  1709 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:34.886269  1710 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:02:34.887075  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:34.887566  8297 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.969s	user 0.000s	sys 0.024s
I20260709 13:02:34.888118  8320 raft_consensus.cc:2764] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:02:34.888306  8323 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:34.888753  8323 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:02:34.888890  8317 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.560s	user 0.020s	sys 0.000s
I20260709 13:02:34.889218  8323 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:34.889308  7929 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:34.889015  8320 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:02:34.893272  8320 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:34.893772  8320 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:34.894502  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 230964us
I20260709 13:02:34.895054  1704 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:34.895488  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:34.895171  8320 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.893821  8055 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.897637  1627 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:34.898476  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:34.887603  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3114) took 1924 ms. Trace:
I20260709 13:02:34.909543  1553 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:34.909839  1635 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:34.910709  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:34.907955  1709 rpcz_store.cc:276] 0709 13:02:32.963200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.886433 (+1923233us) service_pool.cc:224] Handling call
0709 13:02:34.886900 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.887252 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.887278 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.887337 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:34.887389 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.887447 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.887544 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.895813  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2426) took 1956 ms. Trace:
I20260709 13:02:34.915311  1704 rpcz_store.cc:276] 0709 13:02:32.939765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.767626 (+1827861us) service_pool.cc:224] Handling call
0709 13:02:34.857859 (+ 90233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.895511 (+ 37652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.895723 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.919109  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:34.921444  2349 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
W20260709 13:02:34.921344  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 100279us
I20260709 13:02:34.927742  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:34.928445  1633 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:34.931198  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:34.930377  1704 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:34.932893  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3120) took 1953 ms. Trace:
W20260709 13:02:34.934095  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.934554  1709 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:34.935238  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3119) took 1965 ms. Trace:
I20260709 13:02:34.933252  1704 rpcz_store.cc:276] 0709 13:02:32.979228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.916058 (+1936830us) service_pool.cc:224] Handling call
0709 13:02:34.929923 (+ 13865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.932613 (+  2690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.932836 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.937816  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3121) took 1956 ms. Trace:
I20260709 13:02:34.938140  1704 rpcz_store.cc:276] 0709 13:02:32.981289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.936752 (+1955463us) service_pool.cc:224] Handling call
0709 13:02:34.937138 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.937492 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.937524 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.937575 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:34.937602 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.937654 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.937764 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.940232  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:34.941141  1554 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:02:34.941690  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:34.943456  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3113) took 1980 ms. Trace:
I20260709 13:02:34.943892  1703 rpcz_store.cc:276] 0709 13:02:32.962910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.885453 (+1922543us) service_pool.cc:224] Handling call
0709 13:02:34.942656 (+ 57203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.943055 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.943105 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.943176 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:34.943222 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.943285 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.943395 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.935616  1709 rpcz_store.cc:276] 0709 13:02:32.970458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.921306 (+1950848us) service_pool.cc:224] Handling call
0709 13:02:34.934104 (+ 12798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.934973 (+   869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.935175 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.946933  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:34.947474  1709 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:34.947634  1555 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:34.948127  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2443) took 1881 ms. Trace:
I20260709 13:02:34.948462  1709 rpcz_store.cc:276] 0709 13:02:33.066977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.946607 (+1879630us) service_pool.cc:224] Handling call
0709 13:02:34.946960 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.947854 (+   894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.948066 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.944692  1704 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:34.950464  1552 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:34.950590  1553 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:02:34.950922  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3123) took 1891 ms. Trace:
I20260709 13:02:34.951710  8117 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:34.952898  1709 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:02:34.954183  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:34.943847  8323 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.955145  8248 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:34.956964  2344 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:34.954463  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:34.957191  8323 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:34.958897  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:34.960719  1709 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:34.961074  8142 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:34.964365  2349 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:34.965181  1706 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:34.965559  1456 catalog_manager.cc:5697] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 12 to 15, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 15 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:34.968653  1675 connection.cc:256] RPC call timeout handler was delayed by -1.0594703640000489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 13:02:34.970569  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 150673us
I20260709 13:02:34.951304  1704 rpcz_store.cc:276] 0709 13:02:33.059126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.939074 (+1879948us) service_pool.cc:224] Handling call
0709 13:02:34.944369 (+  5295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.950654 (+  6285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.950866 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.973038  8227 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:34.973332  7796 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:34.973309  1633 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:34.925941  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:34.983419  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2431) took 2042 ms. Trace:
I20260709 13:02:34.999532  1706 rpcz_store.cc:276] 0709 13:02:32.941251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.872127 (+1930876us) service_pool.cc:224] Handling call
0709 13:02:34.964832 (+ 92705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.982887 (+ 18055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.983328 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:35.000938  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.926944  1631 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:35.018019  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:35.018507  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 64452.1us
I20260709 13:02:35.021831  1705 raft_consensus.cc:2368] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18.
I20260709 13:02:35.024446  8284 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:35.025727  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.026086  1555 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
W20260709 13:02:35.026995  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51959.4us
I20260709 13:02:35.032177  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:35.033136  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.035055  1702 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 17. Ops: []
I20260709 13:02:35.036602  7796 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:35.037832  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:35.038625  2349 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:35.041873  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:35.043286  8248 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:35.043521  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:35.043777  8248 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:35.044554  8248 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.032481  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.045609  7937 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:35.029910  1707 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:34.896238  8155 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.125s
I20260709 13:02:34.893236  7821 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:34.996345  8227 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:35.046029  1708 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:35.060837  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.062094  1554 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:35.063081  7937 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:35.063493  8227 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:35.064558  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.064402  7937 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.066570  8181 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:35.068048  8149 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:35.068642  8181 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:35.069397  1636 raft_consensus.cc:2368] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17.
I20260709 13:02:35.069803  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.070817  1627 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:35.070739  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:35.071702  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:35.071792  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:35.072470  1628 raft_consensus.cc:2376] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17. Re-sending same reply.
I20260709 13:02:35.073315  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:35.073458  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.074237  1628 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:35.069914  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:35.075125  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.028643  8055 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:35.064497  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:35.080340  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:35.087841  8332 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:35.088513  7796 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:35.089251  7796 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:35.089790  7796 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:35.092880  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2425) took 2153 ms (client timeout 2096 ms). Trace:
I20260709 13:02:35.089296  1636 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 22 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 21. Current term is 22. Ops: []
W20260709 13:02:35.094354  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 65357.3us
W20260709 13:02:35.095137  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.101716  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:35.103076  8298 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:35.082194  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:35.103439  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:35.103811  8298 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:35.105760  8284 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:35.105947  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.106348  8284 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:35.106767  1708 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:35.107242  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:35.107652  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.107378  8284 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.108182  1559 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:35.108811  1708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:35.070880  8323 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.111171  8323 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.112900  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:35.114382  8333 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:35.105144  8298 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:35.116742  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 1021) took 1142 ms. Trace:
W20260709 13:02:35.117008  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.118139  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3117) took 2154 ms. Trace:
I20260709 13:02:35.117146  1560 rpcz_store.cc:276] 0709 13:02:33.973981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.974107 (+   126us) service_pool.cc:224] Handling call
0709 13:02:34.458348 (+484241us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:34.459779 (+  1431us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:34.679646 (+219867us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.858091 (+178445us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:34.865812 (+  7721us) log.cc:844] Serialized 52 byte log entry
0709 13:02:34.866017 (+   205us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:34.866065 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.866174 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:35.116357 (+250183us) raft_consensus.cc:1685] finished
0709 13:02:35.116381 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.116531 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7686,"prepare.queue_time_us":409484,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":54,"raft.queue_time_us":65824,"raft.run_cpu_time_us":582,"raft.run_wall_time_us":8774,"thread_start_us":341762,"threads_started":2,"wal-append.queue_time_us":337316}
I20260709 13:02:35.118542  1702 rpcz_store.cc:276] 0709 13:02:32.963899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.894073 (+1930174us) service_pool.cc:224] Handling call
0709 13:02:34.894521 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.035536 (+141015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.117864 (+ 82328us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54002a76d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:35.118049 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":318496640}
I20260709 13:02:35.119554  1627 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:35.119921  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:35.120802  2344 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:35.085353  8090 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:35.114279  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:35.110760  1674 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:35.111521  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:35.125376  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 48274.7us
I20260709 13:02:35.125567  2349 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
W20260709 13:02:35.093397  1710 rpcz_store.cc:269] 0709 13:02:32.939433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.124700 (+185267us) service_pool.cc:224] Handling call
0709 13:02:35.092609 (+1967909us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400250dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:35.092792 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3376496000}
I20260709 13:02:35.127169  1703 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:35.109403  1708 raft_consensus.cc:606] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:35.129664  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:35.129714  1708 raft_consensus.cc:614] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:35.126859  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:35.126776  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 185631us
I20260709 13:02:35.131932  1707 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:35.138427  1628 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:02:35.141963  1633 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.138904  1628 pending_rounds.cc:85] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 11
I20260709 13:02:35.120843  8262 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:02:35.144630  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48831.1us
I20260709 13:02:35.145852  1631 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.150118  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.152197  1555 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:35.123080  8323 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:35.145067  1559 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:35.158156  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.143915  1628 pending_rounds.cc:107] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:02:35.168680  1628 raft_consensus.cc:2889] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:35.126682  8055 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:35.170984  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3124) took 2108 ms. Trace:
I20260709 13:02:35.171654  1703 rpcz_store.cc:276] 0709 13:02:33.062934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.945234 (+1882300us) service_pool.cc:224] Handling call
0709 13:02:34.945592 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.132219 (+186627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.170693 (+ 38474us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54002a71d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:35.170894 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416887680}
I20260709 13:02:35.128563  8211 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:35.149451  8152 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:35.178675  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 219766us
I20260709 13:02:35.182268  1602 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:35.188958  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.189258  2344 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:35.194150  8336 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:35.148241  8142 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:02:35.180171  8117 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:35.201881  8117 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:35.202374  8300 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:35.202948  8300 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:35.203097  8117 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:35.204785  8117 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:35.205506  1707 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 21. Current term is 22. Ops: []
W20260709 13:02:35.205858  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.206509  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.207060  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:35.208168  1702 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 12: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:35.208159  8340 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:35.154250  8090 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:35.148806  8262 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:35.197588  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:35.211557  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:35.189525  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.217017  8340 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:35.218060   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:35.220121  8340 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:35.221818  8312 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:35.222893  8312 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:35.223245  8312 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:35.223601  8270 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:35.224500  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:35.224211  8312 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.193245  8106 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:35.226040  8322 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:35.179445  8320 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.226718  8322 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:35.226369  8055 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:35.227764  8322 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.232015  8312 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.235471  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:35.236617  1704 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:35.204099  8300 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:35.246816  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.202677  7929 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:35.285778  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:35.291364  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 395794us
W20260709 13:02:35.300557  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 90706us
I20260709 13:02:35.301600  1703 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:35.309860  1709 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:35.311247  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 200482us
I20260709 13:02:35.311704  7796 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:35.312777  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:35.304009  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 113227us
I20260709 13:02:35.306716  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:35.360116  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47370.9us
W20260709 13:02:35.363827  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64270.3us
I20260709 13:02:35.314693  8090 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.308969  1633 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:35.322926  8281 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:35.299024  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:35.381156  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:35.388674  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 151099us
W20260709 13:02:35.376185  8245 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.656s	user 0.018s	sys 0.003s
I20260709 13:02:35.352324  7929 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:35.380254  8106 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:35.400812  1674 leader_election.cc:400] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17.
I20260709 13:02:35.402271  1526 leader_election.cc:400] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18.
I20260709 13:02:35.402660  1526 leader_election.cc:403] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:02:35.402169  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:35.393585  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:35.405107  7796 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:35.380167  8281 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:35.409536  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.400569  8106 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.410981  8106 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:35.412138  8195 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.585s	user 0.012s	sys 0.004s
I20260709 13:02:35.413111  1634 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:02:35.392714  8025 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:35.413686  8342 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:35.414507  2317 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:35.420037  1703 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:35.420535  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:35.421597  1710 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:35.422569  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:35.422426  7957 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:35.422931  7929 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.422879  8211 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:35.424765  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.421371  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:35.434330  1705 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:35.398926  8245 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15.
I20260709 13:02:35.424093  8248 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:35.446628  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 55406.9us
I20260709 13:02:35.449640  1702 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:35.453373  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:35.456302  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:35.456705  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.457791  7821 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:35.458117  8216 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:35.407565  8281 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.470817  1703 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:35.406087  8270 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:35.467502  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.459177  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.458418  1631 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:35.471973  1559 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:35.463318  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 67769.6us
I20260709 13:02:35.472611  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.474586  1701 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:35.474987  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:35.476766  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:35.485080  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:35.463750  7796 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:35.459398  8300 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.459143  7957 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:35.470585  8322 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:35.489460  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:35.422590  8342 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
W20260709 13:02:35.493947  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.500108  1706 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
W20260709 13:02:35.519480  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 105010us
I20260709 13:02:35.479858  8149 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:35.484273  8216 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:35.526113  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:35.526521  8216 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.528652  8343 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18.
I20260709 13:02:35.532167  1704 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:35.540829  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:35.543820  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 43813.6us
I20260709 13:02:35.529951  8245 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:35.555188  1526 leader_election.cc:400] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16.
I20260709 13:02:35.555768  1526 leader_election.cc:403] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:35.549414  8337 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:35.560917  8337 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:35.561606  8337 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:35.563832  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:35.566382  8344 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
W20260709 13:02:35.567625  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 103855us
I20260709 13:02:35.577215  1525 leader_election.cc:400] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16.
I20260709 13:02:35.577808  8194 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:02:35.536731  8025 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:35.582589  8025 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:35.587433  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 125396us
W20260709 13:02:35.573159  8227 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.001s
I20260709 13:02:35.470903  8262 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:35.540234  8322 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:35.536478  7957 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:35.596071  2317 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:35.597177  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:35.576977  8117 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:02:35.598296  7796 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:35.612447  7957 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.613991  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:35.616076  1629 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:35.558127  8245 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:02:35.620232  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 141575us
W20260709 13:02:35.631203  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.621601  1708 raft_consensus.cc:2410] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:02:35.646124  1559 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
W20260709 13:02:35.648177  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.603826  8342 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 15. Current term is 16.
I20260709 13:02:35.651325  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:35.648063  1675 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:35.620627  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.662803  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48880.4us
I20260709 13:02:35.664836  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:35.669732  2344 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:02:35.598562  8227 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:35.647119  8245 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:35.679838  2348 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:35.660262  8149 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:35.681618  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:35.683133  8284 raft_consensus.cc:2764] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:02:35.694512  8090 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:35.683916  7796 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:35.699138  8248 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:35.704180  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 114757us
I20260709 13:02:35.674813  8216 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:35.660290  8355 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:02:35.682958  8357 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:02:35.726276  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 81919.7us
W20260709 13:02:35.728880  8353 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.084s	user 0.001s	sys 0.001s
I20260709 13:02:35.730453  8356 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:35.714882  8216 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:35.733312  8216 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:35.750619  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 74462.8us
I20260709 13:02:35.719887  8262 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:35.763702  2348 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:35.781636   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:35.782879  1456 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 13, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 13 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:35.784715  8216 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:35.788837  1602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.010s	sys 0.023s
W20260709 13:02:35.789906  1602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.011s	sys 0.023s
W20260709 13:02:35.790426  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 607892us
I20260709 13:02:35.796914  1456 catalog_manager.cc:5697] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 7 to 15. New cstate: current_term: 15 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } }
W20260709 13:02:35.806057  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.807430  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:35.808724  8142 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:35.810355  1629 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:35.810567  8055 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:35.811237  8055 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:35.810379  8142 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:35.813301  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.814363  1629 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:35.814962  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:35.807663  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.828298  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:35.818014  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:35.818666  8338 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.630s	user 0.005s	sys 0.001s
I20260709 13:02:35.760766  8281 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.725057  8322 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.819597  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:35.802645  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.771128  8357 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.819828  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.849522  1631 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:35.765223  8343 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:02:35.761852  8356 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.821619  8106 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:35.822409  8320 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.595s	user 0.014s	sys 0.000s
I20260709 13:02:35.822237  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.825098  8098 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:35.825848  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:35.826669  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:35.826885  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 155909us
I20260709 13:02:35.798668  8227 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:35.827328  1526 leader_election.cc:400] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18.
I20260709 13:02:35.863915  1526 leader_election.cc:403] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:02:35.828338  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 88267.4us
W20260709 13:02:35.818140  8359 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:02:35.832224  2349 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:35.830444  1602 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:35.853227  1635 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:02:35.854344  1627 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:35.812518  8055 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:35.864104  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:35.848738  8355 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:35.873036  2346 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:35.873644  1633 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:35.878736  1632 raft_consensus.cc:2410] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:02:35.881743  1629 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:35.849901  8299 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:35.887745  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 59519us
W20260709 13:02:35.883966  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 58557.8us
I20260709 13:02:35.891773  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:35.849365  8117 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:02:35.874136  8149 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:35.879052  8142 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:35.899909  1674 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:35.903138  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.890411  7796 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:35.901307  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:35.874231  8262 consensus_queue.cc:1059] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:02:35.890008  8227 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18.
I20260709 13:02:35.913228  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:35.902626  8149 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.878237  8356 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:35.932953  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 336815us
I20260709 13:02:35.968376  8195 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:35.971065  8357 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:35.972000  8347 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:35.973974  8347 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:35.923804  8098 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:35.976858  8098 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.977360  8098 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:35.928012  8298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.014s	sys 0.003s
W20260709 13:02:35.987726  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 50987.6us
I20260709 13:02:35.927753  8142 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:35.929760  8299 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:35.936743  8149 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:35.998425  2347 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEARNER]: Advancing to term 17
W20260709 13:02:35.987411  8298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.015s	sys 0.004s
I20260709 13:02:35.938020  8055 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.939013  7796 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:35.956527  8312 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:36.043465  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 152370us
I20260709 13:02:36.044486  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:36.035171  8227 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.996291  8211 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:36.049502  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:36.050849  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:36.053644  1708 raft_consensus.cc:2368] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18.
I20260709 13:02:36.011284  8299 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.055716  8227 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.056522  1554 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:36.059125  8365 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
W20260709 13:02:36.060714  8331 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.900s	user 0.003s	sys 0.000s
I20260709 13:02:36.039374  8320 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.065827  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:36.071029  2351 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:02:36.088963  1554 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:36.088780  8142 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.090235  1553 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:36.088641  8298 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:36.050352  8211 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:36.063388  8299 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:36.091589  1630 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:36.097230  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:36.097157  1633 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:36.099238  1627 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:36.017908  7929 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:02:36.101989  1635 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:02:36.110260  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:36.045897  8336 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 13:02:36.094273  8337 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.002s
I20260709 13:02:36.115439  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:36.116094  1558 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.111967  1559 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.098024  2347 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEARNER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:02:36.118566  2347 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 9
I20260709 13:02:36.097656  8355 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:36.125607  2347 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:02:36.063886  8195 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:02:36.061419  7796 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.096794  8211 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:36.102135  8281 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:36.137786  2347 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:36.143162  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 109275us
W20260709 13:02:36.144698  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 73039.3us
I20260709 13:02:36.095919  8320 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.146659  2352 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:36.112618  8312 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:02:36.169737  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 125268us
I20260709 13:02:36.134711  8106 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:36.193840  1674 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:36.135289  8090 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:02:36.197844  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2466) took 1642 ms. Trace:
W20260709 13:02:36.179752  8333 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:02:36.138620  8355 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:36.203538  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:36.204136  8284 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:36.134505  8248 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:02:36.206394  8377 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:36.206971  8262 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:36.210209  8227 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:36.210882  1710 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:36.212052  8376 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.135622  8356 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:36.210420  1526 leader_election.cc:400] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18.
I20260709 13:02:36.216365  8336 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.126922  8337 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.126053  7937 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
W20260709 13:02:36.192732  8300 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.674s	user 0.015s	sys 0.001s
W20260709 13:02:36.214857  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 104643us
I20260709 13:02:36.230613  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:36.232295  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1222) took 1165 ms. Trace:
I20260709 13:02:36.227227  8284 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:36.235061  8378 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:02:36.236227  8383 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:36.237067  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45935us
W20260709 13:02:36.241175  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.242081  8355 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:36.243304  8372 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:36.185351  8116 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.252s
I20260709 13:02:36.198351  1556 rpcz_store.cc:276] 0709 13:02:34.555475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.555623 (+   148us) service_pool.cc:224] Handling call
0709 13:02:34.556190 (+   567us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.556890 (+   700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.556930 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:34.557899 (+   969us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.769562 (+211663us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:34.769787 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.769894 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:35.159591 (+389697us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b180010e040. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:36.059962 (+900371us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:36.197608 (+137646us) raft_consensus.cc:1685] finished
0709 13:02:36.197634 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.197775 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3527,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":217,"spinlock_wait_cycles":177226880,"thread_start_us":207136,"threads_started":1,"wal-append.queue_time_us":209112}
I20260709 13:02:36.246356  8347 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:36.191293  7929 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:02:36.199879  8322 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:36.266327  1557 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:36.250993  2352 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:02:36.258204  1561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3204) took 1318 ms (client timeout 1536 ms). Trace:
I20260709 13:02:36.269037  1456 catalog_manager.cc:5697] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:36.292469  1627 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:36.296872  1632 rpcz_store.cc:276] 0709 13:02:35.067253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.070480 (+  3227us) service_pool.cc:224] Handling call
0709 13:02:36.225264 (+1154784us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b540017be54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:36.232198 (+  6934us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854763264}
I20260709 13:02:36.306409  1559 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
W20260709 13:02:36.306373  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 84407.4us
W20260709 13:02:36.345679  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 231854us
W20260709 13:02:36.257629  8245 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.566s	user 0.013s	sys 0.000s
I20260709 13:02:36.295320  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:36.350958  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 55700.3us
W20260709 13:02:36.318667  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 124839us
W20260709 13:02:36.323441  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.316620  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 265136us
I20260709 13:02:36.353854  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:02:36.265233  1527 leader_election.cc:336] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.537s (SENT)
I20260709 13:02:36.354694  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:36.252543  8343 raft_consensus.cc:3055] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:36.358214  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:36.358996  2347 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
W20260709 13:02:36.324783  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 114434us
I20260709 13:02:36.259600  8342 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:36.233124  8377 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:36.360178  8342 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:36.264850  8320 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:36.361686  8181 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:36.361310  8342 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.362216  8181 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:36.362560  8342 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.329332  1554 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:36.253778  8341 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:36.363600  8341 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:36.363333  8181 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.364073  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:36.365010  1708 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:02:36.364679  8341 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:36.366469  8341 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:36.367929  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:02:36.344130  1561 rpcz_store.cc:269] 0709 13:02:34.939767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.006860 (+ 67093us) service_pool.cc:224] Handling call
0709 13:02:36.257978 (+1251118us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b54003205d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:36.258135 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2182889216}
I20260709 13:02:36.346741  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:36.371882  8195 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.372418  1675 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:36.372553  8195 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.373955  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:36.267820  8354 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:02:36.374935  8195 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:36.361107  8377 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.326522  8214 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:36.346885  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:36.355911  8343 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:36.343627  7957 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:36.343130  8322 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:36.321758  8098 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.429824  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:36.404515  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 43523us
I20260709 13:02:36.379318  8090 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:36.299840  8376 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.431346  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:36.433463  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59587.8us
I20260709 13:02:36.432660  8366 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:36.438306  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 70433.9us
I20260709 13:02:36.425623  8106 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:36.425798  8300 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:36.461199  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 195889us
I20260709 13:02:36.463146  8387 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:36.427899  8356 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:36.430145  8355 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:36.465348  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.466776  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 120028us
I20260709 13:02:36.490509  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:36.492902  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:36.472903  1527 leader_election.cc:336] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.570s (SENT)
I20260709 13:02:36.493795  8117 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:36.494460  8117 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:36.494735  1527 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.645s (SENT)
I20260709 13:02:36.494974  8117 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:36.495468  1527 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:36.498821  1456 catalog_manager.cc:5697] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 8 to 13, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 13 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:36.500797  8366 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:36.501668  8366 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:36.346642  8336 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.502511  8366 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:36.502879  8366 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:36.503716  8366 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.504240  8336 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:36.378641  8245 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.407243  8149 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.505096  8366 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:36.465205  8382 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:36.506599  8245 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:36.439162  8090 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.477550  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:36.507216  1629 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:36.500101  1525 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:36.456473  8370 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:36.513998  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.533138  1525 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:36.478312  8377 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:36.432756  8214 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.515005  8336 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:36.517289  8384 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:02:36.444216  8347 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:02:36.539798  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:02:36.541813  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 83105us
I20260709 13:02:36.508044  8365 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:02:36.511593  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 80737.3us
I20260709 13:02:36.507737  8298 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.489219  8356 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:36.513660  8374 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:36.497398  8262 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:36.514241  8380 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:36.591456  8380 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:36.592685  8380 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.593652  8389 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 13:02:36.594491  8380 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.595867  8342 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:36.456434  7957 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.559381  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:36.598982  8337 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:36.517146  7796 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
W20260709 13:02:36.601572  8400 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:02:36.527716  8248 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:36.508190  8090 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:36.513134  8098 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:36.510869  8312 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:36.557880  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:36.562101  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:36.529325  8387 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:36.611835  8402 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:02:36.574520  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 195108us
I20260709 13:02:36.589771  8363 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.109s
I20260709 13:02:36.590369  8376 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:36.590857  8373 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:36.606199  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44164.8us
I20260709 13:02:36.485932  8284 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.543912  8383 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:02:36.572095  8356 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.560388  8336 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.611210  8214 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:36.572695  8365 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:36.572516  8374 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:36.649252  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:36.589152  8300 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:02:36.614400  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 137625us
W20260709 13:02:36.614984  8055 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.561s	user 0.007s	sys 0.005s
I20260709 13:02:36.555152  8227 consensus_queue.cc:1059] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 13:02:36.627079  1558 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 13:02:36.658087  1558 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:36.631289  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:36.519901  8322 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.618559  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:36.630753  7796 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.618252  8216 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:02:36.638418  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 80630.1us
I20260709 13:02:36.640920  8398 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:36.539656  8343 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.641880  1553 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:36.623668  8337 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:02:36.612555  8387 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.642257  8386 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:02:36.645628  7937 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:36.645553  8281 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.008s
W20260709 13:02:36.690279  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.691790  1675 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:36.691879  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:36.693017  1558 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:36.646526  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:36.694077  1710 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:36.541669  8370 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.587925  8357 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
W20260709 13:02:36.658931  1527 leader_election.cc:336] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.864s (SENT)
I20260709 13:02:36.630751  8262 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.625797  8342 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:36.626289  1456 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 14 to 16, now has a pending config: VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194), VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 16 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } } pending_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:36.652879  8374 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:36.679731  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61221.7us
I20260709 13:02:36.653904  8300 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.705655  8300 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:36.645192  8117 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.711253  8117 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:36.713587  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 74843.3us
I20260709 13:02:36.685860  8376 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:36.687647  1708 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:36.684660  8398 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.685313  8149 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:36.678123  8404 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.703553  1702 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:36.721380  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:36.690374  8373 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:36.688750  8248 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:36.729063  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51241us
W20260709 13:02:36.703275  8211 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.002s
I20260709 13:02:36.682971  7796 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:36.736997  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.744333  1674 leader_election.cc:400] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17.
I20260709 13:02:36.700791  8343 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:36.756820  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:36.723107  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 95320.4us
I20260709 13:02:36.681591  8181 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:02:36.724220  8142 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:36.753618  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 94665.2us
I20260709 13:02:36.754705  1674 leader_election.cc:403] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:36.758208  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 53674.4us
W20260709 13:02:36.768481  1527 leader_election.cc:336] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.873s (SENT)
I20260709 13:02:36.771384  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:36.764709  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 72949.6us
I20260709 13:02:36.766186  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:36.766464  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:36.725695  8398 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:36.732422  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.785723  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:36.787191  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:36.774243  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:36.783643  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:36.788185  1705 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:36.687052  8312 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.724707  8106 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:36.746595  8373 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:36.791167  1701 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:36.792358  1553 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 22: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:36.796237  8248 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.797878  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:36.799022  2353 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:36.799322  2344 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEARNER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:36.800055  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:36.800855  2353 raft_consensus.cc:2368] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18.
I20260709 13:02:36.800699  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:36.801604  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:36.801771  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:36.801877  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:36.802263  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:36.802585  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:36.803597  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:36.804189  8386 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:36.804574  8098 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:36.804705  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:36.805886  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:36.760241  8211 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.806960  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:36.768062  7957 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:36.807755  7796 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:36.804297  8383 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:36.747680  8262 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.773461  8376 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:36.817366  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:36.821062  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:36.803128   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:36.824086  1456 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 9 to 11, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 11 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:36.820657  1710 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:36.845453  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:36.820555  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:36.827140  2348 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:36.784520  8216 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:36.831557  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:36.832566  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.835933  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 67460.2us
I20260709 13:02:36.836591  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:36.723912  8342 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:36.848381  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.849121  1558 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:36.849224  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:36.852119  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:36.849740  1527 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 2.199s (SENT) [suppressed 73 similar messages]
I20260709 13:02:36.855548  8401 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:36.855963  8401 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:36.856125  8299 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.001s
I20260709 13:02:36.858146  8342 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:36.858558  8181 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:36.858737  8322 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.859115  8181 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:36.859843  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:36.860019  8181 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:36.862010  8403 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:36.862449  1710 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 20. Current term is 21. Ops: [17.13-20.15]
I20260709 13:02:36.862074  8376 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:36.867782  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 143565us
I20260709 13:02:36.862622  8403 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:36.790966  8142 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.872217  2353 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:36.867882  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:36.894467  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.896634  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:36.900893  1601 connection.cc:256] RPC call timeout handler was delayed by -2.46467512199996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 13:02:36.902176  1601 connection.cc:256] RPC call timeout handler was delayed by -2.3801051890000053s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
I20260709 13:02:36.902297  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:36.903239  8409 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.136s
W20260709 13:02:36.902908  1601 connection.cc:256] RPC call timeout handler was delayed by -2.3262509979999777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
I20260709 13:02:36.848640  8106 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:36.906108  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:36.841637  8098 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:36.906817  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 57070us
I20260709 13:02:36.853747  8245 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:36.905082  1601 connection.cc:256] RPC call timeout handler was delayed by -2.2169153270000379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
I20260709 13:02:36.767534  8365 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.909459  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:36.910099  1601 connection.cc:256] RPC call timeout handler was delayed by -2.0354051890000164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2599s for a response
I20260709 13:02:36.897855  8357 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:36.911391  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7672051889999807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2897s for a response
I20260709 13:02:36.900753  8312 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:36.913720  1601 connection.cc:256] RPC call timeout handler was delayed by -1.7645051889999763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29s for a response
W20260709 13:02:36.916143  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47582.8us
I20260709 13:02:36.918183  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:36.918915  2348 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:36.915203  1601 connection.cc:256] RPC call timeout handler was delayed by -1.5791263450000201s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
I20260709 13:02:36.922297  8298 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:36.913837  8405 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:02:36.925634  8142 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:36.925860  2352 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:36.927146  8408 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:02:36.927407  2352 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: [14.11-16.12]
I20260709 13:02:36.928086  8216 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.907138  7937 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:36.930284  7937 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.894832  8299 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:36.940799  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.943403  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.920945  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4882526020000455s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2952s for a response
I20260709 13:02:36.917727  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:36.949877  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 191454us
I20260709 13:02:36.951339  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:36.956645  2348 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:02:36.962353  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:36.964968  1707 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:36.913589  8245 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.976341  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:36.916013  8357 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:36.982834  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 65956.7us
I20260709 13:02:36.920325  8312 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.955271  8298 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:36.953152  8409 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:02:37.008491  8423 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:36.908391  8373 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.012102  8142 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:37.020344  8090 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:36.941972  8403 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.021821  8403 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.022800  8245 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:37.023139  8409 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:37.023556  8409 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:37.029497  1674 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:37.017388  8366 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.000s
W20260709 13:02:37.039449  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 77121.8us
I20260709 13:02:37.025187  7929 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.046761  8245 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.022825  8090 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:37.027976  8423 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:37.065176  1559 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:37.057217  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:37.003122  8106 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.070009  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:37.070786  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:37.055109  1629 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:37.072808  1631 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:37.072968  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:37.073814  1628 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:37.074702  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:37.075188  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:37.075322  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:37.076133  1635 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:37.076238  1627 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:37.076911  1632 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:02:37.014309  8298 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.051110  8025 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:37.080490  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:37.058394  8373 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.081271  1629 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.087647  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:37.082651  1634 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:37.097895  2347 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:37.101661  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:37.107064  2348 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:02:37.108197  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:37.112917  8355 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.584s	user 0.004s	sys 0.009s
I20260709 13:02:37.113932  1629 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:37.116348  1708 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:37.119002  1631 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:37.119519  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.121662  1629 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:37.058233  7929 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:37.125459  2318 leader_election.cc:336] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.576s (SENT)
I20260709 13:02:37.082441  8090 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.093681  8142 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:37.126045  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:37.057799  8245 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.126060  2318 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:37.128312  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:37.129889  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:37.131775  8421 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:02:37.118795  8055 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:37.132754  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:37.132887  8389 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:37.133080  8055 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.133625  8055 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.134375  8389 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:37.135246  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:37.138010  2352 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:37.173862  8090 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.174527  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:37.179301  1631 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:37.120752  8025 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:37.183133  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 95509us
I20260709 13:02:37.192056  8090 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:37.198611  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 73123.6us
W20260709 13:02:37.200435  2318 leader_election.cc:336] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.627s (SENT)
I20260709 13:02:37.200894  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:37.138934  8142 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:37.204003  2352 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:37.209827  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:37.210692  8245 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:37.212114  8025 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.214080  8298 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:37.215452  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:37.168700  8355 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:37.210661  1708 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:37.214149  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:37.222991  1636 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:37.224496  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:37.116519  8356 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:37.233460  2318 leader_election.cc:336] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Timed out: RequestConsensusVote RPC to 127.0.107.195:45431 timed out after 1.674s (SENT)
W20260709 13:02:37.233515  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 103607us
I20260709 13:02:37.237676  1674 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:37.240552  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.240938  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.246388  1636 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:37.222066  8142 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:37.256472  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:37.257339  1561 raft_consensus.cc:2376] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15. Re-sending same reply.
I20260709 13:02:37.236259  8245 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:37.261708  1552 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:37.180563  8406 consensus_queue.cc:1059] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:02:37.262156  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:37.264256  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:37.264581  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:37.264673  1702 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:37.264863  1555 raft_consensus.cc:2376] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15. Re-sending same reply.
I20260709 13:02:37.200316  8300 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.266116  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:37.238014  8336 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:37.262713  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:37.288174  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:37.295043  1561 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:37.290762  1558 raft_consensus.cc:2376] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15. Re-sending same reply.
I20260709 13:02:37.291501  1708 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:37.293637  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:37.290409  8388 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.816s	user 0.000s	sys 0.009s
I20260709 13:02:37.245903  8214 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.300158  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:37.304994  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.307387  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56089us
I20260709 13:02:37.313297  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:37.258836  8312 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:37.256140  8117 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.262921  8387 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:37.288241  8422 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:02:37.319897  8356 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:37.321676  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3147) took 1281 ms. Trace:
I20260709 13:02:37.237881  8149 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:02:37.342383  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:37.338770  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:37.261435  8389 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:02:37.264268  8366 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:02:37.364279  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:37.340551  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 93120.9us
I20260709 13:02:37.354112  1636 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:37.366626  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:37.338102  1701 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:37.355628  1555 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:37.217543  8423 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.369805  1636 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:37.373142  1561 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:02:37.367002  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 53792.9us
I20260709 13:02:37.308791  8386 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:37.327154  8194 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:37.378388  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:37.366402  1706 rpcz_store.cc:276] 0709 13:02:36.039767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.050612 (+ 10845us) service_pool.cc:224] Handling call
0709 13:02:36.065904 (+ 15292us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:36.112036 (+ 46132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:36.112080 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:36.170284 (+ 58204us) log.cc:844] Serialized 52 byte log entry
0709 13:02:36.313163 (+142879us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:36.427442 (+114279us) log.cc:844] Serialized 14 byte log entry
0709 13:02:36.442910 (+ 15468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.442996 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:36.706327 (+263331us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b18004e15a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f
0709 13:02:36.707717 (+  1390us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b48001e1a58. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:02:37.229231 (+521514us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:37.321257 (+ 92026us) raft_consensus.cc:1685] finished
0709 13:02:37.321288 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.321590 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42938,"prepare.queue_time_us":175088,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":76694,"raft.run_cpu_time_us":4496,"raft.run_wall_time_us":252604,"spinlock_wait_cycles":72960128,"thread_start_us":186684,"threads_started":2,"wal-append.queue_time_us":186048}
W20260709 13:02:37.365053  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 111848us
I20260709 13:02:37.346805  8430 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:37.365610  8363 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:37.382210  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.328449  8336 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.346777  8387 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.375708  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:37.390354  1561 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: [17.16-17.16]
I20260709 13:02:37.393939  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:37.400281  8366 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.391824  8430 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:37.401537  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:37.340955  8214 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.392904  7796 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:37.399143  8377 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.852s	user 0.010s	sys 0.002s
I20260709 13:02:37.435338  2347 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:37.441637  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:37.346669  8117 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:37.396066  8429 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.057s	user 0.001s	sys 0.001s
I20260709 13:02:37.398429  8387 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.400915  8366 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.398301  8336 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.453660  1636 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:37.454180  8430 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.454741  8214 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:37.455371  8357 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:37.403302  8284 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:37.456321  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:37.457221  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:37.474583  1555 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
W20260709 13:02:37.458873  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 63436us
W20260709 13:02:37.460788  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.419241  7929 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:37.438745  8424 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 13:02:37.419636  8363 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:37.421392  8427 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:02:37.421641  8181 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:02:37.445176  8300 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:37.463882  8398 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.647s	user 0.008s	sys 0.002s
I20260709 13:02:37.460402  7796 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:37.432823  8055 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.488608  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:37.490681  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:37.491384  8440 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:37.491779  8440 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:37.491935  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:37.493002  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:37.492617  8440 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.479243  8363 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.478296  8386 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:37.537133  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:37.537765  1558 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.540630  8386 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:37.542181  8426 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 13:02:37.517238  8181 consensus_queue.cc:1243] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:02:37.543668  8439 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.544025  8322 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.685s	user 0.005s	sys 0.005s
I20260709 13:02:37.544242  8439 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.546198  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:37.550024  1706 raft_consensus.cc:2410] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:37.514394  8430 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.554683  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:37.554693  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 98413.8us
I20260709 13:02:37.552721  1634 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:37.536237  8227 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:37.519146  8363 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:37.551821  8403 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.001s
I20260709 13:02:37.566121  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:37.566731  1634 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:37.565042  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:37.570629  8055 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:37.572773  8374 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.846s	user 0.009s	sys 0.001s
I20260709 13:02:37.573177  8374 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.575291  8374 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:37.577271  1675 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:37.579005  1675 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:37.579829  8435 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:37.580641  8437 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:02:37.488490  8398 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.562427  8227 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:37.582612  1561 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:02:37.583621  8398 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:37.584331  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:37.582293  1558 raft_consensus.cc:2410] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:02:37.585515  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:37.586078  1555 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.583374  8227 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:37.590088  8433 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:02:37.590472  8227 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.585515  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:37.597759  1555 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:37.552258  7929 raft_consensus.cc:2764] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:02:37.599176  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:37.596895  8142 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.555393  8440 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.602279  1708 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
W20260709 13:02:37.608994  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43999.9us
I20260709 13:02:37.618608  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:37.619292  1555 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:37.623762  1555 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:37.625846  1555 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:02:37.630553  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:37.602907  8435 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:37.609436  8181 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:37.636489  1628 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 22: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:37.631179  1556 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:37.626442  1555 pending_rounds.cc:85] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 10
I20260709 13:02:37.637943  1633 raft_consensus.cc:2376] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16. Re-sending same reply.
I20260709 13:02:37.637233  1555 pending_rounds.cc:107] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:02:37.617839  8055 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:37.620195  8322 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.638391  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:37.652041  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:37.638543  1555 raft_consensus.cc:2889] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:37.641003  8432 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:37.641908  8117 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:37.639348  8055 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.645756  8366 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:37.646935  8383 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.801s	user 0.010s	sys 0.000s
W20260709 13:02:37.647626  8343 consensus_meta.cc:287] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.880s	user 0.010s	sys 0.000s
W20260709 13:02:37.650693  8438 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:02:37.663683  1634 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:37.609573  8403 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:02:37.671774  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:37.653512  8300 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:37.682086  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:37.690817  1634 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:37.692263  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:37.694689  8214 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:37.694604  8347 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:37.696159  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:37.696249  8347 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:37.697172  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:37.698055  8441 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:37.698115  1634 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.671446  8055 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.609512  7796 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.667155  8356 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:37.677745  8383 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17.
I20260709 13:02:37.700968  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:37.636711  8106 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:02:37.672614  8181 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:37.717805  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 64674.8us
I20260709 13:02:37.716727  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:37.722925  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:37.719909  8437 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:37.717564  8398 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.720396  8336 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:37.710238  8300 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:02:37.745011  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:37.749053  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:37.750448  1634 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:37.752847  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:37.745014  8214 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:37.753710  1556 raft_consensus.cc:2393] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:37.753964  8407 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.946s	user 0.005s	sys 0.000s
I20260709 13:02:37.754529  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:37.756023  8181 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.756492  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 1043) took 1204 ms. Trace:
I20260709 13:02:37.756593  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:37.757580  1634 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:37.756989  1554 rpcz_store.cc:276] 0709 13:02:36.551980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.552093 (+   113us) service_pool.cc:224] Handling call
0709 13:02:36.552485 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:36.571683 (+ 19198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:36.571713 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:36.587182 (+ 15469us) log.cc:844] Serialized 26 byte log entry
0709 13:02:36.683425 (+ 96243us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:36.683492 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.683596 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.753248 (+1069652us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:37.756202 (+  2954us) raft_consensus.cc:1685] finished
0709 13:02:37.756222 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.756435 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34097,"raft.queue_time_us":18866,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":9324,"spinlock_wait_cycles":42038656,"thread_start_us":85059,"threads_started":1,"wal-append.queue_time_us":193888}
I20260709 13:02:37.755172  1556 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:37.759032  1702 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:37.759801  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:37.753813  8214 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.761688  8214 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.762341  8370 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:37.764185  1634 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 18. Current term is 19. Ops: [16.12-18.13]
I20260709 13:02:37.740749  8383 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:02:37.757073  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.786895  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.770745  8300 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:37.771631  8415 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:02:37.772785  8404 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.016s	user 0.009s	sys 0.000s
I20260709 13:02:37.735535  8435 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.773977  8211 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:37.774469  8055 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.775451  8423 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.777097  8373 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.679s	user 0.008s	sys 0.001s
I20260709 13:02:37.779158  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:37.780314  8298 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.783525  8453 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:37.740636  8441 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:37.751338  8439 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:37.811175  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:37.815821  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 96830.7us
I20260709 13:02:37.785670  8383 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:37.748131  8403 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:02:37.826421  1558 raft_consensus.cc:2410] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:02:37.790596  8300 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:37.813055  8356 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:37.838337  8356 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:37.842377  8357 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:37.846446  8441 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.847088  8441 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.849967  1553 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:37.851362  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:37.852461  1553 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 22: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:37.853868  8383 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:37.854620  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.793823  8415 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:37.797263  8404 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:37.750190  8398 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.809342  8142 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:37.863758  8412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.051s	user 0.017s	sys 0.040s
W20260709 13:02:37.872215  8412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.059s	user 0.017s	sys 0.041s
I20260709 13:02:37.827854  8453 consensus_queue.cc:1243] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:02:37.881121  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.903378  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:37.876066  8322 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:37.826280  8298 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.811817  8423 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.911751  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:37.887305  8106 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:37.917613  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:37.864507  8437 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:37.915038  8357 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.934245  8055 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:37.941983  8450 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:02:37.944289  8214 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.916512  8439 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.919888  8370 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.951143  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.951660  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.952824  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:37.954031  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:37.925727  8410 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:37.954545  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:37.960842  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:37.932138  8211 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:37.963584  1554 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:37.964939  1554 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:37.929386  8106 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.970652  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:37.958114  8410 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.971448  8410 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.947108  8357 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:37.974838  8298 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:37.976351  8357 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:37.977685  8262 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:37.977764  8337 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:37.978381  8337 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:37.979301  8337 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.985359  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:37.990427  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:37.991075  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:37.991088  1702 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:37.980383  8337 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.955055  8439 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.993134  8380 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:37.994946  8373 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:38.002303  8336 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:02:37.921214  8415 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.007026  8415 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:37.924661  8300 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:37.969280  8423 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:37.995752  8245 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:38.015027  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:37.993966  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:38.001340  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:38.020713  1559 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
W20260709 13:02:38.001294  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.021942  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.997592  8347 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:38.035017  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:38.074378  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:37.955211  8370 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.076084  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:38.076834  8262 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:38.043149  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 49245.2us
I20260709 13:02:38.050197  8460 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:37.983004  8387 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:38.055699  1631 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:02:38.056121  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:38.058476  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:38.059906  8389 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:38.042074  8380 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.064148  8453 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.064889  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:38.066089  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:38.067404  8461 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:38.068018  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2483) took 1294 ms. Trace:
I20260709 13:02:38.068640  1552 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:38.069684  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:38.017724  8430 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:38.086247  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.079591  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:37.955174  8363 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.166836  1558 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:02:38.099750  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:38.101724  1627 raft_consensus.cc:2410] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:02:38.101816  1708 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:38.093894  1630 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:38.104367  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:38.077277  8262 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.105392  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:38.106305  1560 rpcz_store.cc:276] 0709 13:02:36.774591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.774705 (+   114us) service_pool.cc:224] Handling call
0709 13:02:38.067785 (+1293080us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b54002ff654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:38.067957 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1489395072}
W20260709 13:02:38.082219  8448 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.344s	user 0.003s	sys 0.000s
W20260709 13:02:38.083067  8447 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:02:38.110222  1635 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:38.050184  8300 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:38.038766  8423 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:38.106303  8443 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.550s	user 0.000s	sys 0.002s
I20260709 13:02:38.095032  8336 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:38.094048  8389 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:38.077323  8370 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:38.078197  8214 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:38.097998  8373 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:38.147116  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:38.149529  8055 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:38.147619  7796 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:02:38.151831  8299 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:38.153225  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2455) took 1309 ms. Trace:
I20260709 13:02:38.155539  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:38.158607  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2447) took 1454 ms (client timeout 1928 ms). Trace:
I20260709 13:02:38.137042  8430 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.134979  8322 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:38.107090  8380 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:38.225721  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:38.162030  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:38.194757  8262 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:38.140936  8461 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:38.174464  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:38.177623  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 162579us
I20260709 13:02:38.193256  8456 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.037678  8437 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:38.197095  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 97390us
I20260709 13:02:38.147771  8398 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:38.111063  8347 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:02:38.136273  7929 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 13:02:38.251812  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.220937  1709 rpcz_store.cc:269] 0709 13:02:36.704581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.704835 (+   254us) service_pool.cc:224] Handling call
0709 13:02:38.158436 (+1453601us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b5400223b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:38.158550 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2046949632}
I20260709 13:02:38.220708  1701 rpcz_store.cc:276] 0709 13:02:36.843491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.843766 (+   275us) service_pool.cc:224] Handling call
0709 13:02:36.852864 (+  9098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.374319 (+521455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.152976 (+778657us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b540014d2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:38.153152 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1089193728}
I20260709 13:02:38.221055  1703 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:38.110443  8453 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:38.224383  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 77308.4us
I20260709 13:02:38.161582  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:38.226625  1559 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 12.
I20260709 13:02:38.174106  1556 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:38.173826  8440 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.173655  8149 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.239068  1627 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:38.097448  8211 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
W20260709 13:02:38.240980  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.195921  8336 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.194991  8423 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.245091  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.245713  8379 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:38.246518  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:38.247845  8430 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:38.248811  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.270020  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:38.249459  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.271006  1627 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:38.199174  8343 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:02:38.259589  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 116169us
I20260709 13:02:38.269850  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:38.273237  1701 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:38.255096  1636 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:38.274077  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:38.274432  2349 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:38.274626  1701 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 12.
I20260709 13:02:38.262156  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:38.262360  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:38.276503  1559 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:38.276682  1630 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:38.277815  1559 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:02:38.278148  1559 pending_rounds.cc:85] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 10
I20260709 13:02:38.262187  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:38.193569  8356 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:02:38.263638  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.225703  8181 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:38.313951  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:38.314631  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:38.266090  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.315433  1636 raft_consensus.cc:2410] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:38.266044  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:38.316684  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:38.278407  1559 pending_rounds.cc:107] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:02:38.283449  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:38.283483  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.317507  1627 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:38.278263  1630 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:02:38.318776  1627 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:38.320511  1558 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:38.259409  8398 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:38.321066  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:38.301728  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.237771  8322 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:38.253226  8456 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.326049  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.329631  1627 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:38.329960  1636 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:38.317457  1559 raft_consensus.cc:2889] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.327706  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:38.333750  1559 pending_rounds.cc:107] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:02:38.334204  1559 raft_consensus.cc:2889] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.336000  1636 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:02:38.325342  1630 pending_rounds.cc:85] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:02:38.330785  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.330868  1627 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:38.320093  2348 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:38.251847  8366 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:02:38.339680  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:38.316078  8461 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:38.320091  8211 consensus_queue.cc:1243] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:02:38.337838  1630 pending_rounds.cc:107] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:02:38.272718  8336 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:38.272850  8423 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:38.343271  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:38.358590  1708 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:38.273144  8379 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:38.344537  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 78528.7us
I20260709 13:02:38.358158  1630 raft_consensus.cc:2889] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.332111  8055 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:38.269196  8149 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.323076  8347 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:38.267884  8440 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.360752  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:38.383123  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:38.319097  8430 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:02:38.376219  1636 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:38.360692  8373 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.392373  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:38.392969  1559 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:38.358129  8461 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:38.265746  8389 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.391547  1627 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:38.330847  8398 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.378409  8439 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:38.399664  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:38.379702  7929 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:38.388083  8055 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.381357  8387 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:38.402884  1708 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:38.413882  8439 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.414521  8439 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:38.419404  8416 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.565s	user 0.000s	sys 0.017s
I20260709 13:02:38.419950  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:38.421200  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:38.422251  8453 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:38.422514  8322 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:38.423738  8363 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:38.426353  8194 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:38.426993  8194 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.427430  8194 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.427743  8456 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:38.422344  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:38.407924  8055 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:38.449317  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:38.450888  1702 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:38.454555  1708 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:38.429324  8194 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:38.460254  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 100374us
I20260709 13:02:38.399643  8347 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:02:38.461155  2317 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:38.450510  8414 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.596s	user 0.000s	sys 0.006s
I20260709 13:02:38.463295  1708 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:02:38.434021  8363 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:38.464519  1702 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 13:02:38.451318  8482 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.002s
I20260709 13:02:38.434684  7929 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:38.403399  8460 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:02:38.398835  8373 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.473032  8464 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:38.468005  8461 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:38.469419  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:38.467504  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 198953us
I20260709 13:02:38.477887  1559 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:38.460210  8484 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:38.479552  8440 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:38.430056  8379 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.467171  1708 pending_rounds.cc:85] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 14
I20260709 13:02:38.473451  8380 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:38.493768  1708 pending_rounds.cc:107] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:02:38.495653  1708 raft_consensus.cc:2889] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.481833  8483 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:38.491991  8379 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:38.509711  8262 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.504174  2348 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:02:38.511242  2348 pending_rounds.cc:85] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 10
I20260709 13:02:38.511507  2348 pending_rounds.cc:107] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:02:38.511811  2348 raft_consensus.cc:2889] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.513737  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.514036  8380 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.514523  8262 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:38.514647  8380 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:38.515908  8446 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.859s	user 0.004s	sys 0.000s
I20260709 13:02:38.516839  8380 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:38.510247  2349 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:38.510644  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:38.510831  8480 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:02:38.514142  1628 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:38.520248  8055 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.503815  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:38.522891  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2446) took 1917 ms. Trace:
I20260709 13:02:38.505178  1708 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
W20260709 13:02:38.524569  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 55198.4us
I20260709 13:02:38.523272  1704 rpcz_store.cc:276] 0709 13:02:36.605277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.615061 (+  9784us) service_pool.cc:224] Handling call
0709 13:02:36.627793 (+ 12732us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:36.632934 (+  5141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:36.632965 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:36.653653 (+ 20688us) log.cc:844] Serialized 52 byte log entry
0709 13:02:36.824002 (+170349us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:36.834744 (+ 10742us) log.cc:844] Serialized 14 byte log entry
0709 13:02:36.842176 (+  7432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.842257 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:38.363236 (+1520979us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:38.522565 (+159329us) raft_consensus.cc:1685] finished
0709 13:02:38.522663 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.522810 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7651,"prepare.queue_time_us":6682,"prepare.run_cpu_time_us":3602,"prepare.run_wall_time_us":3985,"raft.queue_time_us":16976,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":3868,"spinlock_wait_cycles":93418880,"thread_start_us":130507,"threads_started":1,"wal-append.queue_time_us":173888}
I20260709 13:02:38.481683  8363 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.529568  8488 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:02:38.530017  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.526345  8484 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:38.490376  8464 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:38.566025  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3158) took 1670 ms (client timeout 1673 ms). Trace:
W20260709 13:02:38.566712  1707 rpcz_store.cc:269] 0709 13:02:36.895944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.902052 (+  6108us) service_pool.cc:224] Handling call
0709 13:02:38.565772 (+1663720us) spinlock_profiling.cc:243] Waited 56.5 ms on lock 0x7b54002e81d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:38.565970 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129983232}
I20260709 13:02:38.568368  1710 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:02:38.574331  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:38.515275  8461 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:38.576337  2349 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:38.503239  8483 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:38.578042  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:38.580164  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 185131us
I20260709 13:02:38.585407  2344 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:38.582693  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:38.483047  8401 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:38.591800  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.589059  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.594050  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:38.628487  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 45841.9us
I20260709 13:02:38.629451  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:38.569898  8475 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
W20260709 13:02:38.592708  8410 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.621s	user 0.010s	sys 0.002s
I20260709 13:02:38.651034  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:38.576254  8488 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:38.587447  8262 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:38.581053  8483 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:38.683951  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 54557.2us
I20260709 13:02:38.590520  8149 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:38.693229  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.693957  2347 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:38.694856  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:38.695595  1557 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:38.696486  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:38.696957  1557 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:38.688800  8381 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:02:38.686620  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:38.688522  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.694936  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:38.698173  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1387) took 1365 ms. Trace:
W20260709 13:02:38.706881  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.702052  2352 rpcz_store.cc:276] 0709 13:02:37.315696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.368916 (+ 53220us) service_pool.cc:224] Handling call
0709 13:02:37.381658 (+ 12742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.402592 (+ 20934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.402627 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:37.402673 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:37.402707 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.680793 (+1278086us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b5400796b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:38.680865 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.681056 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47621376}
W20260709 13:02:38.712963  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 128008us
I20260709 13:02:38.653404  8453 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.686549  8401 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:38.591902  8387 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.111s
I20260709 13:02:38.719050  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:38.585399  8380 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.124s
I20260709 13:02:38.594230  8322 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.117s
I20260709 13:02:38.683905  8366 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.730324  2344 raft_consensus.cc:2410] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 13:02:38.697952  8437 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:38.746413  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:38.753159  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.592602  7929 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:38.709681  8149 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:02:38.721056  8300 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.527s	user 0.005s	sys 0.006s
I20260709 13:02:38.700997  8494 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:38.734203  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:38.687580  8410 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:38.770715  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.835151  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3217) took 1431 ms. Trace:
I20260709 13:02:38.835774  1561 rpcz_store.cc:276] 0709 13:02:37.403691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.403934 (+   243us) service_pool.cc:224] Handling call
0709 13:02:37.415537 (+ 11603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.583827 (+168290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.834748 (+1250921us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54002526d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:38.835065 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383330560}
I20260709 13:02:38.723624  8453 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:38.840359  2346 raft_consensus.cc:2410] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 22 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:02:38.846204  8281 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:02:38.861876  8412 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:38.862386  8412 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.862723  8412 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:38.748699  8366 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.744832  8484 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.867384  8366 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:38.710254  8464 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:38.868422  8423 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.869343  8464 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:38.871189  8336 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:38.873845  1559 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:38.874188  8363 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:38.874979  8441 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.027s	user 0.013s	sys 0.000s
I20260709 13:02:38.875450  8423 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:38.958703  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.877790  2353 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:38.946753  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3219) took 1483 ms. Trace:
I20260709 13:02:38.952138  1701 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:38.918318  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1230) took 1402 ms. Trace:
I20260709 13:02:38.876648  2345 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:38.931247  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1239) took 1262 ms. Trace:
I20260709 13:02:38.932164  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:38.909543  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:39.009111  1633 rpcz_store.cc:276] 0709 13:02:37.515691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.551689 (+ 35998us) service_pool.cc:224] Handling call
0709 13:02:38.918025 (+1366336us) spinlock_profiling.cc:243] Waited 69.9 ms on lock 0x7b540010dbd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:38.918229 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160763136}
W20260709 13:02:38.926149  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 1127) took 1692 ms (client timeout 2180 ms). Trace:
W20260709 13:02:39.014279  1706 rpcz_store.cc:269] 0709 13:02:37.234204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.401196 (+166992us) service_pool.cc:224] Handling call
0709 13:02:38.925876 (+1524680us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400246654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:38.926060 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299347200}
W20260709 13:02:38.925374  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.016745  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.947342  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:38.909946  2317 leader_election.cc:336] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.084s (SENT)
I20260709 13:02:38.927902  1702 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:02:39.020520  1702 pending_rounds.cc:85] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 12
I20260709 13:02:39.020982  1702 pending_rounds.cc:107] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:02:39.021499  1702 raft_consensus.cc:2889] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:38.883698  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.023442  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.891151  8245 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.853s	user 0.009s	sys 0.001s
I20260709 13:02:38.920600  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:39.025375  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 104791us
W20260709 13:02:39.030402  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 315237us
I20260709 13:02:39.008203  1552 rpcz_store.cc:276] 0709 13:02:37.463839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.464041 (+   202us) service_pool.cc:224] Handling call
0709 13:02:38.068432 (+604391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.134078 (+ 65646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.946503 (+812425us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b54002ff650. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:38.946666 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1388484864}
W20260709 13:02:38.897749  8490 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:02:38.970247  1456 catalog_manager.cc:5697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:39.009554  1631 rpcz_store.cc:276] 0709 13:02:37.668899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.856028 (+187129us) service_pool.cc:224] Handling call
0709 13:02:38.055427 (+199399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.095312 (+ 39885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.930826 (+835514us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54002db4d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:38.931183 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457331072}
I20260709 13:02:39.038712  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 1051) took 1131 ms. Trace:
I20260709 13:02:39.039194  1557 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:39.039141  1558 rpcz_store.cc:276] 0709 13:02:37.907251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.193921 (+286670us) service_pool.cc:224] Handling call
0709 13:02:38.222657 (+ 28736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.321324 (+ 98667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.038513 (+717189us) spinlock_profiling.cc:243] Waited 97.6 ms on lock 0x7b5400312254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:39.038659 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224500608}
I20260709 13:02:39.040124  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:39.027046  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:38.949326  8373 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:38.934589  8389 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:38.921427  8439 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:38.876075  8410 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:38.956530  8409 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.019398  8341 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:39.049863  8356 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:39.050493  8356 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:39.051688  8356 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.052976  8356 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:39.046097  1552 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:02:39.046241  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:39.056380  1561 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:39.010075  8336 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:39.059058  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 111761us
I20260709 13:02:38.886646  8453 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:38.843075  8488 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.964087  8487 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.050078  8341 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.875592  8441 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:38.922200  8098 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:38.843714  8149 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:39.056835  1552 pending_rounds.cc:85] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 12
I20260709 13:02:39.068073  1552 pending_rounds.cc:107] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:02:39.068555  1552 raft_consensus.cc:2889] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:39.057199  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:39.057838  1558 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:39.041277  8227 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:39.041576  8245 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:39.060547  1555 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:02:39.074435  1558 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:02:39.074828  1558 pending_rounds.cc:85] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 9
I20260709 13:02:39.075033  1558 pending_rounds.cc:107] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:02:39.075745  1552 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:02:39.060518  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:39.057092  1557 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:38.933072  8483 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.070994  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:39.075248  1558 raft_consensus.cc:2889] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:39.076085  1552 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 9
I20260709 13:02:39.081677  1552 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:02:39.082065  1552 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:39.079618  1558 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:39.079910  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:39.066054  8487 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.088243  8055 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:39.088689  8487 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:39.077955  8383 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:39.089735  8214 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:39.090142  8214 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.090534  8214 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:39.090705  8495 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.252s	user 0.003s	sys 0.000s
I20260709 13:02:39.091929  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3232) took 1158 ms. Trace:
I20260709 13:02:39.092741  8106 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:39.073345  8227 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.092276  1560 rpcz_store.cc:276] 0709 13:02:37.933707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.193895 (+260188us) service_pool.cc:224] Handling call
0709 13:02:38.194223 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.319451 (+125228us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:38.319486 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:38.319958 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.678130 (+358172us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:38.678169 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.678235 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.090145 (+411910us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:39.091627 (+  1482us) raft_consensus.cc:1685] finished
0709 13:02:39.091647 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.091869 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2872,"raft.queue_time_us":25950,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3078,"spinlock_wait_cycles":9458560,"thread_start_us":326348,"threads_started":2,"wal-append.queue_time_us":399516}
I20260709 13:02:39.093446  8227 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:39.079535  8098 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:39.095438  8149 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:39.096071  8497 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:39.096513  8497 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:39.097384  8497 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.098658  8497 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:39.100185  2352 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:39.102296  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:39.103449  8381 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:39.104429  1559 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:02:39.107694  8398 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:39.110217  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 200179us
I20260709 13:02:39.111776  8341 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:39.112469  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3162) took 1040 ms. Trace:
I20260709 13:02:39.112866  1701 rpcz_store.cc:276] 0709 13:02:38.072412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.339391 (+266979us) service_pool.cc:224] Handling call
0709 13:02:39.112304 (+772913us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b54001e8f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:39.112410 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1153842048}
I20260709 13:02:39.073803  8337 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.116077  8337 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:39.118434  8441 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:39.121930  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.123653  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.125689  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:39.125909  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 43610us
I20260709 13:02:39.126426  1703 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:39.127249  8398 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.127763  8398 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:39.130162  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.130795  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2488) took 1387 ms. Trace:
I20260709 13:02:39.131001  1703 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:39.131191  1555 rpcz_store.cc:276] 0709 13:02:37.743740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.744762 (+  1022us) service_pool.cc:224] Handling call
0709 13:02:39.130598 (+1385836us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540026b1d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:39.130729 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2992457088}
I20260709 13:02:39.132210  1629 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:39.132506  8387 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:39.140352  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:39.133003  8398 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:39.131662  8098 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:02:39.133888  8412 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:39.134971  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.133260  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3228) took 1293 ms. Trace:
I20260709 13:02:39.062808  8025 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:39.137207  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.160842  1701 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:39.137919  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.161623  2350 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:39.141247  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.142789  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:39.146296  1561 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:39.155404  1554 rpcz_store.cc:276] 0709 13:02:37.839821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.967308 (+127487us) service_pool.cc:224] Handling call
0709 13:02:37.967573 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.001601 (+ 34028us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:38.088363 (+ 86762us) log.cc:844] Serialized 14 byte log entry
0709 13:02:38.168796 (+ 80433us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:38.227115 (+ 58319us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.244887 (+ 17772us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:38.244937 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.132846 (+887909us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5400251cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:39.132914 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.132964 (+    50us) raft_consensus.cc:1685] finished
0709 13:02:39.133047 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.133201 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3145,"prepare.queue_time_us":88373,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":8133,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":22707,"spinlock_wait_cycles":55011584,"thread_start_us":78434,"threads_started":1,"wal-append.queue_time_us":159105}
I20260709 13:02:39.163486  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.163779  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:39.147684  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:39.164359  1703 raft_consensus.cc:2410] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 19 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 15.
I20260709 13:02:39.165836  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:39.166827  1557 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:39.150462  1707 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:39.150974  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.151960  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:39.061439  8460 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:02:39.073407  8464 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:39.157081  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 77198.7us
I20260709 13:02:39.159559  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.167387  1708 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:39.167812  1559 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:39.128572  8214 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.170334  1553 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:02:39.164882  1552 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:39.172082  8386 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.178128  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.179100  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:39.236032  1707 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:02:39.242447  1707 pending_rounds.cc:85] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 13
I20260709 13:02:39.195317  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.242694  1707 pending_rounds.cc:107] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:02:39.073230  8322 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 13:02:39.197798  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:39.195745  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:39.154771  8413 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:02:39.238016  1559 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:39.206192  1706 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:39.255900  1701 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:02:39.203382  1555 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:39.257790  1555 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:02:39.234198  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.162022  8106 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:39.240888  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:39.243024  1707 raft_consensus.cc:2889] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:39.191000  8474 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:39.205641  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:39.250623  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:39.251173  1709 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:39.252214  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56519.6us
I20260709 13:02:39.161760  8245 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:39.254130  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:39.275455  1705 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:02:39.253396  1561 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
W20260709 13:02:39.203687  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.268601  1704 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:02:39.269119  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.159672  8475 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 13:02:39.266507  8336 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:39.271627  1706 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:39.201690  8500 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:39.165844  8194 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 13:02:39.205343  8262 raft_consensus.cc:2764] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:02:39.205858  8464 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.267040  8487 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:39.202333  8508 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:39.240708  8441 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:39.165462  8502 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.108s
I20260709 13:02:39.305413  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:39.309667  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:39.291115  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:39.311767  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:39.313505  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 167269us
I20260709 13:02:39.316211  1627 raft_consensus.cc:1217] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Deduplicated request from leader. Original: 15.13->[16.14-16.14]   Dedup: 16.14->[]
I20260709 13:02:39.316707  1631 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:39.317852  1705 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:39.320206  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1580) took 1056 ms. Trace:
I20260709 13:02:39.320731  2345 rpcz_store.cc:276] 0709 13:02:38.263587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.273075 (+  9488us) service_pool.cc:224] Handling call
0709 13:02:39.319968 (+1046893us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b540061fb54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:39.320133 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1256175488}
I20260709 13:02:39.311441  2349 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:39.324249  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:39.295941  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:39.202745  8440 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:39.250713  8398 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.272482  8055 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:39.304708  1708 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 13.
I20260709 13:02:39.300113  8464 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:39.330050  1636 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:39.297329  8500 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:39.346273  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2460) took 2482 ms. Trace:
I20260709 13:02:39.387531  1710 rpcz_store.cc:276] 0709 13:02:36.863594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.897125 (+ 33531us) service_pool.cc:224] Handling call
0709 13:02:38.568021 (+1670896us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.568956 (+   935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.345875 (+776919us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54002e81d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:39.346175 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3811832064}
I20260709 13:02:39.315215  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1137) took 1073 ms. Trace:
I20260709 13:02:39.388890  1702 rpcz_store.cc:276] 0709 13:02:38.242190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.473272 (+231082us) service_pool.cc:224] Handling call
0709 13:02:38.927409 (+454137us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.022205 (+ 94796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.314973 (+292768us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b5400246650. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:39.315149 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1020832000}
I20260709 13:02:39.358163  1627 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:02:39.391672  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:39.392383  1627 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:39.397570  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:39.398126  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:39.266739  8386 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:39.404196  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:39.405032  1635 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:39.305585  8245 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:39.308954  8441 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:39.328984  1561 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:39.330302  8439 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:39.297746  8336 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:39.331691  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2493) took 1534 ms. Trace:
I20260709 13:02:39.383121  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:39.383755  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:39.296841  8381 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:02:39.412766  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:39.417886  1634 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:39.418215  1631 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:39.419071  1628 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:39.314067  8401 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:39.379771  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:39.421095  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:39.422770  1632 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:02:39.448380  1632 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 12
I20260709 13:02:39.448997  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:39.450106  1561 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:39.284374  8106 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.441926  1636 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:02:39.424046  1553 rpcz_store.cc:276] 0709 13:02:37.796894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.962333 (+165439us) service_pool.cc:224] Handling call
0709 13:02:37.973811 (+ 11478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.236800 (+1262989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.331364 (+ 94564us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54001fd454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:39.331616 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2749325056}
I20260709 13:02:39.442605  1630 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:02:39.442312  1633 raft_consensus.cc:2410] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:02:39.380262  8373 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:39.462404  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:39.368754  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.305161  8487 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:39.472570  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:39.448655  1632 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 17.13
I20260709 13:02:39.483632  1632 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Aborting config change with OpId 17.13: Aborted: Op aborted by new leader
I20260709 13:02:39.484053  1632 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:39.484462  8106 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:39.485208  8494 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:39.495677  1556 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:39.423791  8439 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.291949  8474 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:39.521299  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 48056.2us
I20260709 13:02:39.447669  8401 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.527953  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:39.501055  2345 raft_consensus.cc:2368] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17.
I20260709 13:02:39.529793  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:39.509941  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3100) took 1505 ms. Trace:
I20260709 13:02:39.381449  8398 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:39.530808  1629 rpcz_store.cc:276] 0709 13:02:38.004497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.246262 (+241765us) service_pool.cc:224] Handling call
0709 13:02:39.493289 (+1247027us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b540017be54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:39.493463 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1863433600}
I20260709 13:02:39.523236  1708 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:39.460186  8322 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.533531  8322 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:39.536365  8322 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:39.399506  8440 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:39.531318  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.551084  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.552054  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:39.552815  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:39.308847  8508 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.575021  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:39.518342  8439 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:39.440253  8245 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.581224  2351 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
W20260709 13:02:39.589589  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.593446  2345 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:39.546693  7929 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.520903  8494 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:39.604791  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.550246  8401 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:39.584161  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:39.541109  8483 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:39.561745  8373 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:39.638273  1635 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:39.562148  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:39.646130  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 84064.4us
I20260709 13:02:39.650130  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:39.540536  8460 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:02:39.675655  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:39.627934  8214 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:39.684460  8227 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.001s
I20260709 13:02:39.627131  8245 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:39.685014  8227 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:39.687448  1558 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:39.688242  8428 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:39.688668  8428 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.688867  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2501) took 1407 ms. Trace:
I20260709 13:02:39.689008  8428 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:39.631762  8383 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.000s
I20260709 13:02:39.690357  8383 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:39.692726  8383 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:39.693500  8322 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.694161  8322 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:39.629730  8506 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 13:02:39.629123  8510 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:02:39.689246  1558 rpcz_store.cc:276] 0709 13:02:38.281271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.083267 (+801996us) service_pool.cc:224] Handling call
0709 13:02:39.083522 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.687844 (+604322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.688626 (+   782us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b5400206f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:39.688807 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1387632512}
I20260709 13:02:39.635339  8461 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:39.571048  8379 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.091s
W20260709 13:02:39.724488  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48927.5us
W20260709 13:02:39.628471  8507 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:02:39.622753  8487 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:39.737488  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.074s	sys 0.144s
W20260709 13:02:39.737907  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.074s	sys 0.144s
W20260709 13:02:39.738205  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 587684us
I20260709 13:02:39.751947  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:39.753515  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:39.697351  8322 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:39.560123  8441 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.734086  8404 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:39.627712  8517 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.119s
I20260709 13:02:39.738407  8461 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:39.823505  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 293589us
W20260709 13:02:39.817938  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 437979us
I20260709 13:02:39.584775  8500 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.635500  7929 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.614279  8336 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:39.846598  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:39.844290  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:39.847276  1705 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:39.872747  1703 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.586s	user 0.008s	sys 0.000s
I20260709 13:02:39.873291  1703 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:39.878232  1602 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:39.823575  8404 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.880028  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:39.903860  2346 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:39.838574  8461 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:39.903555  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1393) took 1314 ms. Trace:
I20260709 13:02:39.905467  1602 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:39.908819  2347 rpcz_store.cc:276] 0709 13:02:38.589511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.801904 (+212393us) service_pool.cc:224] Handling call
0709 13:02:38.842288 (+ 40384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.842627 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.842655 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:38.846253 (+  3598us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.157694 (+311441us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:39.157733 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.157804 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.903247 (+745443us) raft_consensus.cc:1685] finished
0709 13:02:39.903273 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.903460 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26108,"raft.run_cpu_time_us":51,"raft.run_wall_time_us":51,"thread_start_us":306493,"threads_started":1,"wal-append.queue_time_us":440465}
W20260709 13:02:39.879357  8513 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:02:39.912653  1706 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:39.864850  8379 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:39.923108  1705 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:39.932811  8508 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:39.934645  8474 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:39.936553  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1207) took 1043 ms. Trace:
I20260709 13:02:39.877918  8487 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.903534  8404 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:39.838306  8456 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:02:39.891897  8214 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.948619  2353 rpcz_store.cc:276] 0709 13:02:38.893168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.055190 (+162022us) service_pool.cc:224] Handling call
0709 13:02:39.058897 (+  3707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.062087 (+  3190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.062130 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.072238 (+ 10108us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.309246 (+237008us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:39.309287 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.309352 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.936274 (+626922us) raft_consensus.cc:1685] finished
0709 13:02:39.936311 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.936477 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6729,"raft.queue_time_us":3244,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":5357,"thread_start_us":215430,"threads_started":1,"wal-append.queue_time_us":515658}
W20260709 13:02:39.951153  1555 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.004s	sys 0.017s
I20260709 13:02:39.903846  8245 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:39.962189  1555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.004s	sys 0.017s
I20260709 13:02:39.842511  8522 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.126s
I20260709 13:02:39.992336  8533 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:39.992548  8456 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:39.993613  8525 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.995860  8098 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.025893  1704 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.002s	sys 0.015s
W20260709 13:02:40.026710  1704 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.002s	sys 0.015s
W20260709 13:02:40.027722  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.030473  1706 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:02:40.032832  1701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.008s	sys 0.013s
W20260709 13:02:40.034046  1701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.009s	sys 0.013s
W20260709 13:02:40.035403  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.984754  8523 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:39.956135  8487 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:39.955889  8379 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:39.907428  8494 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:39.977881  8214 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.033721  8517 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:40.054652  8356 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.993s	user 0.013s	sys 0.000s
W20260709 13:02:40.058220  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.060479  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 299685us
I20260709 13:02:40.071364  1674 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:40.051051  8523 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:40.047253  8387 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.011s	sys 0.030s
W20260709 13:02:40.078657  8387 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.012s	sys 0.030s
I20260709 13:02:40.029361  1525 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:40.080726  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51483.1us
I20260709 13:02:40.081820  1525 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:40.083110  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:40.084631  8383 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:40.042809  8527 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.085965  8527 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.088606  8527 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:39.987893  8245 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.091342  8245 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.022768  8460 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
W20260709 13:02:40.096161  8464 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.686s	user 0.005s	sys 0.005s
I20260709 13:02:40.096711  8464 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.057173  8440 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.098609  8460 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:40.061285  8532 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:02:40.032538  8098 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:40.100806  8497 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.002s	user 0.010s	sys 0.004s
I20260709 13:02:40.065059  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:40.115029  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.117671  1527 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:40.120486  8474 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.127784  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 358840us
I20260709 13:02:40.062075  8227 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:40.150370  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 85334us
I20260709 13:02:40.134204  8473 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:40.154670  8552 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.154889  8341 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:40.155251  8552 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.155745  8341 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:40.156235  8341 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.156602  8454 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.156828  8341 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:40.034194  8525 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.160935  8552 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.141036  8531 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:40.165346  8552 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:40.104604  8523 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:40.180382  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 271056us
I20260709 13:02:40.138971  8440 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:40.159389  8370 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:40.196847  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:40.157166  8454 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:40.205681  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.151648  8474 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.174250  8227 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:40.226691  8530 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:02:40.230165  8534 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:40.232228  8461 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.233572  8227 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.234045  8227 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:40.237215  8525 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.172981  8552 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.239965  8552 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.242497  8461 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.242574  8098 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.243287  8533 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:40.245002  8533 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:40.245857  8533 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.247066  8533 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:40.247298  8404 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:40.248437  8439 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.640s	user 0.010s	sys 0.000s
I20260709 13:02:40.250689  8214 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.252744  8379 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.243904  8227 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.255414  7929 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.154830  8473 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.256613  8227 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.259039  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1585) took 1374 ms. Trace:
I20260709 13:02:40.259573  2346 rpcz_store.cc:276] 0709 13:02:38.884567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.043169 (+158602us) service_pool.cc:224] Handling call
0709 13:02:39.903637 (+860468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.904474 (+   837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.258777 (+354303us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b5400643a50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:40.258957 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1959370496}
W20260709 13:02:40.264901  8428 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.575s	user 0.011s	sys 0.002s
I20260709 13:02:40.267431  8025 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:40.267976  8025 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:40.203352  8370 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:40.269984  8370 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:40.192216  8547 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:40.276264  8547 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:40.277158  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:40.277992  1556 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:40.280383  8524 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:02:40.282260  8464 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.281831  1456 catalog_manager.cc:5697] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 14, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 14 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:40.240873  8545 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:40.283573  1456 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 16, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:40.287334  8454 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.287741  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2476) took 1136 ms. Trace:
I20260709 13:02:40.288121  8454 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.284997  1456 catalog_manager.cc:5697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 14, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 14 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:40.289023  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:40.287191  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:40.290501  1706 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:40.288290  1701 rpcz_store.cc:276] 0709 13:02:39.153020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.281612 (+128592us) service_pool.cc:224] Handling call
0709 13:02:39.282072 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.286378 (+  4306us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:39.286432 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.311058 (+ 24626us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.035045 (+723987us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:40.035100 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.035187 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.287532 (+252345us) raft_consensus.cc:1685] finished
0709 13:02:40.287552 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.287672 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":202621,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":637695,"threads_started":1,"wal-append.queue_time_us":955053}
I20260709 13:02:40.287261  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:40.287791  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:40.299232  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.303031  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:40.311002  1456 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 10 to 13, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 13 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:40.312017  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:40.277205  8547 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.324219  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1203) took 1757 ms. Trace:
I20260709 13:02:40.374859  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:40.311292  8401 consensus_meta.cc:287] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.649s	user 0.007s	sys 0.004s
W20260709 13:02:40.393530  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 196560us
I20260709 13:02:40.345466  2348 rpcz_store.cc:276] 0709 13:02:38.566410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.566859 (+   449us) service_pool.cc:224] Handling call
0709 13:02:38.581292 (+ 14433us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:38.851485 (+270193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.851582 (+    97us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:39.477867 (+626285us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b2c0001d508. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:02:39.478594 (+   727us) log.cc:844] Serialized 52 byte log entry
0709 13:02:39.892554 (+413960us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:40.129566 (+237012us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.133994 (+  4428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.141887 (+  7893us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.302374 (+160487us) raft_consensus.cc:1685] finished
0709 13:02:40.323852 (+ 21478us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.324124 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18415,"prepare.queue_time_us":336543,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":113570,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":22992,"spinlock_wait_cycles":475132288,"thread_start_us":782744,"threads_started":3,"wal-append.queue_time_us":530918}
W20260709 13:02:40.399677  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.400740  1601 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:40.255528  8487 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:40.398612  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 50146.9us
I20260709 13:02:40.430209  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:40.249006  8439 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:40.435111  1456 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:40.338419  8552 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:40.249390  8404 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.269002  8025 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:40.461923  1552 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.023s	sys 0.043s
W20260709 13:02:40.462314  1602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.014s	sys 0.010s
W20260709 13:02:40.462661  1602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.014s	sys 0.010s
W20260709 13:02:40.463192  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 557448us
I20260709 13:02:40.463428  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:40.425815  8106 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.057s	sys 0.071s
I20260709 13:02:40.471459  1456 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:40.472417  1602 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:40.462319  1552 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.023s	sys 0.043s
I20260709 13:02:40.449949  8533 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.474608  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:40.475736  1702 raft_consensus.cc:2368] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16.
I20260709 13:02:40.289796  8454 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:40.492573  8500 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.033s	sys 0.037s
I20260709 13:02:40.465917  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:40.502193  8533 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.503038  1602 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:40.503535  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3242) took 1349 ms. Trace:
I20260709 13:02:40.289820  8428 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.504659  8533 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.506810  8428 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:40.503998  1553 rpcz_store.cc:276] 0709 13:02:39.153788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.486250 (+332462us) service_pool.cc:224] Handling call
0709 13:02:39.526447 (+ 40197us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.526905 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.526944 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.527530 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.061689 (+534159us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:40.061737 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.061806 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.503342 (+441536us) raft_consensus.cc:1685] finished
0709 13:02:40.503363 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.503480 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":450136,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":31856,"thread_start_us":441133,"threads_started":1,"wal-append.queue_time_us":915918}
I20260709 13:02:40.488394  1456 catalog_manager.cc:5697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:40.484433  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 1066) took 1441 ms. Trace:
I20260709 13:02:40.508903  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.509011  1552 rpcz_store.cc:276] 0709 13:02:39.043347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.267754 (+224407us) service_pool.cc:224] Handling call
0709 13:02:39.268070 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.268268 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:39.467724 (+199456us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.480163 (+1012439us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.480232 (+    69us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:40.480313 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.484075 (+  3762us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.484355 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24534,"prepare.queue_time_us":400754,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":242506,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":103,"thread_start_us":1192027,"threads_started":3,"wal-append.queue_time_us":599015}
I20260709 13:02:40.509572  1456 catalog_manager.cc:5697] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 8 to 14, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 14 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:40.510973  1553 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:40.511113  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.501524  8564 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:40.511520  1456 catalog_manager.cc:5697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 13, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:40.512383  8564 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.515287  8564 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.515921  8564 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:40.516494  8564 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.400231  8483 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.078s
I20260709 13:02:40.513250  1456 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 14, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:40.518873  8483 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:40.435050  8055 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:02:40.503301  8500 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.034s	sys 0.038s
W20260709 13:02:40.465389  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.519500  1456 catalog_manager.cc:5697] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 10 to 14, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 14 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:40.509825  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3239) took 1374 ms. Trace:
I20260709 13:02:40.521859  1555 rpcz_store.cc:276] 0709 13:02:39.135639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.258660 (+123021us) service_pool.cc:224] Handling call
0709 13:02:39.259053 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.259483 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.259528 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.286147 (+ 26619us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.973832 (+687685us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:39.973893 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.974005 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.509561 (+535556us) raft_consensus.cc:1685] finished
0709 13:02:40.509589 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.509756 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7866,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":11528,"thread_start_us":579792,"threads_started":1,"wal-append.queue_time_us":1182923}
I20260709 13:02:40.519536  8483 raft_consensus.cc:697] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:40.524859  8483 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.500172  8025 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.533679  8552 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:40.450717  7929 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:40.535511  7929 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.536057  1701 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:02:40.536505  1701 pending_rounds.cc:85] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 13
I20260709 13:02:40.536741  1701 pending_rounds.cc:107] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:02:40.536674  8337 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.537209  1701 raft_consensus.cc:2889] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:40.449879  8474 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:40.460224  8535 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.461154  8487 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.452781  8508 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:02:40.443149  8464 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:02:40.436811  8401 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.539881  1702 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:40.550794  1709 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:40.521086  1456 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:40.553572  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 470223us
W20260709 13:02:40.553809  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50833.2us
W20260709 13:02:40.560871  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 489260us
I20260709 13:02:40.541733  8500 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.578051  1559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.012s	sys 0.012s
W20260709 13:02:40.578507  1559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.012s	sys 0.012s
W20260709 13:02:40.582484  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.583359  8549 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.583966  8549 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.584583  1630 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:40.585656  8214 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.586337  8214 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.586359  8549 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:40.587617  8549 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.588162  8549 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.590669  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:40.591560  1627 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:40.601872  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:40.602725  1629 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
W20260709 13:02:40.599021  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 44290.1us
I20260709 13:02:40.552490  7929 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.592782  8227 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:40.615432  8541 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.638592  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:40.639398  1627 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.597529  8336 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:02:40.594854  8055 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:40.599934  8379 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:40.647054  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:40.651525  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:40.652135  2353 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:40.589319  8214 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.545866  8535 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.656353  8556 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:40.658313  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:40.661696  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 231439us
I20260709 13:02:40.654816  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.608669  1456 catalog_manager.cc:5697] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 15, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), pending config changed. New cstate: current_term: 15 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:40.663851  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:40.548556  8487 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:40.664356  1602 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:40.664623  2348 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 13: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:40.664995  1556 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:40.662730  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:40.668818  1555 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:40.665150  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:40.667239  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:40.643463  8541 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.669651  1456 catalog_manager.cc:5697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 16, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:40.670728  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:40.671288  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.673580  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:40.672832  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.675168  1632 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
W20260709 13:02:40.682670  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.681248  2353 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:40.674149  2317 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:40.676595  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.686313  1556 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:40.600301  8500 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.687471  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.692148  1602 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:40.657614  8545 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.690011  1455 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 274) took 44575 ms (client timeout 15000 ms). Trace:
I20260709 13:02:40.651700  8546 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:40.694149  1456 catalog_manager.cc:5697] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 14, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 14 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:40.692746  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:40.667250  8548 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.697000  8439 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.707937  1627 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
W20260709 13:02:40.714870  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 122197us
I20260709 13:02:40.701999  8461 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:40.729161  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2477) took 1575 ms. Trace:
I20260709 13:02:40.690523  8521 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:40.726233  1602 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:40.687263  8497 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:02:40.751515  1525 leader_election.cc:336] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.554s (SENT)
I20260709 13:02:40.685451  2318 leader_election.cc:400] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16.
I20260709 13:02:40.656983  8556 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.763803  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:40.727460  8549 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.708629  8562 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.724671  8412 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.682219  8532 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:02:40.754537  8521 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:40.759127  2318 leader_election.cc:403] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:02:40.690109  8522 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:40.682346  8214 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:02:40.775265  8563 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:40.691082  8554 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:02:40.771703  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:40.738963  8548 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.709736  8508 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:02:40.735764  8464 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:40.736879  8336 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:40.776413  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.838344  1558 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:40.839318  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:40.839829  8567 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:02:40.840121  1558 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:40.841073  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.774534  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:40.843580  8523 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.000s
I20260709 13:02:40.754264  8337 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:40.753948  1704 rpcz_store.cc:276] 0709 13:02:39.153280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.278496 (+125216us) service_pool.cc:224] Handling call
0709 13:02:39.278827 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.486326 (+207499us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:40.027297 (+540971us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.216411 (+189114us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.216482 (+    71us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:40.216539 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.728893 (+512354us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400077e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:40.728948 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.729095 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":650253,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":4263,"raft.queue_time_us":10228,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":674512000,"thread_start_us":721136,"threads_started":2,"wal-append.queue_time_us":533023}
I20260709 13:02:40.749982  8564 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:40.769968  2349 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.009s	sys 0.010s
W20260709 13:02:40.850793  2349 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.009s	sys 0.010s
I20260709 13:02:40.737534  8533 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.801366  8494 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.723117  8545 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.763443  8561 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:40.757093  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:40.772634  8483 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:40.750173  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.777901  8518 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:02:40.816421  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 130922us
I20260709 13:02:40.776392  8556 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.735464  8055 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.799788  8535 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:40.832387  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 139618us
I20260709 13:02:40.749663  8546 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.805176  8497 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:40.776818  8500 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:40.837160  8336 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.842878  8401 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.843962  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:40.710316  1455 rpcz_store.cc:269] 0709 13:01:56.006412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.006716 (+   304us) service_pool.cc:224] Handling call
0709 13:01:57.182744 (+1176028us) spinlock_profiling.cc:243] Waited 94 ms on lock 0x7b5400179b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27644090 00007fda276e2f3a 00007fda2d4991cc 00007fda2d59737e 00007fda2ba582a4 00007fda2ba58232 00007fda2ba58161 00007fda2ba580dc 00007fda2ba57392 00007fda25a1548c 00007fda25a148c6
0709 13:02:40.559324 (+43376580us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:26.265062 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:26.347996 (+ 82934us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:26.348052 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:02:26.348074 (+    22us) tablet.cc:662] Decoding operations
0709 13:02:26.409765 (+ 61691us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:26.409873 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:26.409904 (+    31us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:02:26.415443 (+  5539us) tablet.cc:701] Row locks acquired
0709 13:02:26.415499 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:02:26.415743 (+   244us) write_op.cc:270] Start()
0709 13:02:26.415919 (+   176us) write_op.cc:276] Timestamp: P: 1783602146415658 usec, L: 0
0709 13:02:26.415961 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:02:26.416948 (+   987us) log.cc:844] Serialized 6705 byte log entry
0709 13:02:26.956273 (+539325us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda245d00ac 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b
0709 13:02:27.552760 (+596487us) op_driver.cc:464] REPLICATION: finished
0709 13:02:27.800059 (+247299us) write_op.cc:301] APPLY: starting
0709 13:02:27.800224 (+   165us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:28.044774 (+244550us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:28.116575 (+ 71801us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:28.829986 (+713411us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b18000d6420. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda27419129 00007fda27465c0a 00007fda274654f7 00007fda27465490 00007fda2741cf48 00007fda2b1311bb 00007fda2b1785d4 00007fda2b0248da 00007fda2b0273cb
0709 13:02:29.163745 (+333759us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:29.164212 (+   467us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 13:02:29.164280 (+    68us) write_op.cc:312] APPLY: finished
0709 13:02:29.164751 (+   471us) log.cc:844] Serialized 156 byte log entry
0709 13:02:29.269272 (+104521us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:29.269669 (+   397us) write_op.cc:454] Released schema lock
0709 13:02:29.269764 (+    95us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:29.270351 (+   587us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:29.270388 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1254611,"reported_tablets":48,"spinlock_wait_cycles":216275712,"child_traces":[["op",{"apply.queue_time_us":236304,"apply.run_cpu_time_us":41587,"apply.run_wall_time_us":1473375,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75187,"num_ops":17,"prepare.queue_time_us":329635,"prepare.run_cpu_time_us":17851,"prepare.run_wall_time_us":1047769,"raft.queue_time_us":356196,"raft.run_cpu_time_us":39105,"raft.run_wall_time_us":262209,"replication_time_us":1107858,"spinlock_wait_cycles":100932608,"thread_start_us":1350038,"threads_started":5,"wal-append.queue_time_us":832519,"wal-append.run_cpu_time_us":38297,"wal-append.run_wall_time_us":2290848}]]}
I20260709 13:02:40.858076  8508 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.815690  8522 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.844696  1558 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:40.861825  8518 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.880479  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.823524  8412 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:02:40.845346  8560 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.845798  8574 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:02:40.846235  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:40.866982  8561 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.887600  8560 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.888208  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:40.889343  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:40.781450  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:40.855422  8337 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:40.891232  8514 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.059s	user 0.000s	sys 0.009s
I20260709 13:02:40.816107  8521 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:40.880764  8522 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:40.860407  8533 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.861624  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:40.856297  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1587) took 1712 ms. Trace:
I20260709 13:02:40.911502  2353 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:40.856108  1552 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:02:40.867954  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:40.871390  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.810508  8562 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:40.913908  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46267.8us
I20260709 13:02:40.913937  1556 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:40.844306  8487 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.860610  8474 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:40.881150  1558 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:40.861035  8483 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:40.867733  8439 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:40.882809  8518 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:40.906162  1627 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:40.860575  8494 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:40.906814  1629 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:40.854486  8523 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.898083  8497 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:40.923274  8566 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:02:40.911479  2349 rpcz_store.cc:276] 0709 13:02:39.144167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.583942 (+439775us) service_pool.cc:224] Handling call
0709 13:02:39.631773 (+ 47831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.631990 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:40.102680 (+470690us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.855901 (+753221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.855958 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:40.856015 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.856115 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.856235 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4679,"prepare.queue_time_us":596859,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":5270,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":3573,"spinlock_wait_cycles":46848,"thread_start_us":1217370,"threads_started":2,"wal-append.queue_time_us":766660}
I20260709 13:02:40.879796  8541 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:40.907672  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 156095us
I20260709 13:02:40.929303  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:40.873127  8575 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:02:40.922253  8298 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:02:40.937466  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.847306  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 122836us
I20260709 13:02:40.879499  8564 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:02:40.938673  8564 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:40.938983  8336 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.939002  8564 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:40.939620  8337 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.940763  8398 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.940488  8564 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.941722  8531 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:40.941869  8564 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.942123  8548 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.944761  8546 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.945803  8545 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.946105  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:40.946843  1709 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:40.947751  8562 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:40.948799  8494 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.949353  1628 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:40.936175  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.942188  8531 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:40.950624  1556 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:02:40.951059  8386 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:40.951784  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:40.958168  8557 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 13:02:40.959311  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.959975  1556 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:40.960566  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.960809  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.952924  8561 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:40.953011  8577 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:02:40.962096  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:40.953629  8298 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:40.931136  1632 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:40.927119  8483 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.963529  8298 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:40.964389  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:40.931766  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:40.961385  1556 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:40.945827  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.020325  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:41.021379  1704 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:41.014824  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:41.022449  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:40.965428  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:40.974931  1702 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:40.962916  1561 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:40.998306  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 123901us
I20260709 13:02:41.024899  1555 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:41.022892  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.025005  1702 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:41.036196  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:41.037922  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.029349  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:41.029842  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:41.026789  2318 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:41.039165  1705 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:41.028707  1627 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:41.036118  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:41.029349  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.036259  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:41.041041  1707 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:41.046482  1709 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:41.047457  8560 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:41.048171  8570 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 13:02:41.049055  1556 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:41.049612  8025 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.000s
I20260709 13:02:41.051637  1710 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:02:41.052342  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:41.042089  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.054128  1627 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:41.058599  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.066450  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.038597  1702 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:41.056869  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:41.063935  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:41.055286  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:40.975072  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.067479  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:41.069072  1703 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:41.068073  1704 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:41.069429  1627 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:41.068969  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.070392  1708 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:41.070329  1630 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:41.068009  1702 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.972805  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:41.075557  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.073052  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:41.078720  1629 raft_consensus.cc:2393] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:41.079552  1705 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:41.081130  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.076134  1630 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:41.081971  1710 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:41.084432  1527 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:40.933498  8584 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.132s
I20260709 13:02:41.091100  1709 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:41.087507  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.092044  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.092779  1629 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:41.094019  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.096240  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.111259  1629 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:41.065650  8574 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:40.958171  8531 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:41.099741  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.057166  8590 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:41.103217  1561 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:02:41.090777  8564 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:41.114403  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:41.107126  8341 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.950s	user 0.014s	sys 0.000s
I20260709 13:02:41.122848  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:41.123216  1704 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
W20260709 13:02:41.124830  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.086081  8573 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:02:41.126497  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:41.312835  1702 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:41.589217  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.137279  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:41.593945  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.535450  8370 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.262s	user 0.003s	sys 0.013s
I20260709 13:02:41.594957  2347 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:41.597673  2348 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
W20260709 13:02:41.403364  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 311795us
I20260709 13:02:41.593313  1629 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:41.602273  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:41.216029  8589 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:02:41.126866  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.606986  1630 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:41.427559  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.608070  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:41.595379  8370 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:41.591041  1627 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.521s	user 0.033s	sys 0.040s
I20260709 13:02:41.614192  1627 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
W20260709 13:02:41.614459  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.615381  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.616567  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.617328  1627 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:41.619531  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:41.620357  1627 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:41.105867  8575 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:41.545193  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3372) took 2226 ms. Trace:
I20260709 13:02:41.626204  1631 rpcz_store.cc:276] 0709 13:02:39.318576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.318706 (+   130us) service_pool.cc:224] Handling call
0709 13:02:39.318972 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.456470 (+137498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.544828 (+2088358us) spinlock_profiling.cc:243] Waited 92.8 ms on lock 0x7b54003cd7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:41.545080 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213470592}
I20260709 13:02:41.596550  8341 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:41.317677  8594 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:02:41.609249  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:41.547277  8597 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.274s	user 0.000s	sys 0.000s
W20260709 13:02:41.281029  8559 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.653s	user 0.000s	sys 0.002s
I20260709 13:02:41.595875  8573 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:41.628324  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:41.220467  8591 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:02:41.221763  8538 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:02:41.219784  8596 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:02:41.272611  8595 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 13:02:41.443925  8551 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.811s	user 0.001s	sys 0.002s
W20260709 13:02:41.435197  8598 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 13:02:41.589504  8508 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.691s	user 0.005s	sys 0.007s
I20260709 13:02:41.655833  8508 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16.
I20260709 13:02:41.657385  8508 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:41.657913  8508 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:41.659770  8522 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.762s	user 0.006s	sys 0.002s
I20260709 13:02:41.659997  8508 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:41.660244  8522 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:41.546038  8512 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 2.021s	user 0.005s	sys 0.000s
I20260709 13:02:41.636803  1629 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:41.663653  1629 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:02:41.664002  1629 pending_rounds.cc:85] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 14
I20260709 13:02:41.664237  1629 pending_rounds.cc:107] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:02:41.664496  1629 raft_consensus.cc:2889] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:41.242058  8523 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:41.666005  8523 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:41.666514  8523 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:41.107048  8518 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:41.643788  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:41.630246  8564 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:41.667908  2353 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:41.668826  1707 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:41.668646  8564 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:41.644788  1631 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:41.632375  8575 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:41.197886  8592 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:41.324901  8483 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:41.166791  8561 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:41.617640  8373 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.672796  2346 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:02:41.681571  8387 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.672416  8561 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:41.682883  8381 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.683439  8381 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:41.684024  8561 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:41.682129  8387 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:41.687196  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:41.688768  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.687239  1705 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: [15.11-16.12]
I20260709 13:02:41.652335  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:41.631075  8409 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:41.668876  8590 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.502s	user 0.003s	sys 0.005s
I20260709 13:02:41.671384  8575 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:41.672510  8373 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:41.673468  8483 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:41.224844  8521 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:41.592235  8454 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:41.649384  1627 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:41.690573  1631 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:41.694057  1630 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:41.694356  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.697237  8387 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:41.700196  1556 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.015s	sys 0.028s
I20260709 13:02:41.703056  1705 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:02:41.731900  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:41.731247  1556 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.016s	sys 0.028s
I20260709 13:02:41.705695  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:41.734475  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:41.706424  1561 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.007s	sys 0.022s
W20260709 13:02:41.701231  8589 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:02:41.735704  1627 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:41.738221  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:41.737125  1561 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.007s	sys 0.022s
I20260709 13:02:41.733892  1631 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:41.697685  8383 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:02:41.701654  8500 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.010s	sys 0.039s
I20260709 13:02:41.739722  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3253) took 1083 ms. Trace:
I20260709 13:02:41.697882  8531 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:41.704317  8590 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:41.744901  8373 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:41.746809  8341 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:41.706669  8575 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:02:41.653008  8573 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:41.749795  8383 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:41.748833  1556 rpcz_store.cc:276] 0709 13:02:40.659441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.050095 (+390654us) service_pool.cc:224] Handling call
0709 13:02:41.050506 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.050705 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:41.190623 (+139918us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.734165 (+543542us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.739021 (+  4856us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:41.739074 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.739149 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.739657 (+   508us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14885,"raft.queue_time_us":21438,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":59,"thread_start_us":666982,"threads_started":2}
I20260709 13:02:41.754366  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.755358  8605 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:41.756000  8522 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:41.748183  8500 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.010s	sys 0.040s
I20260709 13:02:41.766647  8518 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:41.785822  1554 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.027s	sys 0.044s
I20260709 13:02:41.768049  1627 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:41.769404  8381 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:41.771178  8523 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:41.771659  8473 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:41.773912  8521 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:41.774412  8474 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:41.810397  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.775789  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3109) took 1012 ms. Trace:
I20260709 13:02:41.811691  1628 rpcz_store.cc:276] 0709 13:02:40.763338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.766008 (+  2670us) service_pool.cc:224] Handling call
0709 13:02:41.775520 (+1009512us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540010e0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:41.775680 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282405632}
I20260709 13:02:41.727216  8370 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:41.787551  1554 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.028s	sys 0.044s
I20260709 13:02:41.808984  1703 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:41.820575  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:41.822662  8523 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:41.823386  8523 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:41.782919  8456 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.815691  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:41.834174  1556 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:41.835016   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:41.836313  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:41.847551  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:41.848297  1676 connection.cc:256] RPC call timeout handler was delayed by -1.1566907430000128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2687s for a response
I20260709 13:02:41.813627  8474 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:41.849107  1676 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:41.856912  1705 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:41.854071  2346 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:02:41.859338  2346 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 12
I20260709 13:02:41.859587  2346 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 17.13
I20260709 13:02:41.859897  2346 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Aborting config change with OpId 17.13: Aborted: Op aborted by new leader
I20260709 13:02:41.860349  2346 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:41.863679  1676 leader_election.cc:400] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17.
I20260709 13:02:41.864024  1676 leader_election.cc:403] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:41.865177  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:41.848253  8590 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:02:41.858220  2317 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.097s (SENT) [suppressed 14 similar messages]
I20260709 13:02:41.894764  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:41.834379  8456 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:41.897861  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:41.902582  1556 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:41.903119  2351 raft_consensus.cc:2410] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:02:41.900072  2347 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:41.856506  8593 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.015s	sys 0.049s
I20260709 13:02:41.874028  8522 raft_consensus.cc:2764] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:02:41.862579  8508 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:41.901698  8582 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:02:41.896660  8603 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 13:02:41.896801  8614 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:02:41.902031  8590 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:41.875447  8381 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.900689  8298 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17.
W20260709 13:02:41.901144  8597 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:41.852758  8518 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.993767  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 134366us
W20260709 13:02:41.914361  8593 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.016s	sys 0.049s
W20260709 13:02:41.999143  8601 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.136s	user 0.001s	sys 0.003s
I20260709 13:02:42.010958  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:41.999812  8602 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.003s
I20260709 13:02:42.010982  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:41.992630  8508 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:41.994575  8554 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.020066  2347 raft_consensus.cc:2410] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:02:42.020895  8383 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:42.029008  1708 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:42.035989  2348 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:42.038573  1704 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:02:42.034147  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 411236us
I20260709 13:02:42.040786  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:42.041011  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:42.041940  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:42.044339  1556 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:02:42.021452  8508 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:42.046963  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:42.057029  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:42.059088  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:42.061206  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3254) took 1401 ms. Trace:
I20260709 13:02:42.053458  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:42.070735  1556 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.071841  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:42.011709  8381 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.017868  8518 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.094605  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3252) took 1438 ms. Trace:
I20260709 13:02:42.095192  1555 rpcz_store.cc:276] 0709 13:02:40.656171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.039327 (+383156us) service_pool.cc:224] Handling call
0709 13:02:41.039665 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.040119 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:41.112506 (+ 72387us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.124871 (+ 12365us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.125467 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.125631 (+   164us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:41.125659 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.125720 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.094380 (+968660us) raft_consensus.cc:1685] finished
0709 13:02:42.094401 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.094539 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":746083,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":1627,"raft.queue_time_us":462,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":15701,"thread_start_us":58950,"threads_started":2,"wal-append.queue_time_us":669242}
I20260709 13:02:42.098237  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:42.107788  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:42.109040  1555 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:42.061430  8341 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:42.086037  8373 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:42.060091  8592 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:42.087445  8387 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:42.036221  8554 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.102887  8573 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:42.055292  8590 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.115620  8341 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.167814  8341 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.119544  8387 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:42.118680  8300 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:02:42.148365  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 1063) took 3282 ms. Trace:
W20260709 13:02:42.132028  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 273692us
W20260709 13:02:42.148742  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.138371  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 1067) took 3094 ms. Trace:
I20260709 13:02:42.139053  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:42.189520  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:42.153432  1560 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:42.133565  1556 raft_consensus.cc:2410] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:42.191310  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:42.192363  1702 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 13.
I20260709 13:02:42.193105  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:42.193642  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:42.194413  1702 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:42.138438  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3255) took 1477 ms. Trace:
I20260709 13:02:42.195086  1552 rpcz_store.cc:276] 0709 13:02:40.659916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.547069 (+887153us) service_pool.cc:224] Handling call
0709 13:02:41.592641 (+ 45572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.617791 (+ 25150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.617822 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.621579 (+  3757us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.914700 (+293121us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:41.914776 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.914862 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.137203 (+222341us) raft_consensus.cc:1685] finished
0709 13:02:42.137238 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.137445 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24697,"raft.queue_time_us":25525,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":8766,"thread_start_us":215025,"threads_started":1,"wal-append.queue_time_us":474427}
I20260709 13:02:42.194489  1556 raft_consensus.cc:2410] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:02:42.183178  1557 rpcz_store.cc:276] 0709 13:02:38.865935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.234400 (+368465us) service_pool.cc:224] Handling call
0709 13:02:39.274693 (+ 40293us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:39.296580 (+ 21887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.296613 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:39.370114 (+ 73501us) log.cc:844] Serialized 52 byte log entry
0709 13:02:39.761996 (+391882us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:39.763454 (+  1458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.763953 (+   499us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.995602 (+231649us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b4800002958. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:02:40.281548 (+285946us) raft_consensus.cc:1685] finished
0709 13:02:40.281568 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.148185 (+1866617us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b54001ce150. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:42.148301 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":4693,"raft.run_wall_time_us":712386,"spinlock_wait_cycles":96693760,"thread_start_us":295103,"threads_started":1,"wal-append.queue_time_us":705443,"wal-append.run_cpu_time_us":3150,"wal-append.run_wall_time_us":1264331}
I20260709 13:02:42.149362  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 1071) took 1876 ms. Trace:
I20260709 13:02:42.201982  1554 rpcz_store.cc:276] 0709 13:02:40.272542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.278929 (+  6387us) service_pool.cc:224] Handling call
0709 13:02:40.279349 (+   420us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.957182 (+677833us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.957216 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.957863 (+   647us) log.cc:844] Serialized 52 byte log entry
0709 13:02:41.113658 (+155795us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:02:41.816301 (+702643us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.828539 (+ 12238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.148863 (+320324us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b5400230d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:42.148936 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.149002 (+    66us) raft_consensus.cc:1685] finished
0709 13:02:42.149118 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.149293 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":763698,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":8431,"raft.queue_time_us":16759,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":17323,"spinlock_wait_cycles":1556949376,"thread_start_us":833904,"threads_started":2,"wal-append.queue_time_us":232947}
I20260709 13:02:42.206552  1556 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:42.188364  1559 rpcz_store.cc:276] 0709 13:02:39.043607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.268780 (+225173us) service_pool.cc:224] Handling call
0709 13:02:39.275748 (+  6968us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.595099 (+319351us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:02:40.053097 (+457998us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.578957 (+525860us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.579024 (+    67us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:40.579089 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.138044 (+1558955us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b540038a9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:42.138118 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.138272 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4711,"prepare.queue_time_us":106709,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":34,"raft.queue_time_us":3263,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":9820,"spinlock_wait_cycles":733329280,"thread_start_us":917044,"threads_started":2,"wal-append.queue_time_us":628997,"wal-append.run_cpu_time_us":2693,"wal-append.run_wall_time_us":1305113}
I20260709 13:02:42.209558  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2513) took 1482 ms. Trace:
I20260709 13:02:42.210083  1556 rpcz_store.cc:276] 0709 13:02:40.726770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.205592 (+1478822us) service_pool.cc:224] Handling call
0709 13:02:42.206018 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.207137 (+  1119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.209470 (+  2333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:42.212154  8573 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:42.132055  8616 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:02:42.204800  8620 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:42.221863  8456 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:42.133411  8523 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:42.224012  8381 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:42.160557  8621 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:02:42.138756  8592 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.226697  8621 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.227205  8621 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.200702  8387 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:42.147742  8474 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:42.114454  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50592.4us
I20260709 13:02:42.201529  8526 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.212764  8573 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.233022  1559 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:42.243458  2345 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:42.243788  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.242506  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2510) took 1551 ms. Trace:
W20260709 13:02:42.244647  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.244656  1557 rpcz_store.cc:276] 0709 13:02:40.690796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.206304 (+1515508us) service_pool.cc:224] Handling call
0709 13:02:42.241790 (+ 35486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.242116 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.242143 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.242188 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:42.242230 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.242291 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.242443 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:42.235594  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:42.235538  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:42.230245  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3249) took 1575 ms. Trace:
W20260709 13:02:42.242475  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.246441  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:42.242264  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:42.246636  1553 rpcz_store.cc:276] 0709 13:02:40.654735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.655262 (+   527us) service_pool.cc:224] Handling call
0709 13:02:40.671844 (+ 16582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.774970 (+103126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.775008 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.775049 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:40.775086 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.229995 (+1454909us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54001c0054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:42.230044 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.230181 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236544768}
I20260709 13:02:42.243868  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3262) took 1438 ms. Trace:
I20260709 13:02:42.248941  1559 rpcz_store.cc:276] 0709 13:02:40.805832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.232376 (+1426544us) service_pool.cc:224] Handling call
0709 13:02:42.232821 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.243599 (+ 10778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.243809 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:42.234556  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3388) took 1650 ms. Trace:
I20260709 13:02:42.254720  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:42.255580  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 1.31929e+06us
W20260709 13:02:42.256335  1525 leader_election.cc:336] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.596s (SENT)
I20260709 13:02:42.256708  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:42.256949  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:42.256866  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3402) took 1165 ms. Trace:
I20260709 13:02:42.261716  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:42.261638  2349 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:42.273418  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:42.264986  1555 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:42.277523  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:42.250963  1635 rpcz_store.cc:276] 0709 13:02:40.584195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.603746 (+ 19551us) service_pool.cc:224] Handling call
0709 13:02:40.627011 (+ 23265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.627287 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:40.761743 (+134456us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.057900 (+296157us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.225963 (+1168063us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b18000d9fc0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:42.229165 (+  3202us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b18000d9fc0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:02:42.229341 (+   176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.229393 (+    52us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:42.229469 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.234314 (+  4845us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.234479 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":231531,"prepare.queue_time_us":248080,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":92,"raft.queue_time_us":1377767,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":14241,"spinlock_wait_cycles":36295680,"thread_start_us":203020,"threads_started":2,"wal-append.queue_time_us":169071,"wal-append.run_cpu_time_us":2015,"wal-append.run_wall_time_us":1194336}
I20260709 13:02:42.230189  8518 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:42.232285  8474 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:42.229429  8621 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:42.237869  8494 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:42.284723  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:42.270088  1702 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:42.245229  8373 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:42.250640  8300 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.292086  8300 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.292835  8494 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.291402  8227 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:42.249060  8383 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:42.259912  8456 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:42.316979  8456 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.317526  8456 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.317849  1702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:42.323676  1702 raft_consensus.cc:606] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:42.280455  8461 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:42.328023  1702 raft_consensus.cc:614] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:42.274600  8312 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:42.254536  8508 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.294579  8300 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:42.345086  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 63453.6us
I20260709 13:02:42.353368  8494 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:42.356240  8454 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:42.318735  8383 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.362295  8383 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:42.353662  1552 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 13:02:42.343513  8461 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:42.311128  8227 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:42.284060  8616 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:42.373898  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2492) took 1335 ms. Trace:
I20260709 13:02:42.374483  1707 rpcz_store.cc:276] 0709 13:02:41.038452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.042216 (+  3764us) service_pool.cc:224] Handling call
0709 13:02:42.373643 (+1331427us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b5400247054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:42.373829 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1364991744}
I20260709 13:02:42.340278  8633 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:42.321646  8631 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:42.373652  8616 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.376354  8631 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:42.376833  8631 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:42.379787  2353 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 1404) took 1408 ms (client timeout 1604 ms). Trace:
W20260709 13:02:42.380270  2353 rpcz_store.cc:269] 0709 13:02:40.971411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.972915 (+  1504us) service_pool.cc:224] Handling call
0709 13:02:42.379606 (+1406691us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b5400550354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:42.379723 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457115008}
I20260709 13:02:42.277717  8523 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:02:42.284876  8464 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.397785  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:42.398787  1710 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:42.331037  8621 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:42.407652  8527 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:42.397509  8494 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.437742  8441 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:42.438184  8494 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.439699  8341 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.440503  8494 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:42.440846  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:42.440533  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3130) took 1602 ms. Trace:
I20260709 13:02:42.442101  1702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:42.442612  8554 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:42.420437  8625 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:02:42.261538  1631 rpcz_store.cc:276] 0709 13:02:41.091366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.784728 (+693362us) service_pool.cc:224] Handling call
0709 13:02:41.785203 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.789605 (+  4402us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:41.814064 (+ 24459us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.256324 (+442260us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.256394 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:42.256506 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.256610 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.256781 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":440502,"prepare.queue_time_us":25189,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":444111}
I20260709 13:02:42.422437  8621 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:42.254185  8526 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.442965  1702 raft_consensus.cc:606] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:42.442574  1636 rpcz_store.cc:276] 0709 13:02:40.838271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.662928 (+824657us) service_pool.cc:224] Handling call
0709 13:02:42.439626 (+776698us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b5800135698. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5eee 0000000000375939 00007fda2c6291e0 00007fda2c6f20c4 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:02:42.439809 (+   183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.440125 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.440155 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.440201 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:42.440275 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.440341 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.440456 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1005,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":81,"spinlock_wait_cycles":25071104}
I20260709 13:02:42.462623  1702 raft_consensus.cc:614] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:42.481669  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:42.486439  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:42.488226  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:42.489584  1702 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:02:42.494647  1710 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:42.462741  8621 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:42.497552  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:42.493754  8055 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:42.396538  8464 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.535946  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:42.392836  8373 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:42.557965  8629 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:02:42.534965  8560 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:42.560251  8601 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:02:42.611348  8639 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:02:42.622103  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.623040  1631 raft_consensus.cc:2410] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:02:42.618921  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:42.619933  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 335100us
W20260709 13:02:42.625874  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 275935us
I20260709 13:02:42.626621  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:42.626812  1527 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:42.620395  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 228399us
I20260709 13:02:42.538061  8055 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.650215  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:42.623858  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:42.657995  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:42.662494  1525 connection.cc:256] RPC call timeout handler was delayed by -1.5196529539999801s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2033s for a response
W20260709 13:02:42.664361  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.664744  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:42.665889  1705 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:42.666301  1555 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:42.667229  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:42.667896  1555 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:02:42.668649  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:42.668939  1555 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:42.667943  1556 raft_consensus.cc:2410] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:42.669267  2344 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:42.666304  1525 connection.cc:256] RPC call timeout handler was delayed by -1.4693818559999841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
I20260709 13:02:42.663400  8370 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:42.672371  8370 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.672775  8370 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:42.673866  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:42.674149  8633 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:42.674547  8633 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:42.670279  1525 connection.cc:256] RPC call timeout handler was delayed by -1.3693529539999645s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
W20260709 13:02:42.675074  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51283us
I20260709 13:02:42.675616  1628 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:02:42.676126  1525 connection.cc:256] RPC call timeout handler was delayed by -1.1173097279999524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
W20260709 13:02:42.676532  1525 connection.cc:256] RPC call timeout handler was delayed by -1.0678097279999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
I20260709 13:02:42.679394  2344 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:42.679318  1635 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:42.674577  1631 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:42.682677  8536 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:42.683200  8536 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.683526  8536 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.675838  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:42.696861  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:42.650764  8573 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:42.704560  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:42.714466  1555 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:42.716285  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2500) took 1603 ms. Trace:
I20260709 13:02:42.716572  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3391) took 2131 ms. Trace:
I20260709 13:02:42.716944  1709 rpcz_store.cc:276] 0709 13:02:41.112849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.112982 (+   133us) service_pool.cc:224] Handling call
0709 13:02:41.123146 (+ 10164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.123359 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:41.126405 (+  3046us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.485938 (+1359533us) spinlock_profiling.cc:243] Waited 5.21 ms on lock 0x7b18002a49c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda2b0d23c7 00007fda276e49a7 00007fda276f40bf 00007fda276f4038
0709 13:02:42.486247 (+   309us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.558989 (+ 72742us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b18002a49c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1
0709 13:02:42.715725 (+156736us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.715815 (+    90us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:42.715902 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.716002 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.716180 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":675106,"prepare.queue_time_us":13870,"raft.queue_time_us":10733,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":24149,"spinlock_wait_cycles":40435072}
I20260709 13:02:42.717193  1633 rpcz_store.cc:276] 0709 13:02:40.585249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.592106 (+  6857us) service_pool.cc:224] Handling call
0709 13:02:40.617031 (+ 24925us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.648005 (+ 30974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.648035 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.663914 (+ 15879us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.378010 (+1714096us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b18002cb420. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:02:42.492115 (+114105us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:42.492180 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.492299 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.689515 (+197216us) raft_consensus.cc:1685] finished
0709 13:02:42.689551 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.716468 (+ 26917us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43047,"raft.queue_time_us":46379,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":8144,"spinlock_wait_cycles":3137280,"thread_start_us":113781,"threads_started":1,"wal-append.queue_time_us":1989970}
I20260709 13:02:42.720130  1631 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:02:42.699222  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:42.723572  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:42.726552  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:42.705435  1602 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:42.730686  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:42.731487  1635 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:42.733358  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.739239  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.590559  8560 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.740262  1628 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:42.744478  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.676355  8500 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.686336  8494 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.755568  8214 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:42.663748  8575 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:02:42.687316  8633 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:42.782153  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.785758  1628 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:02:42.785786  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.714602  8592 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.790702  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.791435  1628 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:42.754360  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:42.705880  8412 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:02:42.749750  8341 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:42.814570  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.740410  8409 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:42.794653  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3401) took 1857 ms. Trace:
I20260709 13:02:42.781720  8214 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:42.785127  8575 consensus_queue.cc:1243] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:02:42.795902  8561 raft_consensus.cc:2764] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:02:42.796804  8630 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 13:02:42.730866  8635 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:42.798269  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2487) took 1823 ms. Trace:
I20260709 13:02:42.798537  1635 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
W20260709 13:02:42.794241  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 67789us
I20260709 13:02:42.741010  8605 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:02:42.855122  1701 rpcz_store.cc:276] 0709 13:02:40.975022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.975225 (+   203us) service_pool.cc:224] Handling call
0709 13:02:40.975596 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.976075 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.976118 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.057369 (+ 81251us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.184171 (+126802us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:42.503305 (+1319134us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.796081 (+292776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.797883 (+  1802us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.797967 (+    84us) raft_consensus.cc:1685] finished
0709 13:02:42.797990 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.798190 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1594065,"prepare.queue_time_us":1318883,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":51,"raft.queue_time_us":331685,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":8962,"thread_start_us":118148,"threads_started":1,"wal-append.queue_time_us":757983}
I20260709 13:02:42.686550  8636 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.119s
I20260709 13:02:42.732669  8487 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:02:42.779824  8531 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:42.836040  8560 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:42.836966  8341 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:42.838217  8387 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.005s
I20260709 13:02:42.867831  8387 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.870380  8387 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:42.839332  8464 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:42.871410  8517 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:42.872008  8517 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:42.841043  8526 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:42.843012  8553 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:02:42.847646  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.872967  8553 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:42.847355  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 148145us
I20260709 13:02:42.874363  8553 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.875077  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:42.824021  1630 rpcz_store.cc:276] 0709 13:02:40.937722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.751551 (+813829us) service_pool.cc:224] Handling call
0709 13:02:41.786975 (+ 35424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.255534 (+468559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.255588 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.260764 (+  5176us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.545907 (+285143us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:42.545951 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.546034 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.794413 (+248379us) raft_consensus.cc:1685] finished
0709 13:02:42.794439 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.794591 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":467945,"raft.queue_time_us":470038,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":24170,"thread_start_us":233107,"threads_started":1,"wal-append.queue_time_us":345676}
I20260709 13:02:42.875682  8553 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:42.876238  8214 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.852525  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2502) took 4538 ms. Trace:
I20260709 13:02:42.914671  1560 rpcz_store.cc:276] 0709 13:02:38.314034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.094097 (+780063us) service_pool.cc:224] Handling call
0709 13:02:39.094423 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.191092 (+3096669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.852117 (+661025us) spinlock_profiling.cc:243] Waited 3.06 s on lock 0x7b54002ffb54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:42.852425 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7026930432}
I20260709 13:02:42.916054  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:42.916188  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3264) took 1237 ms. Trace:
I20260709 13:02:42.916708  1560 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:42.916574  1557 rpcz_store.cc:276] 0709 13:02:41.679073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.246030 (+566957us) service_pool.cc:224] Handling call
0709 13:02:42.871821 (+625791us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b5800002798. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:02:42.872096 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.915798 (+ 43702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.915833 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.915880 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:42.915922 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.915989 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.916129 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39215,"raft.queue_time_us":40023,"spinlock_wait_cycles":23163136}
I20260709 13:02:42.777511  8494 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.759479  8500 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.881731  8460 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:42.900157  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.873059  8517 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.878201  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:42.909724  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.924602  1630 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
W20260709 13:02:42.926805  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.928210  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:42.897228  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:42.891031  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 239176us
I20260709 13:02:42.877315  8464 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:42.930878  8464 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:42.931668  8464 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:42.884966  8055 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:42.882232  8573 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:42.891779  8574 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:42.848826  8483 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:02:42.851747  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 153479us
W20260709 13:02:42.912895  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 207610us
I20260709 13:02:42.917910  8341 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:42.942571  1636 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:42.943110  1635 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:42.943962  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:42.911783  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 235889us
I20260709 13:02:42.944330  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.945086  1635 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:42.942338  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:42.946412  1557 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:42.887068  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3256) took 2226 ms. Trace:
I20260709 13:02:42.857254  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:42.814481  8592 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:42.920768  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3115) took 2141 ms (client timeout 1554 ms). Trace:
W20260709 13:02:42.958329  1632 rpcz_store.cc:269] 0709 13:02:40.781202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.843711 (+ 62509us) service_pool.cc:224] Handling call
0709 13:02:42.920573 (+2076862us) spinlock_profiling.cc:243] Waited 55.5 ms on lock 0x7b5400197b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:42.920694 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127624064}
I20260709 13:02:42.923532  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:42.960179  1560 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:42.924765  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:42.935302  8227 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.000s
I20260709 13:02:42.950363  1552 rpcz_store.cc:276] 0709 13:02:40.660087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.204967 (+1544880us) service_pool.cc:224] Handling call
0709 13:02:42.205376 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.369460 (+164084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.886726 (+517266us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54003304d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:42.886986 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339994368}
I20260709 13:02:42.955506  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:42.963169  8613 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:42.966979  8613 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.972786  8636 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:42.974438  1627 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:42.976241  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:42.977016  1636 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:42.964469  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:42.978282  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:42.976130  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.966331  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:42.979285  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:42.979979  1633 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22.
W20260709 13:02:42.981294  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 103135us
W20260709 13:02:42.981858  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 57143.5us
I20260709 13:02:42.983598  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:42.963675  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 106479us
I20260709 13:02:42.987721  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:42.977596  8227 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:42.992153  8227 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:42.992803  8227 consensus_queue.cc:1059] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:42.993927  8341 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.965914  8536 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:42.996361  8536 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:42.923682  8383 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:02:42.997674  1627 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:42.997917  8565 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:02:42.966979  8441 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.529s	user 0.008s	sys 0.004s
I20260709 13:02:42.999254  8379 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.999503  8535 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:42.999785  8379 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:43.000221  8535 consensus_queue.cc:1243] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:02:43.000666  8637 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:02:42.861057  8635 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:43.002573  8565 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:43.003640  8565 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:43.004174  8635 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:43.005035  8635 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:43.006017  1634 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 23.
I20260709 13:02:42.953255  8409 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:43.006897  8409 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:43.008221  1634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1249) took 2138 ms (client timeout 2097 ms). Trace:
I20260709 13:02:43.010666  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:43.011217  8494 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:43.014003  8500 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:43.008754  1634 rpcz_store.cc:269] 0709 13:02:40.869785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.339247 (+469462us) service_pool.cc:224] Handling call
0709 13:02:43.008028 (+1668781us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400138dd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:43.008148 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3212320640}
I20260709 13:02:43.016625  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:43.020776  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:43.020972  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:42.977248  8456 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.659s	user 0.009s	sys 0.003s
I20260709 13:02:43.022182  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:43.024883  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:43.023351  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:42.935484  8573 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:43.032245  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:42.940512  8574 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:42.925866  8387 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:02:42.946484  8483 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:43.036048  1633 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:43.037595  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:43.020187  8526 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:43.023663  8636 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.983608  1552 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:43.031410  8227 raft_consensus.cc:2764] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:02:43.062372  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:43.030165  8635 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:43.057523  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:43.033229  8055 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:43.025058  8654 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:43.032754  8523 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:43.052695  1525 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:43.048382  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:43.076243  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:43.078207  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:43.079226  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:43.080062  2346 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:43.044631  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:43.067452  2344 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:43.063637  1633 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 23.
I20260709 13:02:43.082971  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:43.070839  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:43.036873  8549 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:43.068939  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3266) took 1389 ms. Trace:
I20260709 13:02:43.064393  8636 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:43.045437  8404 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:43.044159  8526 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:43.087872  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:43.031206  8456 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:43.073961  1636 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:43.090157  2344 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:43.061789  8409 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:43.094167  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:43.082019  8631 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.705s	user 0.011s	sys 0.003s
I20260709 13:02:43.084136  8460 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:43.146052  8460 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.145874  8631 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:43.146646  8460 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:43.146884  8648 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 13:02:43.148487  8631 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:43.148955  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1161) took 1457 ms. Trace:
I20260709 13:02:43.149431  8245 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:43.149823  8245 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:43.150084  1552 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:43.151839  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:43.152487  1552 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:43.104341  1634 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:43.074594  8654 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:43.181814  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:43.104984  8526 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:43.075515  8613 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:43.208612  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:43.135679  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:43.211421  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 133232us
I20260709 13:02:43.212451  2352 raft_consensus.cc:2410] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 13:02:43.133767  8523 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:43.132571  8487 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:43.143877  8456 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:43.144974  8644 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 13:02:43.070231  8381 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:02:43.145789  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:43.149338  1704 rpcz_store.cc:276] 0709 13:02:41.691117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.912643 (+221526us) service_pool.cc:224] Handling call
0709 13:02:41.913087 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.039061 (+125974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.148646 (+1109585us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400077954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:43.148883 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284620160}
I20260709 13:02:43.069897  8638 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:43.092952  1559 rpcz_store.cc:276] 0709 13:02:41.679711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.250068 (+570357us) service_pool.cc:224] Handling call
0709 13:02:42.258652 (+  8584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.258974 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:42.626463 (+367489us) log.cc:844] Serialized 14 byte log entry
0709 13:02:43.041093 (+414630us) log.cc:844] Serialized 14 byte log entry
0709 13:02:43.045184 (+  4091us) log.cc:844] Serialized 14 byte log entry
0709 13:02:43.068240 (+ 23056us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:43.068336 (+    96us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:43.068429 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.068549 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.068842 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30087,"prepare.queue_time_us":446617,"prepare.run_cpu_time_us":192,"prepare.run_wall_time_us":8186,"raft.queue_time_us":113191,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":56139,"spinlock_wait_cycles":15549184,"thread_start_us":413583,"threads_started":2,"wal-append.queue_time_us":284918}
I20260709 13:02:43.084268  8055 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:43.204003  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:43.085012  8656 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:43.252216  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:43.253029  1632 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:43.214457  8494 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:43.140909  8604 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:43.215387  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:43.119339  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:43.211310  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:43.205687  8370 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.006s
W20260709 13:02:43.184971  8649 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.082s	user 0.001s	sys 0.002s
I20260709 13:02:43.186404  8227 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:43.101120  8635 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:43.233283  8523 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:43.232882  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 149906us
I20260709 13:02:43.235383  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:43.177301  8657 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:43.151002  8245 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:43.243121  1558 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:43.245860  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3203) took 1151 ms. Trace:
I20260709 13:02:43.287796  1708 rpcz_store.cc:276] 0709 13:02:42.094534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.094699 (+   165us) service_pool.cc:224] Handling call
0709 13:02:43.145603 (+1050904us) spinlock_profiling.cc:243] Waited 7.89 ms on lock 0x7b5800155498. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6a00a0 00007fda2c69f984 00007fda25c1a6d4 00007fda25c1a662
0709 13:02:43.245764 (+100161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18141056}
I20260709 13:02:43.249974  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:43.250262  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:43.291364  1636 raft_consensus.cc:1217] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Deduplicated request from leader. Original: 12.11->[13.12-15.13]   Dedup: 13.12->[15.13-15.13]
I20260709 13:02:43.292152  1636 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: [13.12-15.13]
I20260709 13:02:43.217334  8379 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:43.140005  8522 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:43.257501  8381 consensus_queue.cc:1243] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer 95fc6a4cf5874b7eb91280dffcd3b3fb log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:02:43.088456  8574 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:43.256455  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 355479us
I20260709 13:02:43.257017  2352 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:43.254418  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 209735us
I20260709 13:02:43.275292  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:43.277901  1552 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:02:43.231485  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 178846us
I20260709 13:02:43.294610  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:43.295962  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:43.302767  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:43.152086  8573 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:43.094033  8404 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:43.320948  1633 raft_consensus.cc:2376] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17. Re-sending same reply.
W20260709 13:02:43.304518  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 89769.3us
I20260709 13:02:43.257663  8604 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:43.322889  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:43.276113  8227 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:43.325037  1675 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:43.297344  8657 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:43.326442  8573 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:43.326668  8523 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:43.327100  8381 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:43.327898  8055 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:43.329006  8635 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:43.314210  8149 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:43.331476  8574 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:43.303544  8522 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:43.302958  8494 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.283808  8370 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:43.337428  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.325732  8565 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:43.336853  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:43.356540  2353 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:43.329177  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:43.376699  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51841.6us
I20260709 13:02:43.367206  8227 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:43.333452  8604 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:43.337100  8404 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:43.381788  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:43.401393  8562 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.365134  8653 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:43.446027  1560 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 23: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:43.390525  8245 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:43.434345  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 97522.9us
I20260709 13:02:43.369002  8657 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:43.437603  2347 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:02:43.452092  2347 pending_rounds.cc:85] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 11
I20260709 13:02:43.452404  2347 pending_rounds.cc:107] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:02:43.452289  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:43.452728  2347 raft_consensus.cc:2889] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:43.438418  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 109247us
I20260709 13:02:43.439281  1675 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:43.453616  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:43.458741  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:43.440557  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 58859.6us
I20260709 13:02:43.464710  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:43.441741  1559 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:43.472354  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:43.473322  1628 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:43.478281  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:43.420889  1634 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:43.404098  8669 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:43.449188  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:43.416487  8381 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:43.488597  2353 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:43.484431  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:43.440261  8214 raft_consensus.cc:2764] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:43.445869  8664 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:43.248489  8654 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:43.422127  8494 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:43.482822  8604 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:43.468648  8454 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:43.557986  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 95287us
I20260709 13:02:43.513624  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:43.512197  2353 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
W20260709 13:02:43.528982  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.471956  8379 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:43.480525  8656 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:02:43.517921  8373 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.000s	sys 0.024s
I20260709 13:02:43.497381  8341 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:02:43.574077  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 60554.2us
I20260709 13:02:43.433911  8565 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:43.515682  1559 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:43.451359  8562 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:43.593380  2347 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:02:43.574908  8379 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:43.516629  8664 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:43.532055  8536 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:43.577009  8655 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.362s	user 0.001s	sys 0.000s
I20260709 13:02:43.610350  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:43.615374  1559 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: [17.13-17.13]
W20260709 13:02:43.590312  8373 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.000s	sys 0.024s
I20260709 13:02:43.524484  8381 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:43.628906  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 170143us
I20260709 13:02:43.595749  8654 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:43.521272  8409 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 13:02:43.568074  8454 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.626763  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:43.675576  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 48888us
I20260709 13:02:43.676649  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:43.645535  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:43.486976  8522 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:43.576223  8668 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:43.613354  8379 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:43.672117  1554 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:02:43.612717  8687 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:43.618634  8664 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:43.620500  8536 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:43.690325  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:43.630793  8383 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:43.711695  8383 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:43.700217  8564 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:43.685750  8661 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.349s	user 0.000s	sys 0.002s
I20260709 13:02:43.631160  8497 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:43.683305  8454 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:43.706305  8522 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:43.705444  8613 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:43.707320  8536 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:43.758277  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 304580us
W20260709 13:02:43.719444  8660 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 13:02:43.782061  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:43.784037  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:43.787027  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:43.722515  8564 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:43.788892  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:43.727309  8669 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:43.725369  8584 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:43.788823  8549 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:43.789840  8686 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:43.808624  8674 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:02:43.757426  8533 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:43.763657  8613 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.779799  8370 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:43.791416  8214 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:43.798436  8564 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:43.785614  8341 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:43.889313  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 166773us
I20260709 13:02:43.802327  8584 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:43.920255  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:43.904882  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3265) took 2225 ms. Trace:
W20260709 13:02:43.915797  1525 leader_election.cc:336] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.538s (SENT)
W20260709 13:02:43.927711  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 230155us
I20260709 13:02:43.928754  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:43.921412  1553 rpcz_store.cc:276] 0709 13:02:41.679394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.248298 (+568904us) service_pool.cc:224] Handling call
0709 13:02:42.874451 (+626153us) spinlock_profiling.cc:243] Waited 9.24 ms on lock 0x7b5800002798. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:02:42.874699 (+   248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.912560 (+ 37861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.912593 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.920082 (+  7489us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.972057 (+ 51975us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b4800002958. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:02:43.280313 (+308256us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:43.280364 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.280442 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.904522 (+624080us) raft_consensus.cc:1685] finished
0709 13:02:43.904559 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.904791 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37399,"raft.queue_time_us":43063,"raft.run_cpu_time_us":3111,"raft.run_wall_time_us":52720,"spinlock_wait_cycles":28334720,"thread_start_us":342471,"threads_started":1,"wal-append.queue_time_us":494754}
W20260709 13:02:43.987607  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 67434.2us
I20260709 13:02:43.898062  8613 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:43.988266  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 59603.5us
I20260709 13:02:43.899329  8214 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:43.914574  8686 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:44.005862  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:43.925899  8584 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:44.018796  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 262036us
I20260709 13:02:43.901355  8549 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:43.930723  8494 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.005407  8389 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:44.050380  8553 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.138s	user 0.020s	sys 0.002s
W20260709 13:02:44.048422  8633 consensus_meta.cc:287] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.257s	user 0.021s	sys 0.009s
I20260709 13:02:44.030977  8214 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:44.051637  8685 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:44.060111  8549 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.054529  8681 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:02:44.041987  8669 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.071483  8681 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:44.043207  8381 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:44.043615  8562 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.061336  8633 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.076690  8633 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.077756  1555 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 14: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:44.040905  8565 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:44.070230  8549 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:44.089111  8688 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:44.089700  8688 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:44.107250  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.108031  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.108757  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.054754  8695 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:44.110714  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.108667  8341 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:44.054857  8671 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:02:44.120256  8464 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.188s	user 0.018s	sys 0.015s
I20260709 13:02:44.120011  8613 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:44.054843  8670 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.379s	user 0.001s	sys 0.000s
I20260709 13:02:44.120395  8508 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:44.122906  8227 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.680s	user 0.014s	sys 0.010s
I20260709 13:02:44.126893  1556 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:44.054924  8667 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
W20260709 13:02:44.114439  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 325423us
W20260709 13:02:44.140841  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 83575.6us
W20260709 13:02:44.147190  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.148648  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:44.151423  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:44.153626  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:44.043131  8652 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.212s
I20260709 13:02:44.169410  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:44.170925  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.141202  8700 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:02:44.162938  8635 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.761s	user 0.021s	sys 0.003s
I20260709 13:02:44.161579  8654 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:44.176153  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:44.125388  8454 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:44.175498  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 173401us
I20260709 13:02:44.165036  8702 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:44.203501  1628 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.017s	sys 0.010s
I20260709 13:02:44.204293  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:44.143461  8464 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.174595  8688 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.159526  8562 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:02:44.154866  8227 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.170065  8553 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:02:44.270774  2344 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:44.291183  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:44.291764  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:44.268293  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.290465  1628 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.028s	sys 0.021s
I20260709 13:02:44.270567  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:44.295034  2346 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:44.290822  8487 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.297278  8487 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.299983  8487 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:44.304471  8705 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 13:02:44.206233  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 200400us
W20260709 13:02:44.315210  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 110939us
I20260709 13:02:44.315107  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:44.202958  8621 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.660s	user 0.018s	sys 0.000s
W20260709 13:02:44.321718  8657 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.850s	user 0.012s	sys 0.012s
W20260709 13:02:44.329988  8604 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.757s	user 0.007s	sys 0.025s
I20260709 13:02:44.257642  8456 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:02:44.323199  8693 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 13:02:44.331230  8456 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:44.331472  8604 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:44.332316  8456 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.180197  8494 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:44.335188  8456 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:44.325253  8389 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:44.326149  8409 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:44.326936  8688 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:44.327003  1553 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:44.179137  8700 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.202121  8654 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:44.330109  8657 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.324229  8535 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.198870  8522 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.351675  1675 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:44.352507  2347 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEARNER]: Advancing to term 18
I20260709 13:02:44.203413  8686 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:44.328536  8621 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.346602  8356 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.350466  8669 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:02:44.356356  8245 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.887s	user 0.015s	sys 0.011s
I20260709 13:02:44.335971  8454 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:44.369704  8654 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:44.404001  8564 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.404983  8621 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.405679  1557 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:02:44.406008  8460 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.259s	user 0.008s	sys 0.009s
I20260709 13:02:44.406231  8613 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:44.409206  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 99060.4us
I20260709 13:02:44.361765  8565 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:44.369576  8409 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:44.395752  8404 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.945s	user 0.011s	sys 0.003s
W20260709 13:02:44.415097  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 63503.7us
I20260709 13:02:44.439782  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:44.441618  2346 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
W20260709 13:02:44.361773  8497 raft_consensus.cc:670] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } } [suppressed 6 similar messages]
I20260709 13:02:44.205722  8533 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.191s
W20260709 13:02:44.341398  8573 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.967s	user 0.008s	sys 0.012s
I20260709 13:02:44.414474  8669 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:44.415683  8356 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:44.419579  8536 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.003s
W20260709 13:02:44.422487  8574 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.011s	user 0.016s	sys 0.011s
I20260709 13:02:44.421761  8702 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:44.422982  8657 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.396502  8535 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.424227  8389 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:44.398017  8636 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.293s	user 0.010s	sys 0.006s
W20260709 13:02:44.408169  8684 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.231s	user 0.000s	sys 0.000s
I20260709 13:02:44.336740  8604 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:02:44.398885  8686 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:44.451198  2347 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:02:44.446143  1675 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:44.470089  2347 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 9
I20260709 13:02:44.470513  2347 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:02:44.423343  8464 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.433136  8227 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:44.459098  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 177870us
I20260709 13:02:44.402523  8522 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.444474  8590 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:44.456547  8381 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:44.470942  2347 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:44.442400  8704 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:44.456903  8669 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:44.488628  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:44.488058  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:44.443073  8565 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:44.482421  1708 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:02:44.492663  8668 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.788s	user 0.011s	sys 0.001s
I20260709 13:02:44.493258  8668 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:44.483474  8702 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:44.493793  8214 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:44.494704  8708 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:44.494547  8702 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.496773  8535 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.467278  8389 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.498212  8389 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:44.499220  8356 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.453430  8404 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.501812  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:44.496872  8702 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.503502  8404 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.442706  8460 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.462996  8536 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.506057  8460 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.473094  8686 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:44.507180  8536 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.469635  8636 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.508466  8565 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.510437  8565 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:44.514879  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:44.515473  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.521520  1706 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: [15.11-16.12]
W20260709 13:02:44.489265  8641 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.647s	user 0.004s	sys 0.004s
I20260709 13:02:44.454874  8706 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
W20260709 13:02:44.492667  8379 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.788s	user 0.013s	sys 0.019s
I20260709 13:02:44.525560  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:44.510921  8409 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.520507  8668 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:44.537400  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.515959  8381 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:44.527769  8664 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.813s	user 0.006s	sys 0.007s
I20260709 13:02:44.523625  8604 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:02:44.557926  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:44.551967  1709 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:44.554742  1705 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:44.522497  8584 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:44.569797  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 113030us
I20260709 13:02:44.555819  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3210) took 2174 ms. Trace:
W20260709 13:02:44.576614  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.579325  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:44.578975  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:44.557230  8554 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:44.539655  8613 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 13:02:44.584120  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:44.534425  8379 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.575565  8713 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:02:44.602478  8379 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.602728  1634 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:44.606266  8613 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:44.606767  8613 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:44.596832  8668 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.607784  8613 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.607919  8668 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:44.586346  2346 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:44.609364  8613 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:44.597780  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:44.616397  1552 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:44.599242  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:44.613197  8564 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:44.619095  8584 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.605786  8669 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:44.621322  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.598743  2352 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:44.607570  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3132) took 3768 ms. Trace:
I20260709 13:02:44.624737  1629 rpcz_store.cc:276] 0709 13:02:40.838775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.665365 (+826590us) service_pool.cc:224] Handling call
0709 13:02:41.676079 (+ 10714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.254083 (+578004us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.254139 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.254840 (+   701us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.665194 (+410354us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:42.665246 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.665347 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.355077 (+1689730us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:44.607196 (+252119us) raft_consensus.cc:1685] finished
0709 13:02:44.607224 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.607498 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":566626,"raft.queue_time_us":572090,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":21236,"spinlock_wait_cycles":33966464,"thread_start_us":344613,"threads_started":1,"wal-append.queue_time_us":552334}
I20260709 13:02:44.574692  1710 rpcz_store.cc:276] 0709 13:02:42.381836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.621667 (+239831us) service_pool.cc:224] Handling call
0709 13:02:42.622037 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.622496 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:02:42.622843 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 13:02:44.307422 (+1684579us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b18003dfcc0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:02:44.555062 (+247640us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.555119 (+    57us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:44.555181 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.555599 (+   418us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.555751 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":122,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":5220,"raft.queue_time_us":1513,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":4353,"spinlock_wait_cycles":59732736,"thread_start_us":247225,"threads_started":1}
I20260709 13:02:44.575815  8214 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.645995  1703 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:44.553884  8381 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.555840  8664 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:44.631453  8668 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:44.661689  1704 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:02:44.662225  1704 pending_rounds.cc:85] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 11
I20260709 13:02:44.662537  1704 pending_rounds.cc:107] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:02:44.662968  1704 raft_consensus.cc:2889] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:44.630338  8497 consensus_queue.cc:1175] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" responder_term: 18 status { last_received { term: 18 index: 15 } last_committed_idx: 15 last_received_current_leader { term: 18 index: 15 } }
W20260709 13:02:44.676945  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.665593  8526 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.438s	user 0.004s	sys 0.014s
I20260709 13:02:44.654455  8681 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.664851  8636 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.648916  8584 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:02:44.704072  8681 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.723167  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:44.693846  8564 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.727835  2349 raft_consensus.cc:2410] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 13:02:44.728755  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.733408  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.701586  8522 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.752749  1552 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:44.736421  8564 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.764242  8564 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:44.758340  8719 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:02:44.771268  1525 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.497s (SENT)
I20260709 13:02:44.771915  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:44.773427  8341 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:44.759228  8686 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.779281  8681 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:44.780010  8686 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.794332  8664 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.796487  8669 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:44.798355  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:44.801766  8671 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:44.726778  8409 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.813025  8409 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.815847  8409 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:42.070778  1561 rpcz_store.cc:276] 0709 13:02:40.659703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.125898 (+466195us) service_pool.cc:224] Handling call
0709 13:02:41.129770 (+  3872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.130037 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:41.741693 (+611656us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.060790 (+319097us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.060900 (+   110us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:42.060950 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.061024 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.061147 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9734,"prepare.queue_time_us":777032,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":4761,"raft.queue_time_us":102,"thread_start_us":876303,"threads_started":2}
I20260709 13:02:44.819142  8055 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.703238  8526 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.819839  8055 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.820442  8561 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:44.820874  8561 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:44.821830  8561 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.831585  8669 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.835559  2351 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:44.834487  8669 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.822343  8526 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.839344  8654 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.822348  8055 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.841198  8389 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:44.843302  2346 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:44.844554  2345 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:44.862128  1705 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:44.862241  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:44.863070  2345 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:44.845192  2350 raft_consensus.cc:2410] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:44.848487  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:44.857136  1707 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 20. Current term is 21. Ops: []
W20260709 13:02:44.863722  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.864979  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:44.865953  1635 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:02:44.865592  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:44.861006  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:44.866806  2346 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 23.
I20260709 13:02:44.868993  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:44.869962  1629 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:44.870419  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:44.870720  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:44.871135  1702 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:44.865513  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:44.871953  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:44.872503  1707 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 23.
I20260709 13:02:44.872828  2345 raft_consensus.cc:2410] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:44.874153  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:44.875144  2346 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:44.877000  1710 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:44.877882  2349 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:44.879518  2345 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 23: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:44.843205  8669 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:44.881732  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:44.882932  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:44.884989  1635 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:44.883229  1702 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17. Ops: [14.8-16.9]
I20260709 13:02:44.885107  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:44.885176  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:44.886104  1636 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:44.886965  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:44.887102  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:44.887981  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:44.888578  1628 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:44.888549  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:44.889758  1631 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:44.890065  1635 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:44.891155  1627 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:44.891268  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:44.892211  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:44.892472  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:44.893143  1627 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:02:44.893231  1633 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:44.894294  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:44.894796  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:44.896232  1628 raft_consensus.cc:2393] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:44.897467  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:44.899187  1636 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:44.898644  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:44.899693  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:44.900410  1627 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:44.900889  1635 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:02:44.904824  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:44.905663  1710 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:02:44.907171  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:44.874195  8508 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.907109  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:44.909720  2349 raft_consensus.cc:2410] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:02:44.910949  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:44.915103  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:44.915750  1710 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:44.913003  8508 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.924340  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:44.924448  2349 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:44.926451  8654 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.928887  8389 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:44.931705  2317 leader_election.cc:336] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.097s (SENT)
I20260709 13:02:44.932307  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:44.934213  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.936655  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.943212  8593 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:44.944618  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:44.945587  8593 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:44.946020  8593 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:44.946390  8593 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:44.950377  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:44.952361  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:44.934978  8412 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:44.936456  8584 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.954741  2349 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:44.938421  8712 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.940707  8609 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:02:44.940596  8590 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:02:44.959797  8584 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.968883  8584 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:44.962231  8712 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.970310  8593 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:44.972102  8712 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:44.972772  8593 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:45.031194  1527 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:45.033613  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.035295  1632 raft_consensus.cc:2376] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17. Re-sending same reply.
I20260709 13:02:45.039258  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.045470  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:45.047555  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:45.051246  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.047559  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.129189  8526 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:45.093885  1636 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:45.066619  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:45.111544  2317 leader_election.cc:336] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.800s (SENT)
I20260709 13:02:45.065506  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.138644  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.147105  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.148264  1707 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:45.149797  1710 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:45.104929  8383 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.142215  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.150774  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.136672  8526 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:45.161796  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:45.165409  1627 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:45.167586  1635 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:45.174168  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.177974  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 1074) took 2246 ms (client timeout 1800 ms). Trace:
W20260709 13:02:45.179915  1560 rpcz_store.cc:269] 0709 13:02:42.931172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.977980 (+ 46808us) service_pool.cc:224] Handling call
0709 13:02:45.177458 (+2199478us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b5400312254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:45.177711 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1067510016}
I20260709 13:02:45.079429  8227 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:02:45.183596  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.195921  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.173229  8383 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:45.200369  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.212250  1636 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:45.215698  1635 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:02:45.216075  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:45.219871  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.224465  8412 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:45.225562  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:45.226600  8373 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:45.226668  8483 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:45.172931  8526 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:45.232745  1636 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:45.237192  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:45.236732  8412 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.241505  8412 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:45.242743  8592 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:45.247408  1709 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:45.252278  1627 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
I20260709 13:02:45.254174  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.255887  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:45.247766  8412 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:45.261616  1704 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:45.270701  8700 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:45.283953  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:45.285382  1553 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:02:45.284212  1555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3277) took 2388 ms (client timeout 2540 ms). Trace:
I20260709 13:02:45.286535  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.287788  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2530) took 2600 ms. Trace:
W20260709 13:02:45.287922  2317 leader_election.cc:336] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.893s (SENT)
I20260709 13:02:45.288669  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:45.288357  1556 rpcz_store.cc:276] 0709 13:02:42.687279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.689639 (+  2360us) service_pool.cc:224] Handling call
0709 13:02:42.690339 (+   700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.144785 (+1454446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.285551 (+1140766us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54003c0fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:45.287436 (+  1885us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3301068544}
I20260709 13:02:45.291064  1558 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:45.291340  1627 raft_consensus.cc:2376] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23. Re-sending same reply.
I20260709 13:02:45.293504  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:45.293272  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 1081) took 2359 ms. Trace:
W20260709 13:02:45.286336  1555 rpcz_store.cc:269] 0709 13:02:42.895891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.896905 (+  1014us) service_pool.cc:224] Handling call
0709 13:02:45.283760 (+2386855us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000bf254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:45.284012 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2636493568}
I20260709 13:02:45.272953  8483 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:45.298245  8616 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.298835  8592 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:45.302907  8227 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:45.295008  1558 rpcz_store.cc:276] 0709 13:02:42.933923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.321592 (+387669us) service_pool.cc:224] Handling call
0709 13:02:45.290543 (+1968951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.291739 (+  1196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.293004 (+  1265us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54000bf250. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:45.293180 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4504536832}
I20260709 13:02:45.303685  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3428) took 1751 ms. Trace:
I20260709 13:02:45.303853  8227 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:45.304183  1634 rpcz_store.cc:276] 0709 13:02:43.552370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.553172 (+   802us) service_pool.cc:224] Handling call
0709 13:02:43.553575 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.603674 (+1050099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.303037 (+699363us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002da5d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:45.303422 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2410740992}
I20260709 13:02:45.281164  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:45.308074  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.305660  1635 raft_consensus.cc:2376] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17. Re-sending same reply.
W20260709 13:02:45.310196  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.312531  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.313848  8616 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:45.316874  8616 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:45.317862  8592 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:45.318621  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:45.319042  2350 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1221) took 3293 ms (client timeout 2140 ms). Trace:
I20260709 13:02:45.319674  1635 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:02:45.320601  8592 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:45.320827  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:45.319504  2350 rpcz_store.cc:269] 0709 13:02:42.025258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.025423 (+   165us) service_pool.cc:224] Handling call
0709 13:02:45.318573 (+3293150us) spinlock_profiling.cc:243] Waited 2.8 s on lock 0x7b54004bf0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:45.318810 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6442903296}
W20260709 13:02:45.323493  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 90169.6us
I20260709 13:02:45.330391  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:45.331760  1634 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:45.335541  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47714.6us
I20260709 13:02:45.338521  8700 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:45.343052  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 62215.1us
I20260709 13:02:45.343524  8508 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:45.343636  8698 consensus_queue.cc:1175] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" responder_term: 17 status { last_received { term: 16 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:45.345984  8636 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:45.347047  8636 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:45.345970  8453 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:45.352779  8227 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:45.354665  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:45.355978  1629 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:45.362165  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.362617  8704 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:45.363281  1631 raft_consensus.cc:2393] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:45.366631  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.368047  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.369242  1631 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:45.362708  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61522.6us
I20260709 13:02:45.372237  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.373112  8227 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:45.373436  8704 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:45.362311  1634 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:02:45.374150  8227 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:45.375617  1634 pending_rounds.cc:85] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:02:45.377301  8494 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:45.379225  8494 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:45.294166  1705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:45.382012  1705 raft_consensus.cc:606] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:45.384698  1705 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:02:45.389020  8704 consensus_queue.cc:1243] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:02:45.393510  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 1083) took 2458 ms (client timeout 2057 ms). Trace:
W20260709 13:02:45.395449  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50204.4us
I20260709 13:02:45.396657  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:45.397284  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2514) took 4658 ms. Trace:
I20260709 13:02:45.378597  8383 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:45.405035  1554 rpcz_store.cc:276] 0709 13:02:40.738594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.206401 (+1467807us) service_pool.cc:224] Handling call
0709 13:02:42.206890 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.686247 (+1479357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.396537 (+1710290us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54003309d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:45.397063 (+   526us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3366738176}
I20260709 13:02:45.415489  1710 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:02:45.357151  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.398044  1634 pending_rounds.cc:107] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:02:45.419838  1634 raft_consensus.cc:2889] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:45.404522  1557 rpcz_store.cc:269] 0709 13:02:42.934452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.041694 (+107242us) service_pool.cc:224] Handling call
0709 13:02:45.393021 (+2351327us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400251cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:45.393261 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3031820544}
I20260709 13:02:45.421175  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:45.423648  1633 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.529s	user 0.006s	sys 0.023s
I20260709 13:02:45.424458  1633 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:45.428496  1631 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:02:45.431622  8653 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.432314  8653 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:45.434016  8616 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:45.434747  8704 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:45.434897  8616 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:45.435514  8704 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.443111  1709 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:45.448971  1636 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:45.445531  1635 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:45.451967  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:45.454025  2347 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:45.455307  8554 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:45.437546  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:45.456113  8554 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:45.457263  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:45.458757  8654 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:45.459257  8654 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.459734  8654 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:45.437626  8704 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:45.465977  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:45.470139  1710 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:02:45.471390  8679 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.706s	user 0.000s	sys 0.005s
I20260709 13:02:45.474779  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.475598  1630 raft_consensus.cc:2376] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23. Re-sending same reply.
I20260709 13:02:45.494794  1627 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:45.438007  8573 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.501513  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:45.517467  1634 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:02:45.523908  8312 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:45.524722  8312 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:45.527421  8312 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:45.528749  8409 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:02:45.511268  8592 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.530069  8592 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:45.532783  8592 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:45.547670  8373 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.548813  8373 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:45.552428  8373 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:45.559980  8636 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:45.562304  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1278) took 2352 ms (client timeout 2845 ms). Trace:
I20260709 13:02:45.563524  8149 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:02:45.520421  2317 leader_election.cc:336] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.089s (SENT)
W20260709 13:02:45.573288  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 58049.1us
I20260709 13:02:45.523097  8616 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:45.516731  8526 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:45.568871  1632 rpcz_store.cc:269] 0709 13:02:43.209433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.253791 (+ 44358us) service_pool.cc:224] Handling call
0709 13:02:45.561941 (+2308150us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b540010d1d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:45.562190 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3955769216}
I20260709 13:02:45.546672  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:45.586285  8312 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:45.587200  8526 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:45.588276  8690 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:45.588795  8592 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:45.594828  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.598802  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.602862  1554 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:45.604607  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:45.606117  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 91789.6us
I20260709 13:02:45.606928  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.615046  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.619010  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:45.620443  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.624138  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.625413  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:45.627257  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1166) took 1147 ms. Trace:
I20260709 13:02:45.627921  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.628196  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:45.637425  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.638690  8312 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:45.638618  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.642073  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.643504  1703 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:45.644246  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.656563  1553 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 13.
I20260709 13:02:45.612360  8653 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:45.654665  1706 rpcz_store.cc:276] 0709 13:02:44.479512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.479717 (+   205us) service_pool.cc:224] Handling call
0709 13:02:44.491203 (+ 11486us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:44.522231 (+ 31028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.626660 (+1104429us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b540036ddd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:45.627053 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68209024}
W20260709 13:02:45.656584  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.514945  8573 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:45.633343  8227 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:45.679062  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44942.4us
I20260709 13:02:45.684087  1559 raft_consensus.cc:2393] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:45.687052  1636 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:45.690675  1552 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:45.652839  8592 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:45.659231  8312 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:45.696880  1635 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:45.696700  1560 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:45.690793  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:45.698331  1630 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:45.705243  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.697449  1555 raft_consensus.cc:2376] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23. Re-sending same reply.
I20260709 13:02:45.707855  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.708778  1634 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:45.705862  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:45.712625  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.714269  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.716675  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.721141  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:45.705257  8653 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.726181  1557 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:02:45.728933  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.729161  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.731186  1553 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:45.732443  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.734165  8312 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:45.735724  8312 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:45.738000  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.739648  1556 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:45.740031  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.741842  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.742635  1709 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:45.743381  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:45.751196  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.761291  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.773324  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.803606  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:45.811743  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 305562us
I20260709 13:02:45.815728  1560 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:45.816067  1559 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:45.817826  1559 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:02:45.820935  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:45.821669  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.824420  1555 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:45.787573  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:45.827008  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.753465  8526 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:45.838176  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 193865us
I20260709 13:02:45.834923  1554 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:02:45.840490  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 120477us
I20260709 13:02:45.845348  8055 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:45.846182  8055 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:45.846510  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.848461  1710 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:02:45.848836  1710 pending_rounds.cc:85] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 10
I20260709 13:02:45.848973  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:45.849282  8664 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:45.849956  1706 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
W20260709 13:02:45.850864  8744 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:02:45.851814  1706 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:45.853061  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.839733  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:45.849097  1710 pending_rounds.cc:107] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:02:45.856890  1710 raft_consensus.cc:2889] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:45.849061  8055 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:45.859716  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:45.858619  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 104508us
W20260709 13:02:45.858649  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 104081us
W20260709 13:02:45.867691  1525 leader_election.cc:336] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.633s (SENT)
I20260709 13:02:45.876246  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:45.877766  1560 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:45.873059  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:45.880115  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:45.879388  1630 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:45.884128  1706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5838267321348eb83df088988da6449"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:45.890805  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:45.827982  8653 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:45.895872  1706 raft_consensus.cc:606] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:45.900192  1633 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:45.905021  1705 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:45.905256  1706 raft_consensus.cc:614] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:45.905864  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:45.891541  8526 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:45.890205  8664 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:45.913687  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.918289  8753 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.066s	user 0.001s	sys 0.001s
I20260709 13:02:45.883155  8654 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:45.922487  1630 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:45.928093  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:45.940407  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:45.942485  1632 raft_consensus.cc:2368] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19.
W20260709 13:02:45.943607  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 282087us
I20260709 13:02:45.947731  1559 raft_consensus.cc:2393] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:45.952059  8474 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:45.952659  8474 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:45.953711  8389 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:45.953958  8474 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:45.929981  8055 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:02:45.955353  8474 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:45.956496  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.957755  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:45.959393  8573 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:45.956252  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:45.964381  1674 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:45.976575  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:45.977774  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 140053us
I20260709 13:02:45.974063  8752 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.990254  1627 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
W20260709 13:02:45.992164  8747 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:45.992887  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 1175) took 1467 ms. Trace:
I20260709 13:02:45.993510  1707 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:45.993906  8523 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:45.995030  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:46.015091  1629 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:46.011659  1708 rpcz_store.cc:276] 0709 13:02:44.525086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.525231 (+   145us) service_pool.cc:224] Handling call
0709 13:02:45.992411 (+1467180us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400077e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:45.992662 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2752465152}
W20260709 13:02:46.036455  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66851.2us
W20260709 13:02:46.041302  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 53837.8us
W20260709 13:02:46.043403  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.047011  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:46.024214  8704 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.553s	user 0.015s	sys 0.000s
I20260709 13:02:46.051945  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:46.044010  2317 leader_election.cc:336] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.845s (SENT)
I20260709 13:02:46.044598  1706 raft_consensus.cc:2376] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16. Re-sending same reply.
I20260709 13:02:46.016752  8523 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:46.063812  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
W20260709 13:02:46.066898  1525 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.640s (SENT)
I20260709 13:02:46.070847  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:46.064546  1710 raft_consensus.cc:606] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:46.072902  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:46.064128  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:46.076735  1710 raft_consensus.cc:614] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:46.040329  8389 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.082820  1705 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:46.086258  1635 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:46.089995  1631 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:02:46.082321  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:46.089377  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 149725us
I20260709 13:02:46.088819  1674 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:46.098412  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:46.069307  8523 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:46.107712  8754 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:02:46.108533   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:46.109807  2349 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:46.109023  8441 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.163s
W20260709 13:02:46.108975  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 58874us
W20260709 13:02:46.111027  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 74602.3us
I20260709 13:02:46.111925  8517 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:46.112839  8389 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:46.114809  8653 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:46.117638  8526 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:46.117940  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:46.169577  1552 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:46.078183  8381 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.133s
I20260709 13:02:46.124835  1557 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17. Ops: [12.7-16.9]
I20260709 13:02:46.125908  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:46.126914  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:46.127916  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:46.108560  8752 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:46.148378  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:46.149461  8664 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:46.184389  1559 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:46.157116  1636 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:46.158007  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:46.163614  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.192124  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.184361  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:46.059556  8704 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:46.088443  8453 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.198859  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:46.283890  8759 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.195s	user 0.002s	sys 0.000s
I20260709 13:02:46.284361  8453 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:46.285179  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 86433.3us
W20260709 13:02:46.287135  8755 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.178s	user 0.002s	sys 0.000s
I20260709 13:02:46.200810  8762 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:46.288424  8756 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:46.289291  8704 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:46.202811  1554 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 13:02:46.292768  8750 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
I20260709 13:02:46.203752  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:46.293390  8762 consensus_queue.cc:1243] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:02:46.204285  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 152829us
I20260709 13:02:46.296212  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:46.204901  8752 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:46.206141  1555 raft_consensus.cc:2410] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:02:46.207316  8227 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:46.297989  8752 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:46.298462  8752 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:46.298542  8227 raft_consensus.cc:1068] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Unable to promote non-voter 95fc6a4cf5874b7eb91280dffcd3b3fb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:46.299461  1631 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:02:46.299993  8149 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.300520  8149 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:46.300323  8752 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:46.301728  8698 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:46.303820  8698 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:46.306221  8387 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:46.306985  8387 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:46.307458  8767 consensus_queue.cc:1059] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:46.287155  8453 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:46.209676  8654 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:46.306073  8573 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:46.203745  8298 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:46.309811  8654 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:46.310500  8298 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:46.310207  8573 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:46.311467  8767 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:46.216548  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2519) took 1360 ms. Trace:
I20260709 13:02:46.313696  1707 rpcz_store.cc:276] 0709 13:02:44.856256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.186710 (+330454us) service_pool.cc:224] Handling call
0709 13:02:45.993264 (+806554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.036165 (+ 42901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.216300 (+180135us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b5400077e50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:46.216483 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816051968}
I20260709 13:02:46.219192  8389 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:46.209457  8494 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:46.315681  8494 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:46.233348  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:46.317919  1705 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:02:46.233681  1552 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:02:46.319051  1636 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:46.319334  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.234138  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:46.320705  1557 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:46.206271  8454 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:46.321834  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.244130  1703 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:46.322852  1631 raft_consensus.cc:2393] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 23: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:46.323938  8453 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:46.325134  8453 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:46.321610  8454 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.336815  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.337126  8454 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:46.342190  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:46.343127  1552 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 23.
W20260709 13:02:46.250286  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 208089us
I20260709 13:02:46.345932  8573 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:46.346611  8573 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.346643  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:46.256943  8554 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.800s	user 0.015s	sys 0.000s
I20260709 13:02:46.349522  8554 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:46.349555  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:46.354255  2317 leader_election.cc:336] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.871s (SENT)
I20260709 13:02:46.354486  1631 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
W20260709 13:02:46.258396  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 214810us
I20260709 13:02:46.262346  8706 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.264963  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.356115  8706 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:46.266569  8561 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:46.268046  8653 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:46.361711  8653 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:46.362409  1525 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.662s (SENT)
I20260709 13:02:46.363379  8653 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:46.364817  8653 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:46.347167  8573 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:46.366006  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:46.362926  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:46.271575  8690 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:46.366777  1555 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:46.367482  8453 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:46.277006  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.368835  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:46.369074  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.270602  2344 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:46.276605  8748 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.373s	user 0.001s	sys 0.001s
I20260709 13:02:46.289753  8756 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:46.202442  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:46.203145  1553 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:46.308089  8387 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:46.310542  8654 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:46.321511  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.246552  1706 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:46.327565  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:46.328783  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:46.364820  1525 leader_election.cc:336] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.733s (SENT)
I20260709 13:02:46.392438  8227 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:46.392833  1559 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:46.393018  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:46.393016  8227 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.394932  8227 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:46.395395  8573 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:46.398430  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.400753  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:46.309866  8549 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:02:46.241441  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 81613us
W20260709 13:02:46.208655  8742 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.523s	user 0.003s	sys 0.000s
I20260709 13:02:46.329860  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.330260  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:46.331295  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:46.407191  1635 raft_consensus.cc:2376] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16. Re-sending same reply.
I20260709 13:02:46.331130  8389 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:46.369211  8453 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:46.226131  8592 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:46.333757  8298 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:46.305114  8698 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:02:46.246558  8523 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:46.356381  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.366212  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:46.368016  1701 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
W20260709 13:02:46.278815  8757 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.316s	user 0.000s	sys 0.002s
W20260709 13:02:46.279608  1456 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 190) took 22656 ms (client timeout 15000 ms). Trace:
I20260709 13:02:46.271633   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:02:46.409785  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 302946us
I20260709 13:02:46.405401  1634 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:46.408667  1632 raft_consensus.cc:2376] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18. Re-sending same reply.
I20260709 13:02:46.332672  1674 leader_election.cc:400] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19.
I20260709 13:02:46.416917  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.424683  1674 leader_election.cc:403] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:46.418411  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 72187.2us
I20260709 13:02:46.419500  1636 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:46.419652  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:46.420682  1552 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:46.430366  1635 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:46.431150  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.412744  8549 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:46.361720  8706 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:46.462352  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.417778  8554 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:46.476145  1636 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: [15.13-17.14]
W20260709 13:02:46.478330  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 113479us
I20260709 13:02:46.479354  1635 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:46.483887  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 151450us
I20260709 13:02:46.487895  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:46.505489  8298 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:46.506253  8298 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:46.507575  8298 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:46.509898  1525 leader_election.cc:336] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.865s (SENT)
I20260709 13:02:46.510632  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:46.476199  8592 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:46.419054  8149 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:46.525195  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:46.526255  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 1171) took 2039 ms (client timeout 2031 ms). Trace:
I20260709 13:02:46.527465  1704 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:46.530467  1454 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 13 to 14, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
W20260709 13:02:46.548060  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.553169  8654 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:46.555243  8055 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:46.557065  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1614) took 1166 ms. Trace:
I20260709 13:02:46.607678  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:46.602157  1701 rpcz_store.cc:269] 0709 13:02:44.487077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.488305 (+  1228us) service_pool.cc:224] Handling call
0709 13:02:46.525945 (+2037640us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b54001865d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:46.526141 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4317057280}
I20260709 13:02:46.563004  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:46.565827  2344 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:02:46.566787  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:46.568203  2347 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:46.570766  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:46.571954  8549 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:46.574450  8523 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:46.575378  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.499464  8706 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:02:46.609817  2344 pending_rounds.cc:85] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 11
I20260709 13:02:46.509315  8298 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:46.584180  8752 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:46.625036  2345 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:02:46.597671  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.625675  2344 pending_rounds.cc:107] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:02:46.626008  2344 raft_consensus.cc:2889] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:46.627408  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:46.604820  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 151470us
I20260709 13:02:46.609112  2349 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:46.615713  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2521) took 1748 ms. Trace:
I20260709 13:02:46.640033  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.488791  8573 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.138s
I20260709 13:02:46.640925  1631 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:46.631850  1704 rpcz_store.cc:276] 0709 13:02:44.867693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.264427 (+396734us) service_pool.cc:224] Handling call
0709 13:02:46.527044 (+1262617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.603091 (+ 76047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.615372 (+ 12281us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001865d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:46.615628 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2900532992}
W20260709 13:02:46.641268  1633 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.052s	sys 0.100s
I20260709 13:02:46.607617  2348 rpcz_store.cc:276] 0709 13:02:45.390487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.390748 (+   261us) service_pool.cc:224] Handling call
0709 13:02:45.391305 (+   557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.391986 (+   681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.392021 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.392663 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.485233 (+ 92570us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:45.485276 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.485349 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.207627 (+722278us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:46.556450 (+348823us) raft_consensus.cc:1685] finished
0709 13:02:46.556483 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.556803 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1797,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":211,"spinlock_wait_cycles":16055552,"thread_start_us":91928,"threads_started":1,"wal-append.queue_time_us":172881}
I20260709 13:02:46.643065  1635 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:46.642779  2350 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
W20260709 13:02:46.644309  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 92016.6us
I20260709 13:02:46.616279  8055 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:46.520848  8474 consensus_queue.cc:1059] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
I20260709 13:02:46.650542  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.647856  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:46.650900  1633 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.053s	sys 0.100s
I20260709 13:02:46.618063  8616 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:46.660831  1630 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:02:46.665189  1630 pending_rounds.cc:85] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 12
I20260709 13:02:46.665515  1630 pending_rounds.cc:107] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:02:46.665786  1630 raft_consensus.cc:2889] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:46.635273  8654 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:46.632933  8526 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:46.669497  1635 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:46.660162  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:46.662510  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 137404us
I20260709 13:02:46.526307  8635 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.122s
I20260709 13:02:46.675439  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3444) took 1030 ms. Trace:
I20260709 13:02:46.685282  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3445) took 1047 ms. Trace:
I20260709 13:02:46.690188  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:46.667542  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.018s	sys 0.030s
W20260709 13:02:46.686151  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 165686us
W20260709 13:02:46.691310  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.019s	sys 0.030s
I20260709 13:02:46.668417  8523 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:46.684381  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:46.659857  8441 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:46.688361  1630 rpcz_store.cc:276] 0709 13:02:45.637810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.949613 (+311803us) service_pool.cc:224] Handling call
0709 13:02:45.981501 (+ 31888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.675681 (+694180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.680078 (+  4397us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b540010d6d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:46.685189 (+  5111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1555870592}
W20260709 13:02:46.706331  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.690479  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:46.699740  1633 rpcz_store.cc:276] 0709 13:02:45.637507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.908898 (+271391us) service_pool.cc:224] Handling call
0709 13:02:45.923980 (+ 15082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.667112 (+743132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.667151 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.667202 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:46.667238 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.667312 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.667450 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21257,"raft.queue_time_us":217606,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":28631,"thread_start_us":689631,"threads_started":1}
I20260709 13:02:46.668701  8767 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:46.674965  8704 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
W20260709 13:02:46.698993  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.677873  8549 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:46.721735  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:46.709949  8766 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:02:46.695926  8526 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:46.741688  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 231732us
W20260709 13:02:46.422715  1456 rpcz_store.cc:269] 0709 13:02:23.623329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.623713 (+   384us) service_pool.cc:224] Handling call
0709 13:02:24.333993 (+710280us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b5400179b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27644090 00007fda276e2f3a 00007fda2d4991cc 00007fda2d59737e 00007fda2ba582a4 00007fda2ba58232 00007fda2ba58161 00007fda2ba580dc 00007fda2ba57392 00007fda25a1548c 00007fda25a148c6
0709 13:02:46.279280 (+21945287us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:42.104692 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:42.168961 (+ 64269us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:42.169042 (+    81us) write_op.cc:435] Acquired schema lock
0709 13:02:42.169070 (+    28us) tablet.cc:662] Decoding operations
0709 13:02:42.237738 (+ 68668us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:42.260705 (+ 22967us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:42.272502 (+ 11797us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:02:42.367971 (+ 95469us) tablet.cc:701] Row locks acquired
0709 13:02:42.368047 (+    76us) write_op.cc:260] PREPARE: finished
0709 13:02:42.368363 (+   316us) write_op.cc:270] Start()
0709 13:02:42.368597 (+   234us) write_op.cc:276] Timestamp: P: 1783602162368261 usec, L: 0
0709 13:02:42.368633 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:02:42.495300 (+126667us) log.cc:844] Serialized 9713 byte log entry
0709 13:02:43.725196 (+1229896us) op_driver.cc:464] REPLICATION: finished
0709 13:02:44.087314 (+362118us) write_op.cc:301] APPLY: starting
0709 13:02:44.087496 (+   182us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:44.799722 (+712226us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:44.799890 (+   168us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:44.971531 (+171641us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:44.971874 (+   343us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=0,mrs_lookups=0
0709 13:02:44.971904 (+    30us) write_op.cc:312] APPLY: finished
0709 13:02:44.972320 (+   416us) log.cc:844] Serialized 204 byte log entry
0709 13:02:44.974800 (+  2480us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:44.975067 (+   267us) write_op.cc:454] Released schema lock
0709 13:02:44.975186 (+   119us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:44.975761 (+   575us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:44.975800 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5073237,"reported_tablets":36,"spinlock_wait_cycles":14631296,"child_traces":[["op",{"apply.queue_time_us":322727,"apply.run_cpu_time_us":41641,"apply.run_wall_time_us":888480,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100267,"num_ops":23,"prepare.queue_time_us":768428,"prepare.run_cpu_time_us":112987,"prepare.run_wall_time_us":793141,"raft.queue_time_us":335216,"raft.run_cpu_time_us":59173,"raft.run_wall_time_us":407878,"replication_time_us":1331296,"thread_start_us":1044842,"threads_started":5,"wal-append.queue_time_us":489431,"wal-append.run_cpu_time_us":24932,"wal-append.run_wall_time_us":2545330}]]}
I20260709 13:02:46.756124  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.757742  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:46.758402  1631 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:46.758730  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.759567  1635 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:46.760767  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:46.763135  1635 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:46.776714  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:46.730142  8767 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:46.797459  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.798755  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.805855  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:46.809592  1704 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:46.816275  8760 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.027s	sys 0.030s
I20260709 13:02:46.793658  8149 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:46.756513  8381 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:46.824477  1633 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:46.788262  8764 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:02:46.826566  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.830655  1703 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:46.837528  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:46.838778  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 148346us
I20260709 13:02:46.789114  8664 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:46.833109  1706 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 20. Current term is 21. Ops: [20.13-20.13]
W20260709 13:02:46.822008  8760 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.028s	sys 0.030s
I20260709 13:02:46.836856  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:46.851895  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:46.858211  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 81546.7us
I20260709 13:02:46.723721  8523 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:46.862085  1709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.000s	sys 0.023s
W20260709 13:02:46.873253  1709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.000s	sys 0.024s
I20260709 13:02:46.741940  8441 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:46.868686  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.863118  1629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.009s	sys 0.017s
W20260709 13:02:46.873667  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 395764us
W20260709 13:02:47.020694  1629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.018s	sys 0.029s
W20260709 13:02:47.022801  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.872542  8383 raft_consensus.cc:1027] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: attempt to promote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:02:46.864523  8389 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:47.017534  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 146645us
I20260709 13:02:46.886163  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:47.018257  1705 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
W20260709 13:02:47.018350  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.926476  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:46.876062  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.027457  1602 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:47.030911  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.020161  1675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.748s (SENT) [suppressed 40 similar messages]
I20260709 13:02:46.821120  8055 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.046793  1704 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:47.040503  1631 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:47.020151  8764 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 21 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:47.044968  8523 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:46.853653  8381 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.056020  1602 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:47.063162  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.018734  8312 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:02:47.043979  8454 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.706s	user 0.018s	sys 0.003s
W20260709 13:02:47.054811  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0677521599999977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
W20260709 13:02:47.084815  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0110521599999629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
I20260709 13:02:47.076637  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:47.077697  1552 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 13:02:47.070581  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 218719us
I20260709 13:02:47.041349  8635 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:47.090874  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.090744  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:47.075762  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:47.072190  1674 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:46.782864  1454 catalog_manager.cc:5697] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 6 to 15, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:47.099131  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:47.092128  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:47.050350  8389 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:47.098857  1627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.069s	sys 0.060s
W20260709 13:02:47.126132  1627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.075s	sys 0.060s
W20260709 13:02:47.106024  2317 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.003s	sys 0.052s
W20260709 13:02:47.128487  1634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.025s	sys 0.035s
W20260709 13:02:47.128973  1634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.026s	sys 0.035s
W20260709 13:02:47.128548  2317 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.003s	sys 0.052s
W20260709 13:02:47.131873  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 608713us
I20260709 13:02:47.068873  8149 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:47.133088  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:47.135848  1454 catalog_manager.cc:5697] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 4 to 12, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:47.142831  1633 raft_consensus.cc:2468] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:02:47.131219  1526 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:47.071028  8312 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:47.056697  8387 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:02:47.146836  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3449) took 1502 ms. Trace:
I20260709 13:02:47.153141  1630 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:47.156622  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:47.078202  8704 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:02:47.109344  8454 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:47.065852  8526 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.139945  8149 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.192006  8770 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.185s
I20260709 13:02:47.250108  8149 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.191536  8765 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:47.251468  8765 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.251936  8765 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:47.192339  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 522253us
I20260709 13:02:47.254652  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:47.199306  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:47.255698  1554 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:47.257280  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:47.257957  1554 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:47.198465  8785 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:02:47.180265  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.267131  1674 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:47.231654  2345 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.002s
I20260709 13:02:47.268002  2345 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 24.
I20260709 13:02:47.138549  8698 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:47.131167  8517 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.154517  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:47.270234  8785 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:47.112144  8055 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:47.205559  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:47.173364  1552 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.171227  1627 rpcz_store.cc:276] 0709 13:02:45.644535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.037038 (+392503us) service_pool.cc:224] Handling call
0709 13:02:46.043638 (+  6600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.168091 (+124453us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:46.385686 (+217595us) log.cc:844] Serialized 14 byte log entry
0709 13:02:47.146324 (+760638us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:47.146406 (+    82us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:47.146476 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.146580 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.146751 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5025,"prepare.queue_time_us":29918,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"thread_start_us":666065,"threads_started":1}
I20260709 13:02:47.292009  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:47.208015  1557 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:47.173328  8769 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.454s	user 0.002s	sys 0.000s
I20260709 13:02:47.191095  1454 catalog_manager.cc:5697] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 14 to 15, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 15 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:47.211084  8764 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:47.246896  8454 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:47.247887  8664 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:02:47.265275  8227 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.870s	user 0.009s	sys 0.006s
W20260709 13:02:47.270594  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 449419us
I20260709 13:02:47.293774  8573 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:47.222951  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.114012  8389 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:47.150393  1555 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
W20260709 13:02:47.234831  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 165790us
I20260709 13:02:47.292778  1630 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.292747  1704 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 24.
I20260709 13:02:47.208036  8635 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:47.195515  8381 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:47.198740  8441 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:47.207582  8690 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.420653  8690 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:47.407778  8454 raft_consensus.cc:2804] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:02:47.329501  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.271360  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 93993.4us
I20260709 13:02:47.423095  8690 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:47.423461  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:47.423703  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2543) took 1302 ms. Trace:
I20260709 13:02:47.424886  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:47.396042  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.206833  8767 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:47.425395  8055 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:47.395884  1602 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:47.426332  8772 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 13:02:47.427634  2345 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:47.427846  8779 raft_consensus.cc:3055] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:47.428231  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1191) took 1210 ms. Trace:
I20260709 13:02:47.428325  8779 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:47.409114  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.405012  8654 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.728s	user 0.013s	sys 0.000s
I20260709 13:02:47.193897  8781 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.207s
I20260709 13:02:47.412389  8664 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.400309  8227 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.419500  8149 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:47.421337  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 166684us
I20260709 13:02:47.415100  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:47.388126  2353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.045s	sys 0.061s
W20260709 13:02:47.479558  2353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.046s	sys 0.061s
W20260709 13:02:47.448156  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.430356  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:47.438476  1628 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:47.400772  8573 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.424122  1560 rpcz_store.cc:276] 0709 13:02:46.121109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.123027 (+  1918us) service_pool.cc:224] Handling call
0709 13:02:46.130630 (+  7603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:46.130958 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.130995 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:46.131793 (+   798us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.412262 (+280469us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:46.432064 (+ 19802us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.449895 (+ 17831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.449991 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.423479 (+973488us) raft_consensus.cc:1685] finished
0709 13:02:47.423503 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.423639 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51231,"prepare.queue_time_us":19390,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":78,"raft.queue_time_us":29541,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":16087,"thread_start_us":279999,"threads_started":1,"wal-append.queue_time_us":533792}
I20260709 13:02:47.428718  1709 rpcz_store.cc:276] 0709 13:02:46.217590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.234622 (+ 17032us) service_pool.cc:224] Handling call
0709 13:02:46.235398 (+   776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:46.235913 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:46.236381 (+   468us) log.cc:844] Serialized 14 byte log entry
0709 13:02:47.017860 (+781479us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:47.030761 (+ 12901us) log.cc:844] Serialized 26 byte log entry
0709 13:02:47.030993 (+   232us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:47.031041 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.031163 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.428022 (+396859us) raft_consensus.cc:1685] finished
0709 13:02:47.428044 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.428173 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":143687,"prepare.queue_time_us":246,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":167479,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":21658,"thread_start_us":637161,"threads_started":1,"wal-append.queue_time_us":901146}
I20260709 13:02:47.382469  8517 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:47.483829  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 74734.4us
W20260709 13:02:47.490465  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.437114  8441 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:02:47.491568  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 358360us
I20260709 13:02:47.449703  8454 raft_consensus.cc:697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:47.432731  8773 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:47.429828  8635 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:47.518239  1561 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:47.519685  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:47.486819  8573 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:47.521306  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 91012.7us
I20260709 13:02:47.523589  1552 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:47.437044  8389 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:47.452021  8381 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
W20260709 13:02:47.520665  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 120988us
I20260709 13:02:47.533229  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:47.492796  8387 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:47.539696  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.512348  8767 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:47.416531  8312 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.540997  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:47.429445  8779 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.514778  8441 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 24 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:47.540467  8786 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:47.546494  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 415156us
I20260709 13:02:47.509200  8785 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.551043  8389 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:47.540747  8387 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:47.551822  8387 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:47.553574  8784 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:47.554188  8149 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:47.554756  8780 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 13:02:47.555423  8782 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:02:47.546272  8698 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:47.561965  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:47.565322  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 58429.7us
I20260709 13:02:47.544739  8055 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:47.548631  8653 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.183s	user 0.010s	sys 0.008s
I20260709 13:02:47.571398  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:47.574905  1635 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:47.569999  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:47.576398  2344 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:47.578495  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 59015.3us
I20260709 13:02:47.578225  8698 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:47.579843  8517 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:47.579926  2317 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:47.540180  1454 catalog_manager.cc:5697] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:47.581080  8794 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:47.581658  8794 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.582168  8794 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:47.582782  8664 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:47.583503  8227 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:47.585844  8786 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:47.586508  8786 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:47.587044  8298 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.954s	user 0.008s	sys 0.006s
I20260709 13:02:47.587765  8786 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.573693  8781 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:47.588531  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1297) took 1258 ms. Trace:
I20260709 13:02:47.575713  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:47.565814  8312 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.556955  8441 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.543068  8635 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.597898  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:02:47.615568  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.601583  2317 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:47.600026  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:47.596990  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 207487us
W20260709 13:02:47.652484  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 54656.1us
I20260709 13:02:47.618537   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:47.596257  1632 rpcz_store.cc:276] 0709 13:02:46.330491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.424665 (+ 94174us) service_pool.cc:224] Handling call
0709 13:02:46.452396 (+ 27731us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:46.481108 (+ 28712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.481163 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:46.577293 (+ 96130us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b2c00070338. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:02:46.651255 (+ 73962us) log.cc:844] Serialized 26 byte log entry
0709 13:02:47.118389 (+467134us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:47.118455 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.118566 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.588114 (+469548us) raft_consensus.cc:1685] finished
0709 13:02:47.588157 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.588458 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101133,"raft.queue_time_us":12995,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":10918,"spinlock_wait_cycles":15446912,"thread_start_us":441298,"threads_started":1,"wal-append.queue_time_us":704795}
W20260709 13:02:47.658492  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61593.6us
W20260709 13:02:47.607494  8453 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.183s	user 0.008s	sys 0.008s
I20260709 13:02:47.663072  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:47.600700  8706 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.976s	user 0.011s	sys 0.004s
I20260709 13:02:47.657663  1630 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
W20260709 13:02:47.657043  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 90721.8us
I20260709 13:02:47.601073  8779 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:47.595742  8381 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:47.603121  8227 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:02:47.687956  8381 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:47.688499  8381 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:47.689374  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:47.593322  8592 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.992s	user 0.009s	sys 0.005s
I20260709 13:02:47.667594  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1296) took 1333 ms. Trace:
I20260709 13:02:47.598582  8783 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:47.595059  1454 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 12 to 13, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 13 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:47.595310  8454 consensus_queue.cc:237] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:47.677848  8616 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:47.680238  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:47.682581  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:47.692049  1628 rpcz_store.cc:276] 0709 13:02:46.330121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.419580 (+ 89459us) service_pool.cc:224] Handling call
0709 13:02:47.662716 (+1243136us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b5400010ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:47.662890 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2241054464}
I20260709 13:02:47.704433  1701 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
W20260709 13:02:47.694377  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.695051  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 146643us
I20260709 13:02:47.659580  8793 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.707759  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:47.696993  8764 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:47.689821  8791 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:02:47.699834  1702 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:02:47.700505  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:47.704008  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:47.703562  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:47.707418  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:47.707813  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 107852us
I20260709 13:02:47.713371  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:47.714937  1561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 1111) took 1350 ms (client timeout 1525 ms). Trace:
I20260709 13:02:47.716341  1707 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:02:47.718171  1561 rpcz_store.cc:269] 0709 13:02:46.357925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.362154 (+  4229us) service_pool.cc:224] Handling call
0709 13:02:47.707797 (+1345643us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001ce154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:47.707958 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2567940992}
I20260709 13:02:47.721843  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:47.701426  8706 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19.
I20260709 13:02:47.704236  8526 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.730382  8790 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:47.730505  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:47.731251  1704 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:47.733808  8526 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.708637  8781 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:47.742599  1704 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:47.739962  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:47.727552  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:47.749521  2344 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:47.711792  8784 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:47.714537  8616 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:47.750592  8592 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:47.692395  8804 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:02:47.753276  8561 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:47.753902  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 68480.7us
I20260709 13:02:47.756367  1602 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:47.746682  8664 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:47.769241  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61552.2us
I20260709 13:02:47.759644  8767 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:47.770094  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:47.771329  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:47.756201  8592 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:47.765030  8764 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:47.770177  8752 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.125s	user 0.012s	sys 0.003s
I20260709 13:02:47.762952  8807 raft_consensus.cc:2764] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
W20260709 13:02:47.775844  8549 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.033s	user 0.006s	sys 0.010s
I20260709 13:02:47.770747  8808 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:47.775868  8592 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:47.797731  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1435) took 1224 ms. Trace:
W20260709 13:02:47.803632  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.806136  1709 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:47.806459  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:47.808125  8635 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:47.808951  8635 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.798082  8706 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:47.810930  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 48221.8us
I20260709 13:02:47.812338  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:47.810527  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1437) took 1246 ms. Trace:
I20260709 13:02:47.800137  2352 rpcz_store.cc:276] 0709 13:02:46.563614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.568709 (+  5095us) service_pool.cc:224] Handling call
0709 13:02:46.569226 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:47.047565 (+478339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:47.047611 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:47.068472 (+ 20861us) log.cc:844] Serialized 26 byte log entry
0709 13:02:47.400983 (+332511us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:47.401029 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.401118 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.787331 (+386213us) raft_consensus.cc:1685] finished
0709 13:02:47.787358 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.787516 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":628493,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"thread_start_us":623070,"threads_started":2,"wal-append.queue_time_us":435106}
I20260709 13:02:47.808080  1525 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:47.778146  8767 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:47.800324  8790 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:47.798715  8704 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.800330  8773 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.789279  8453 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:47.811213  8809 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.813815  2353 rpcz_store.cc:276] 0709 13:02:46.564540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.636049 (+ 71509us) service_pool.cc:224] Handling call
0709 13:02:46.678512 (+ 42463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:47.480126 (+801614us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:47.480179 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:47.492823 (+ 12644us) log.cc:844] Serialized 26 byte log entry
0709 13:02:47.595359 (+102536us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:47.595418 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.595521 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.810253 (+214732us) raft_consensus.cc:1685] finished
0709 13:02:47.810280 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.810455 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25111,"raft.queue_time_us":522662,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7282,"thread_start_us":804924,"threads_started":2,"wal-append.queue_time_us":184000}
W20260709 13:02:47.826066  8802 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:02:47.821043  8706 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:47.835639  1710 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:02:47.823180  8792 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:47.809449  8381 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.848558  1706 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:47.829262  8790 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.838786  8592 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:47.854679  8381 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:47.855382  8592 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:47.831499  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:47.829041  8664 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:47.856887  8664 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:47.857399  8664 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:47.827313  8616 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:47.853062  8810 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:02:47.864431  8573 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.865917  8767 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:47.867825  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.844746  8801 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.131s	user 0.001s	sys 0.001s
I20260709 13:02:47.849915  8792 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:47.817899  8784 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.905329  8793 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:47.877720  8790 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:47.906802  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 57989.7us
I20260709 13:02:47.922499  8573 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:47.922945  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:47.924736  1601 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:47.924851  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 96537.3us
W20260709 13:02:47.925747  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 228835us
I20260709 13:02:47.926668  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:47.927091  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:47.927687  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:47.844884  8453 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.843391  8809 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.842916  8773 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.936446  8812 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:47.921700  1454 catalog_manager.cc:5697] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 12 to 16, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:47.830744  8704 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:47.906809  8807 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:02:47.928202  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 120162us
I20260709 13:02:47.914206  8792 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.933570  8441 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:02:47.935418  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.935773  8809 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:47.930054  8765 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.678s	user 0.016s	sys 0.000s
I20260709 13:02:47.945652  8573 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:47.949798  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.936962  8812 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:47.946730  8814 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:02:47.957993  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.931599  8149 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:47.961267  8812 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:47.977073  8549 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:47.966334  8809 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:47.978155  8809 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.978749  8809 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.980930  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:47.965565  8698 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:47.959379  8381 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:02:47.982810  8793 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:47.983340  8381 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:47.983443  1525 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:47.983872  8381 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:47.984930  8381 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.977392  8781 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:47.970466  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:47.992978  1675 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:47.997498  8664 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.959468  8765 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.000016  8664 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:47.992848  8149 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:48.001081  8149 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:47.946820  1454 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:48.004011  8149 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:47.997877  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:48.005826  1703 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:48.005739  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:48.002928  1454 catalog_manager.cc:5697] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: config changed from index 2 to 3, term changed from 15 to 11, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194), VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 11 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:02:48.004316  8387 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.011813  1454 catalog_manager.cc:5697] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
W20260709 13:02:48.040232  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 112593us
I20260709 13:02:48.046310  2317 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:48.042244  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:47.995802  8781 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:47.995357  8792 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:48.049400  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 64051.6us
I20260709 13:02:47.995908  8793 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:48.050316  1703 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:48.049541  8664 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:48.040848  8441 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:48.039901  8767 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.056515  8573 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:48.064175  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 106153us
I20260709 13:02:48.039566  8387 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:48.046555  8381 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:48.047658  8773 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:48.059837  8453 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:48.078178  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:48.081066  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 88199.5us
I20260709 13:02:48.063396  8765 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:48.082911  2352 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:48.074575  8635 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:48.086843  8788 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:48.087519  8788 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:48.065814  8786 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:02:48.091197  8822 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:48.091653  8822 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:48.077992  8812 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.092595  8822 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.094702  8812 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:48.053649  8794 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.098291  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:48.099797  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:48.100456  1703 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:48.101063  8704 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:48.088963  8788 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.046900  1454 catalog_manager.cc:5697] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 8 to 12, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 12 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:48.100965  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:48.073328  8793 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:48.102329  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.103216  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:48.076399  8767 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.103991  8765 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:48.098342  8794 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:48.080192  8549 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:48.106189  8788 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:48.139271  8790 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:48.121954  8819 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:02:48.138669  8793 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.069772  8664 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:48.142465  8312 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.535s	user 0.012s	sys 0.002s
I20260709 13:02:48.143450  8664 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:48.137418  8549 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:48.153579  8549 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.138068  8771 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:48.174435  2353 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:02:48.164026  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3456) took 1470 ms. Trace:
W20260709 13:02:48.164945  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.155448  1710 raft_consensus.cc:1240] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
W20260709 13:02:48.165385  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2539) took 1328 ms (client timeout 1639 ms). Trace:
I20260709 13:02:48.159338  8785 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:48.172175  8765 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:48.127207  1703 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
W20260709 13:02:48.178989  1706 rpcz_store.cc:269] 0709 13:02:46.855434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.855653 (+   219us) service_pool.cc:224] Handling call
0709 13:02:48.165202 (+1309549us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b540016ba54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:48.165324 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1649583872}
W20260709 13:02:48.222543  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 60115us
I20260709 13:02:48.176011  1635 rpcz_store.cc:276] 0709 13:02:46.693867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.764182 (+ 70315us) service_pool.cc:224] Handling call
0709 13:02:46.764527 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.575361 (+810834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.156846 (+581485us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b540017c854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:48.163954 (+  7108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1861504256}
W20260709 13:02:48.227699  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 126722us
I20260709 13:02:48.232514  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:48.126036  8764 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:48.138440  8812 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:48.227852  1602 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:48.264519  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.265632  1674 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:48.267064  1675 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:48.267879  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 164697us
I20260709 13:02:48.270810  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:48.136350  1708 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:48.161878  8790 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:48.277127  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:48.270643  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:48.276527  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:48.172788  8771 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:48.170660  1454 catalog_manager.cc:5697] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 9 to 12, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:02:48.229421  8765 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:48.271160  8383 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:48.286429  8794 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:48.272956  8807 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:48.285292  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:48.248384  8822 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:48.285499  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 358233us
I20260709 13:02:48.246147  8616 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:48.270370  8706 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.310686  2350 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:02:48.250648  8312 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:02:48.274484  8704 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:48.274075  8752 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:48.312543  2344 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:48.332172  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:48.280344  8792 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:02:48.317106  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46779.4us
I20260709 13:02:48.338533  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:48.307075  8765 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:48.277959  8793 consensus_queue.cc:1059] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:02:48.341293  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:48.342227  8592 consensus_queue.cc:1059] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:48.343384  1454 catalog_manager.cc:5697] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 4 to 12, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
W20260709 13:02:48.344962  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.307935  8526 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:48.275045  8812 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:48.345736  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 70466.5us
I20260709 13:02:48.304566  8387 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:02:48.330336  8704 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.318018  8794 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:48.307214  8809 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.304808  8664 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
W20260709 13:02:48.383373  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51273.6us
W20260709 13:02:48.385564  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44360.8us
W20260709 13:02:48.386276  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.317775  8804 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:02:48.363629  8149 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:48.354530  8635 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:48.391470  2352 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:02:48.393024  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:48.396432  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.389056  8807 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.397830  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:48.328994  8706 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.315125  8227 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:02:48.381681  8704 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:48.400553  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:48.398711  1635 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:48.401041  1630 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:48.380587  8698 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:48.432937  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:48.396216  2352 pending_rounds.cc:85] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 10
I20260709 13:02:48.433826  2352 pending_rounds.cc:107] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:02:48.434346  2352 raft_consensus.cc:2889] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:48.437291  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:48.438048  1631 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:48.439896  8794 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.383561  8792 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:48.355366  8752 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.403184  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:48.447203  1633 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 22: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:48.399313  8804 consensus_queue.cc:1243] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 20.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:02:48.395706  8764 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:48.469736  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:48.452900  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:48.453868  8767 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:48.403967  8592 consensus_queue.cc:1243] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 13:02:48.460662  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 142303us
I20260709 13:02:48.485797  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:48.476596  1631 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:48.403308  8773 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.482620  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 80914us
I20260709 13:02:48.490273  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:48.470744  2350 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:48.483081  1630 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:48.483188  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 96831.7us
W20260709 13:02:48.488487  8821 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.441s	user 0.004s	sys 0.009s
I20260709 13:02:48.494760  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:48.502712  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.503026  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:48.507908  2352 raft_consensus.cc:2368] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17.
I20260709 13:02:48.511200  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:48.514611  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.469230  8764 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.359617  8771 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.489280  8773 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.524662  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:48.513497  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:48.494326  8149 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:48.449295  8781 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:48.531561  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 264356us
I20260709 13:02:48.528283  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:48.521288  8764 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:48.520187  8698 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:48.504369  8804 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 13:02:48.520334  8812 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:48.523315  8767 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:02:48.553368  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 63132.5us
I20260709 13:02:48.509207  8664 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:02:48.560946  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:48.565603  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 54471.8us
W20260709 13:02:48.570295  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 45700.1us
W20260709 13:02:48.565009  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47083us
I20260709 13:02:48.565101  8616 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:48.528855  8839 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:48.571107  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:48.538905  8825 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.495s	user 0.014s	sys 0.000s
I20260709 13:02:48.576992  8706 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:48.578830  8773 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:48.540050  8836 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:48.558597  1454 catalog_manager.cc:5697] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:48.588577  8635 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:48.589136  8635 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:48.582388  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:48.591416  8828 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.322s	user 0.000s	sys 0.011s
I20260709 13:02:48.558915  8829 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:48.522078  8549 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:02:48.558136  8817 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:48.599721  8812 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:48.600073  8817 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:48.600647  8817 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:48.554764  8698 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.558385  8592 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:02:48.547482  8792 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.578881  8149 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:48.596176  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:48.630884  1552 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:48.573918  8616 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.583168  8771 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:48.617731  8752 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:48.622720  8781 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:48.624882  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 53829.1us
W20260709 13:02:48.626896  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.680481  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:48.681346  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:48.681766  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:48.615852  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:48.683758  1557 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:48.570010  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:48.638753  8809 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:48.639032  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:48.639441  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:48.685982  1702 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:48.639997  8767 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:48.646620  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:48.650588  8840 raft_consensus.cc:3055] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:48.702275  8840 raft_consensus.cc:740] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:48.702430  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:48.702028  8635 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.703238  8840 consensus_queue.cc:260] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.703830  8387 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:48.703882  8843 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:48.704561  8840 raft_consensus.cc:3060] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:48.705364  8387 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:48.705816  8387 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:48.611357  8698 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:48.673086  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.674940  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:48.679804  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:48.626852  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54988.8us
W20260709 13:02:48.689525  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 119494us
I20260709 13:02:48.694244  8592 raft_consensus.cc:1064] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:48.694952  1553 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
W20260709 13:02:48.695079  8831 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.432s	user 0.002s	sys 0.000s
I20260709 13:02:48.695639  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:48.696219  8765 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.697196  8653 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:48.700059  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:48.703603  8635 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:48.608924  8836 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:48.682621  1556 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:48.712145  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:48.771679  1706 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:48.670539  8149 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:48.672487  8616 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.779376  8592 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:48.782217  8381 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.704s	user 0.006s	sys 0.007s
I20260709 13:02:48.784560  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:48.716895  8835 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:48.786541  8835 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.787694  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:48.721060  8526 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:02:48.791302  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 277735us
I20260709 13:02:48.726279  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:48.793613  1559 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:48.716234  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 450102us
I20260709 13:02:48.754972  1702 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
W20260709 13:02:48.772908  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.774580  2353 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:48.775125  1525 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:48.775365  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:48.797850  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:48.702030  8664 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:48.777858  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:48.806910  1709 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:48.778671  8149 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:48.774333  8383 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.844198  1527 leader_election.cc:400] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17.
I20260709 13:02:48.716913  8767 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:48.788396  1552 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:48.785388  1557 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:48.847038  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:48.850044  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:48.725543  1560 raft_consensus.cc:2393] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 19: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:48.805409  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:48.851191  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:48.847586  1527 leader_election.cc:403] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:48.852370  1633 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:48.805511  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:48.857149  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:48.812770  1702 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:02:48.855091  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.818161  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.706815  8387 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.840602  8804 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:48.862762  1675 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:48.787871  2345 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEARNER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:48.863992  2352 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:48.865187  1636 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:48.857857  1628 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:48.791327  8526 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:48.807345  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:48.871536  1631 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.806152  8381 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:48.773316  8653 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.868013  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:48.844041  1706 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:48.883420  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:48.887934  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:48.883976  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 124870us
I20260709 13:02:48.885944  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:48.890779  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:48.850109  8793 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:48.845331  8849 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:48.846084  8843 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.887316  2349 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
W20260709 13:02:48.895205  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.854805  8664 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:48.898792  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:48.912695  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:48.889484  8785 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.660s	user 0.013s	sys 0.000s
I20260709 13:02:48.901043  1710 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:02:48.847697  8839 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:48.901167  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:48.862071  8454 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:48.887490  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:48.844937  8752 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 13:02:48.913061  1702 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:02:48.923409  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:48.924271  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:48.926987  2345 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:48.927146  1552 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:48.900168  8773 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.928673  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:48.900256  8561 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:48.947105  2349 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:48.929850  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:48.857632  8383 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.950567  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:48.911725  8453 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:48.918396  1560 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:48.888981  8804 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.969877  1552 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:48.965984  1632 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:48.924842  8809 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.922497  8822 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:48.979897  2348 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:48.894333  8381 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:48.931528  8793 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:48.998778  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.006937  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:48.912263  8849 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:49.014655  1631 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:02:48.915803  8764 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:49.016772  1634 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:02:48.962592  8835 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:48.993646  8828 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:48.968526  8561 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:49.004024  8834 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.744s	user 0.007s	sys 0.014s
I20260709 13:02:49.017257  8853 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:48.928493  8474 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 13:02:48.993760  8704 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.529s	user 0.012s	sys 0.000s
I20260709 13:02:49.025068  1556 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:49.052117  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:49.022125  8794 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:48.923406  8843 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.995289  8822 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:49.017449  8832 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.749s	user 0.005s	sys 0.000s
I20260709 13:02:49.049847  1557 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:49.051425  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:48.932489  8454 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:49.104276  8454 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:49.021548  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:49.105603  1701 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:49.089061  8817 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:49.110984  8839 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:49.113193  8817 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:49.116436  1561 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:02:49.114655  1559 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:48.963688  8773 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:49.060164  1636 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:49.069728  1632 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:49.118817  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:49.072968  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:48.993304  8767 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:49.121381  8764 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:49.070189  8822 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:49.066439  8664 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:48.973529  8836 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:49.021610  8526 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:49.126528  8526 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:49.083559  8812 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:49.067257  8704 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:49.073316  8381 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:49.089986  8851 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:02:49.090401  8561 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:49.168483  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:49.169441  1559 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 15.
I20260709 13:02:49.090941  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:49.097409  8856 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.099089  8843 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:49.066754  8765 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:49.100386  8804 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:49.067171  8848 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:02:49.119515  8793 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:49.082000  8849 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:49.072678  8794 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:02:49.129566  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.071663  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:49.182945  2344 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:49.004962  8809 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:49.137282  1631 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:02:49.144397  1560 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:49.186621  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:49.187372  1560 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:49.188372  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:49.119120  8773 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:49.089053  8698 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:49.185557  1630 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:02:49.192945  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:49.196663  1560 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:49.198037  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:49.200176  1560 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:02:49.202407  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:49.202983  1559 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:49.209864  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:49.202196  8854 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:02:49.181980  8794 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:49.224047  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 363782us
I20260709 13:02:49.221930  1559 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:49.228982  1635 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:02:49.175041  8856 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:49.229451  1635 pending_rounds.cc:85] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 8
I20260709 13:02:49.229947  1635 pending_rounds.cc:107] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 16.9
W20260709 13:02:49.230059  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 367130us
I20260709 13:02:49.230348  1635 raft_consensus.cc:2889] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:49.231523  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 280318us
I20260709 13:02:49.237723  1559 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:49.238174  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:49.240909  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:49.242049  2344 raft_consensus.cc:2368] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21.
I20260709 13:02:49.238828  2345 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:49.210134  8653 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:49.245457  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:49.251046  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:49.254841  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.256699  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:49.263360  1628 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:49.265002  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:49.266003  1628 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:49.245867  8860 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:49.257136  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:49.274056  1527 leader_election.cc:400] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21.
I20260709 13:02:49.274505  1527 leader_election.cc:403] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:49.270227  1632 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:49.253712  8383 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:49.282214  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:49.285542  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:49.245445  8773 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
W20260709 13:02:49.297171  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 376972us
W20260709 13:02:49.288940  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.267031  8765 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:49.351296  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:49.348150   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:49.352216  1634 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
W20260709 13:02:49.283664  8616 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.000s
W20260709 13:02:49.349572  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64096.6us
I20260709 13:02:49.378959  1601 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:49.343149  8804 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:49.282421  8852 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.382834  8653 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:49.383471  8653 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 22 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:49.384713  8653 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:49.288375  8849 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:02:49.402096  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:49.400302  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:49.402807  1710 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:49.412935  8809 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:49.417351  8453 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:49.418318  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.420650  8453 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:49.421262  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.422807  8453 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:49.422917  8381 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:49.423933  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:49.424456  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.424839  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 191341us
I20260709 13:02:49.425194  8799 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:49.425710  1675 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:49.426146  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:49.427202  8799 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:49.427754  8799 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.427546  8771 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:49.428198  8383 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:49.428752  8843 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:49.429057  8381 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:49.429548  8843 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:49.430251  8843 consensus_queue.cc:1243] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:49.430642  8799 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:49.430543  8381 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:02:49.432186  8381 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:49.436928  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:49.437847  8871 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:49.444061  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:49.445940  1676 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:49.438190  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:49.275943  8860 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:49.467809  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 71974us
I20260709 13:02:49.468142  1676 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:49.473183  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:49.473610  8800 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:49.474278  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:49.474862  1703 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:49.475265  8800 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:49.394156  8878 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.476653  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:49.482434  1702 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:49.392762  8804 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:49.481195  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:49.485790  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:49.437346  8822 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:49.404721  8704 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:49.473678  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:49.494211  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:49.494702  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:49.288489  8794 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:49.443010  8767 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:49.498834  1705 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:49.396394  8765 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 13:02:49.465498  8871 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:49.487942  8872 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:02:49.467061  8752 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:49.508208  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:49.509528  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:49.486603  8804 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:49.503347  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:49.508989  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:49.486897  8800 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:49.393078  8616 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:49.514012  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:49.492651  8792 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:49.491932  8822 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:49.512532  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:49.397011  8852 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:49.525125  1707 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:49.525223  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:49.526033  1702 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:49.506995  8698 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:49.514120  8653 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:49.528954  1559 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:49.531752  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:49.531906  1559 raft_consensus.cc:1240] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:49.533382  1556 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:49.516126  8767 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:49.534925  1706 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:49.517127  8752 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:49.522837  8817 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:49.538949  1705 raft_consensus.cc:1275] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:02:49.542582  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:49.538273  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:49.545861  1707 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:49.546252  8862 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:49.547547  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:49.548043  8880 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:49.492727  8381 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:50.034663  8867 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.675s	user 0.006s	sys 0.007s
I20260709 13:02:50.035013  8381 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:02:50.035367  8876 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.530s	user 0.007s	sys 0.006s
W20260709 13:02:49.601514  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.513705  8664 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
W20260709 13:02:50.019645  8840 consensus_meta.cc:287] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.315s	user 0.025s	sys 0.000s
I20260709 13:02:49.536777  8773 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:49.537940  8752 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:50.018389  8698 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.607416  8441 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.112s
W20260709 13:02:49.862551  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.534291  8704 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:49.542044  8792 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.021576  1556 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:50.038125  8880 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:50.046183  8866 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.730s	user 0.000s	sys 0.013s
I20260709 13:02:50.043541  8561 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.038125  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3250) took 1165 ms. Trace:
I20260709 13:02:50.039924  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:49.493029  8878 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.041715  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:49.551800  8817 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:50.044190  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.017267  8653 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.046244  1554 raft_consensus.cc:1275] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:02:50.041532  8526 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.913s	user 0.013s	sys 0.039s
I20260709 13:02:49.715418  8800 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:50.024099  1710 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.020s
I20260709 13:02:50.099700  1710 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
W20260709 13:02:50.025657  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 487113us
I20260709 13:02:50.046561  8773 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.085642  1709 rpcz_store.cc:276] 0709 13:02:48.872875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.874935 (+  2060us) service_pool.cc:224] Handling call
0709 13:02:48.899706 (+ 24771us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:48.903638 (+  3932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.903665 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:48.916630 (+ 12965us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.247850 (+331220us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:49.247887 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.247945 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.037909 (+789964us) raft_consensus.cc:1685] finished
0709 13:02:50.037931 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.038063 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3554,"raft.queue_time_us":22917,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":22383,"thread_start_us":269751,"threads_started":1,"wal-append.queue_time_us":541187}
I20260709 13:02:50.088506  1559 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:50.088573  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.049849  8765 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:50.054361  8704 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:50.053875  8843 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.124091  1705 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:02:50.071794  8880 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:50.061959  8875 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:50.102288  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:50.109767  8799 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.013s
I20260709 13:02:50.092867  8817 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:50.064070  8698 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.111861  8773 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:50.110831  8765 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.118227  8704 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:50.148912  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 606389us
I20260709 13:02:50.149446  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.150761  1558 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:50.069131  8792 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.160820  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:50.093611  8561 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.140600  8862 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:50.134809  8767 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.171365  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.143281  8616 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.109948  8653 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:50.170707  8888 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.481s	user 0.000s	sys 0.019s
W20260709 13:02:50.180966  8889 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 13:02:50.181339  8868 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.769s	user 0.007s	sys 0.010s
W20260709 13:02:50.182078  8879 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
W20260709 13:02:50.183163  8897 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 13:02:50.184098  8896 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:02:50.207753  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1451) took 1321 ms. Trace:
I20260709 13:02:50.174746  8852 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.208455  8852 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:50.209056  8474 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:50.209439  8474 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:50.209882  8852 consensus_queue.cc:1059] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:50.154667  8765 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:50.210335  8474 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.210923  8561 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.211565  8474 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:50.212921  8561 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.148375  8764 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:50.213857  8764 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.215854  8764 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.216455  8764 raft_consensus.cc:3055] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:50.216781  8764 raft_consensus.cc:740] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:50.153316  8875 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:50.218271  8875 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:50.219071  8781 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.219816  8781 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.136145  8843 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.208176  2352 rpcz_store.cc:276] 0709 13:02:48.886524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.886821 (+   297us) service_pool.cc:224] Handling call
0709 13:02:48.910709 (+ 23888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.911592 (+   883us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:48.928426 (+ 16834us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.044874 (+116448us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.207287 (+1162413us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b1800493f60. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:02:50.207430 (+   143us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:50.207476 (+    46us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:50.207529 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.207593 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.207696 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1138732,"prepare.queue_time_us":45645,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":17057,"raft.queue_time_us":1170373,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":12594,"spinlock_wait_cycles":26069120,"thread_start_us":115899,"threads_started":1,"wal-append.queue_time_us":253238}
I20260709 13:02:50.223141  8843 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.223275  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.224107  1559 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:50.146816  8799 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.225082  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:50.225984  1559 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:50.171192  8855 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.219267  8808 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:50.172562  8862 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:50.227717  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1450) took 1348 ms. Trace:
I20260709 13:02:50.207080  8698 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:50.249014  1559 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
W20260709 13:02:50.234249  8848 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.040s	user 0.015s	sys 0.005s
I20260709 13:02:50.236307  2350 rpcz_store.cc:276] 0709 13:02:48.885730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.886745 (+  1015us) service_pool.cc:224] Handling call
0709 13:02:48.915522 (+ 28777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.915764 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:49.268150 (+352386us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.227358 (+959208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:50.227413 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:50.227482 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.227555 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.227662 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":859691,"prepare.queue_time_us":106113,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"thread_start_us":423890,"threads_started":2,"wal-append.queue_time_us":173032}
I20260709 13:02:50.263765  1559 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
W20260709 13:02:50.237046  8849 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.794s	user 0.013s	sys 0.008s
I20260709 13:02:50.266523  1559 pending_rounds.cc:85] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 11
I20260709 13:02:50.266849  1559 pending_rounds.cc:107] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:02:50.267165  1559 raft_consensus.cc:2889] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.239902  8862 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:50.259793  8149 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.396s	user 0.018s	sys 0.005s
I20260709 13:02:50.250383  8785 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.261740  8807 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.280984  8804 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.749s	user 0.008s	sys 0.022s
I20260709 13:02:50.308072  8653 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:50.309221  8890 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:02:50.306088  8878 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.262231  8840 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:50.325416  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:50.327365  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3325) took 1614 ms. Trace:
I20260709 13:02:50.329538  2352 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:50.316411  1676 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.840s (SENT)
I20260709 13:02:50.329102  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:50.328953  8799 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:50.233801  8561 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.282719  8849 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21.
I20260709 13:02:50.282128  8848 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17.
I20260709 13:02:50.342970  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:50.343194  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:50.241374  8855 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.346102  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:50.338608  1555 rpcz_store.cc:276] 0709 13:02:48.713197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.713335 (+   138us) service_pool.cc:224] Handling call
0709 13:02:48.771033 (+ 57698us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:48.771436 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.771479 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:48.788915 (+ 17436us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.047583 (+258668us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:49.047622 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.047690 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.327110 (+1279420us) raft_consensus.cc:1685] finished
0709 13:02:50.327139 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.327293 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2731,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":138,"thread_start_us":185218,"threads_started":1,"wal-append.queue_time_us":402794}
W20260709 13:02:50.336884  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 176316us
I20260709 13:02:50.340950  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:50.342676  2347 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:50.344880  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:50.357762  2353 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:50.345790  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:50.358711  2348 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
W20260709 13:02:50.361976  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 45595.2us
I20260709 13:02:50.251674  8808 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.300158  8149 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.239414  8764 consensus_queue.cc:260] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.340520  8653 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.326033  8785 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.330133  8807 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.333458  8804 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.359434  8835 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.354096  8781 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.360323  8783 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:50.357180  8910 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:50.380177  8387 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:50.385012  8910 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.385342  2352 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:50.385460  8910 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:50.366380  8871 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:50.386909  8871 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.376679  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1632) took 1421 ms. Trace:
I20260709 13:02:50.387794  2351 rpcz_store.cc:276] 0709 13:02:48.815395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.882897 (+ 67502us) service_pool.cc:224] Handling call
0709 13:02:48.902026 (+ 19129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.902246 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:49.240040 (+337794us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.236148 (+996108us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:50.236205 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:50.236249 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.236316 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.236420 (+   104us) inbound_call.cc:177] Queueing success response
0709 13:02:50.308596 (+ 72176us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
Metrics: {"mutex_wait_us":836944,"prepare.queue_time_us":106267,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":860937,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":17131,"spinlock_wait_cycles":41212800,"thread_start_us":472594,"threads_started":2,"wal-append.queue_time_us":965494}
I20260709 13:02:50.368784  8698 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:50.383649  8785 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.389082  8871 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.366365  8909 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:50.391187  8878 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.391670  8878 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.388998  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3476) took 1854 ms. Trace:
I20260709 13:02:50.389626  8698 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.394342  1630 rpcz_store.cc:276] 0709 13:02:48.534328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.534571 (+   243us) service_pool.cc:224] Handling call
0709 13:02:48.535070 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.191862 (+656792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.388753 (+1196891us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b540017c854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:50.388941 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1494295936}
I20260709 13:02:50.401911  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3334) took 1542 ms. Trace:
I20260709 13:02:50.402410  1560 rpcz_store.cc:276] 0709 13:02:48.860006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.380810 (+520804us) service_pool.cc:224] Handling call
0709 13:02:49.381456 (+   646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.382062 (+   606us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:49.485394 (+103332us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.020764 (+535370us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:50.032949 (+ 12185us) log.cc:844] Serialized 26 byte log entry
0709 13:02:50.033090 (+   141us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:50.033286 (+   196us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.140315 (+107029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.140410 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.384248 (+243838us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b18003dd320. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:50.401696 (+ 17448us) raft_consensus.cc:1685] finished
0709 13:02:50.401723 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.401858 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18279,"prepare.queue_time_us":125897,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":1562,"raft.queue_time_us":307636,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":39484,"spinlock_wait_cycles":27339392,"thread_start_us":331638,"threads_started":2,"wal-append.queue_time_us":546130}
W20260709 13:02:50.407066  1561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3314) took 1825 ms (client timeout 1897 ms). Trace:
W20260709 13:02:50.407497  1561 rpcz_store.cc:269] 0709 13:02:48.581274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.581452 (+   178us) service_pool.cc:224] Handling call
0709 13:02:50.406852 (+1825400us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54003c0fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:50.407007 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1208858368}
I20260709 13:02:50.408782  1553 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:02:50.409165  1553 pending_rounds.cc:85] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 9
I20260709 13:02:50.409391  1553 pending_rounds.cc:107] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:02:50.409677  1553 raft_consensus.cc:2889] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.374112  8906 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.410807  1552 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
W20260709 13:02:50.415632  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.398517  8698 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:50.423260  1708 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 24.
I20260709 13:02:50.373281  8653 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:50.417819  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 59092.8us
W20260709 13:02:50.428581  1676 leader_election.cc:336] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.527s (SENT)
I20260709 13:02:50.385033  8783 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.429462  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:50.433518  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3331) took 1654 ms. Trace:
I20260709 13:02:50.433982  1554 rpcz_store.cc:276] 0709 13:02:48.778852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.785589 (+  6737us) service_pool.cc:224] Handling call
0709 13:02:48.785952 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.089879 (+1303927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.433220 (+343341us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001fd454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:50.433434 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2895797632}
I20260709 13:02:50.436324  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.438938  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:50.441291  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.443635  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:50.494812  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:50.503472  1561 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:50.423811  8785 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:50.504012  1561 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:50.502760  2353 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:50.505275  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3255) took 1018 ms. Trace:
W20260709 13:02:50.506047  1701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2553) took 1657 ms (client timeout 1586 ms). Trace:
W20260709 13:02:50.510172  1701 rpcz_store.cc:269] 0709 13:02:48.848204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.862267 (+ 14063us) service_pool.cc:224] Handling call
0709 13:02:50.505825 (+1643558us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b5400222c54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:50.505962 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498236288}
I20260709 13:02:50.509526  1706 rpcz_store.cc:276] 0709 13:02:49.487051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.487235 (+   184us) service_pool.cc:224] Handling call
0709 13:02:49.487628 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.535301 (+ 47673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.504950 (+969649us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b5400223b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:50.505184 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108239232}
I20260709 13:02:50.512969  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.513891  1554 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:50.400813  8835 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.494091  8800 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.521742  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 78152.5us
I20260709 13:02:50.528015  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:50.528797  8849 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:50.529662  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:50.530761  8919 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:50.531347  8919 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:50.531816  8919 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:50.542496  8855 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.543507  8855 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:50.504487  8860 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:50.528294  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.529803  8849 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:50.509377  1561 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.562107  1561 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:50.511991  8871 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:50.575817  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.425515  8906 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.516297  8387 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.502348  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 346439us
I20260709 13:02:50.523715  8381 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.584167  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:50.585703  1632 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:50.584564  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.586712  1554 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:50.545439  8855 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.590507  1636 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:02:50.590173  8387 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.545751  8792 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.547348  8783 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:50.572590  8898 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 13:02:50.574301  8635 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:50.572098  8860 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:50.515669  8785 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.607443  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:50.570883  8800 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.610745  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2559) took 1065 ms. Trace:
I20260709 13:02:50.591271  8474 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:50.596030  8784 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:50.623636  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:50.594097  8381 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.627331  1702 rpcz_store.cc:276] 0709 13:02:49.545026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.545177 (+   151us) service_pool.cc:224] Handling call
0709 13:02:50.018554 (+473377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:50.040850 (+ 22296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.040886 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:50.046423 (+  5537us) log.cc:844] Serialized 26 byte log entry
0709 13:02:50.344271 (+297848us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:50.572449 (+228178us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.610331 (+ 37882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.610431 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.610486 (+    55us) raft_consensus.cc:1685] finished
0709 13:02:50.610505 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.610674 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59732,"prepare.queue_time_us":20751,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":61310,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":18053,"thread_start_us":418272,"threads_started":2,"wal-append.queue_time_us":313154}
I20260709 13:02:50.630825  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.634563  8704 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.646243  8800 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:50.646461  8381 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:50.646732  8704 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.608350  8561 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.608754  8860 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.647585  8906 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.649147  8860 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.638509  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:50.650271  1627 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:50.639956  1632 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:02:50.643733  1559 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:50.637107  8785 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.652529  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:50.656934  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.658912  8792 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:50.662791  1636 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:50.596521  8922 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:02:50.624763  8149 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.682523  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:50.684351  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:50.666430  1561 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:02:50.686278  1634 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:02:50.686682  1561 pending_rounds.cc:85] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 8
I20260709 13:02:50.687201  1561 pending_rounds.cc:107] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:02:50.687575  1561 raft_consensus.cc:2889] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.687726  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:50.688601  1628 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:50.680125  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.693886  8792 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:50.694347  8792 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:50.695209  8792 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.696388  8792 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:50.698724  1559 raft_consensus.cc:2410] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 20 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:02:50.686158  1636 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:50.707161  1634 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:50.714970  1627 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:50.597512  8855 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.718459  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:50.719237  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 130905us
I20260709 13:02:50.720064  8381 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:50.720613  8381 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.664875  8804 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.697126  8919 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:50.722644  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 130232us
I20260709 13:02:50.729080  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:50.730825  1552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 1118) took 1889 ms (client timeout 1872 ms). Trace:
W20260709 13:02:50.731220  1552 rpcz_store.cc:269] 0709 13:02:48.841605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.090662 (+249057us) service_pool.cc:224] Handling call
0709 13:02:50.730656 (+1639994us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400313154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:50.730759 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2755300096}
I20260709 13:02:50.732268  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.732859  1552 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:50.650899  8474 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.729773  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.735383  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:50.698263  8808 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:50.737350  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:50.665863  8800 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:02:50.740443  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:50.740602  1636 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
W20260709 13:02:50.724155  8765 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.002s
I20260709 13:02:50.741880  1707 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:50.735538  1561 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:50.746631  1628 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 23.
I20260709 13:02:50.744457  1706 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:50.654505  8784 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:50.736692  8880 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.001s
I20260709 13:02:50.749778  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.751473  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.754189  1706 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:02:50.759339  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:50.759656  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.760588  1706 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:50.761250  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.760212  1636 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:50.736891  8843 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:50.770022  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.771790  1602 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:50.738812  8922 consensus_queue.cc:1048] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:02:50.782089  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.782544  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.773603  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:50.775118  1602 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:50.751449  8765 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.794052  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:50.794867  1704 raft_consensus.cc:2368] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18.
I20260709 13:02:50.795535  8765 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.796213  8765 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:50.761011  8925 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:50.776258  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 154105us
I20260709 13:02:50.803351  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.803679  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.804032  1703 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:50.804448  1705 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:50.804950  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.767590  8653 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.806903  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:50.796700  8765 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.808265  8387 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.809818  8765 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.812054  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.813165  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.813858  1704 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
W20260709 13:02:50.814625  8930 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:02:50.818596  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.819381  1709 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:50.819109  8698 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:50.820389  8817 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.656s	user 0.004s	sys 0.004s
I20260709 13:02:50.821643  8765 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:50.822490  8765 consensus_queue.cc:1243] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:02:50.822777  8474 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.823195  8822 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.823690  8822 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.823685  8149 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:50.825238  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:50.826400  8919 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.827044  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:50.827867  8784 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.828426  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.829620  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.829737  8855 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.830355  1552 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 14: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:50.781525  8884 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:02:50.831136  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.820858  8817 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.831811  1552 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:50.831287  8884 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:50.833130  1552 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:50.832827  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:50.837206  1601 leader_election.cc:400] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18.
I20260709 13:02:50.837574  1601 leader_election.cc:403] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:02:50.761651  8880 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.782406  8055 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:50.806342  8925 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.770726  8843 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:50.868162  1552 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:50.843532  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.849776  8910 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.770087  8704 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:02:50.874589  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.880245  1634 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:50.877513  1561 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
W20260709 13:02:50.878296  8924 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:02:50.881413  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.843078  8784 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.835340  8884 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:50.846700  8149 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:50.846517  8653 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:50.845644  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:50.851233  8783 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:50.883785  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:50.875559  8773 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.711s	user 0.011s	sys 0.000s
W20260709 13:02:50.899420  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 53817.1us
I20260709 13:02:50.903595  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:50.905242  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:50.905891  1561 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:50.880674  8822 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:50.879366  8925 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:50.907838  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:50.902384  8783 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.892681  8653 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:50.913655  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2554) took 1512 ms. Trace:
I20260709 13:02:50.931762  1708 rpcz_store.cc:276] 0709 13:02:49.401310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.402159 (+   849us) service_pool.cc:224] Handling call
0709 13:02:50.913431 (+1511272us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b54002e7cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:50.913581 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175558528}
I20260709 13:02:50.907130  8474 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:50.937033  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:50.893399  8784 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:50.908824  8808 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:02:50.904314  8785 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:50.946655  8922 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:50.954136  8922 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 24 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:50.955925  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.955420  8922 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.956550  1560 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:50.957433  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:50.914673  8822 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:50.915566  1559 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:50.907258  8773 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:50.966190  8822 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.966706  8822 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:50.966814  1635 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 13:02:50.967720  8773 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:50.972071  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3328) took 2194 ms. Trace:
I20260709 13:02:50.972451  1553 rpcz_store.cc:276] 0709 13:02:48.777870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.785626 (+  7756us) service_pool.cc:224] Handling call
0709 13:02:50.408437 (+1622811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.418188 (+  9751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.971883 (+553695us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54003c0fd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:50.972016 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3728905344}
W20260709 13:02:50.975412  8892 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.428s	user 0.004s	sys 0.011s
I20260709 13:02:50.976472  8817 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.976627  1557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3333) took 2117 ms. Trace:
I20260709 13:02:50.938620  8783 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:50.976974  1557 rpcz_store.cc:276] 0709 13:02:48.859684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.279719 (+420035us) service_pool.cc:224] Handling call
0709 13:02:49.280016 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.280232 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.280257 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:49.280713 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.413272 (+132559us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:49.877811 (+464539us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.143560 (+265749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.143639 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.974857 (+831218us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:50.976325 (+  1468us) raft_consensus.cc:1685] finished
0709 13:02:50.976426 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.976571 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93430,"prepare.queue_time_us":127479,"prepare.run_cpu_time_us":12079,"prepare.run_wall_time_us":392102,"raft.queue_time_us":31189,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":15009,"spinlock_wait_cycles":8486144,"thread_start_us":606544,"threads_started":3,"wal-append.queue_time_us":239929}
W20260709 13:02:50.982124  8899 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.777s	user 0.003s	sys 0.000s
I20260709 13:02:50.982558  8880 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.983891  8910 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.985127  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3479) took 2034 ms. Trace:
I20260709 13:02:50.986219  8835 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.932833  8653 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:50.979871  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 204719us
I20260709 13:02:50.992028  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:50.992923  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:50.993775  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:51.004225  1707 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:50.994774  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:51.005182  1554 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:50.912377  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:50.985519  1629 rpcz_store.cc:276] 0709 13:02:48.954749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.967819 (+ 13070us) service_pool.cc:224] Handling call
0709 13:02:49.010682 (+ 42863us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:49.334242 (+323560us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:49.355940 (+ 21698us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.522403 (+166463us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:49.530178 (+  7775us) log.cc:844] Serialized 78 byte log entry
0709 13:02:49.530344 (+   166us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:49.530572 (+   228us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.530747 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.530825 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.981544 (+1450719us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:50.984913 (+  3369us) raft_consensus.cc:1685] finished
0709 13:02:50.984933 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.985073 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27889,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":11838,"raft.queue_time_us":28503,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":15095,"spinlock_wait_cycles":46099968,"thread_start_us":357329,"threads_started":2,"wal-append.queue_time_us":780097}
I20260709 13:02:51.005389  1561 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
W20260709 13:02:51.002943  8862 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.689s	user 0.010s	sys 0.000s
I20260709 13:02:51.004271  1705 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:51.005335  1602 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:51.002136  8843 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:51.010149  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 97776.1us
I20260709 13:02:51.013386  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:51.014537  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:51.015110  1708 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:02:51.015915  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:51.016180  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.016723  2348 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
W20260709 13:02:50.992020  8752 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.920s	user 0.013s	sys 0.000s
I20260709 13:02:51.014048  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:51.024722  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.013883  8387 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:51.020020  1554 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:51.025520  8387 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.024890  2351 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:02:51.018452  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.027915  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.028558  2350 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:51.029765  2352 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:02:51.032464  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.034068  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.036558  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:51.036883  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.040200  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.035051  2348 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:51.040995  2344 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:51.037572  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:51.042058  2349 raft_consensus.cc:2410] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 22 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:51.042868  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.044440  2352 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:51.045835  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.047510  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:51.048087  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:51.049810  2348 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:51.042856  8856 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:50.979470  8884 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.056443  8906 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.056954  8906 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.059063  8906 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:51.012701  8910 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.065532  8910 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.018164  8785 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:51.068727  8785 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.069141  8785 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:51.036268  8808 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:51.073844  8808 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.074218  8808 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.040472  8752 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.077694  8752 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:51.078404  1627 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:51.027583  8387 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:51.017912  8862 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.061633  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.145931  1636 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:51.129133  8884 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:51.146474  1636 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:51.145860  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:51.147303  1636 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.148072  1629 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.148447  1636 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:51.116020  8910 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:51.129829  8784 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:51.131145  8871 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.003s
I20260709 13:02:51.152000  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2578) took 1018 ms. Trace:
I20260709 13:02:51.139909  8698 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:51.153195  1558 rpcz_store.cc:276] 0709 13:02:50.133279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.151578 (+ 18299us) service_pool.cc:224] Handling call
0709 13:02:51.151781 (+1000203us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400311854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:51.151940 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550183168}
I20260709 13:02:51.146217  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.154438  8878 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:51.154657  8871 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:51.099908  8943 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.155457  8943 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.156589  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:51.156893  8862 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:51.138597  8948 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:02:51.131484  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:51.100386  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.157186  1634 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:51.083199  8822 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.140354  1559 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:51.161756  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:51.182099  1554 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:51.080749  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 75461.1us
I20260709 13:02:51.079202  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:51.078574  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:51.141908  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.183401  2348 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:51.183995  1602 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:51.128118  1707 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:51.101210  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.150739  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:51.177013  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:51.175449  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 126538us
I20260709 13:02:51.179800  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.180437  1629 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:51.191905  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.193349  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.194162  1631 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:51.195075  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44403us
I20260709 13:02:51.189293  1630 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:51.187601  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:51.218354  1636 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:51.218540  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.218919  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:51.192797  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.133677  8794 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:51.195374  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:51.219249  2352 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:51.237087  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:51.229818  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.225499  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.160382  8906 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:51.233377  2353 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:51.238137  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.229872  8793 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:51.229970  8784 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:51.255174  2318 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:51.240370  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:51.240592  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:51.261439  1632 raft_consensus.cc:2368] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20.
I20260709 13:02:51.240367  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:51.262959  1558 raft_consensus.cc:2410] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 22 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:02:51.242996  2346 raft_consensus.cc:2410] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
W20260709 13:02:51.243057  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47715.9us
I20260709 13:02:51.243906  2352 raft_consensus.cc:2410] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 22 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:02:51.245039  2349 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:02:51.264318  2349 pending_rounds.cc:85] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 11
I20260709 13:02:51.264622  2349 pending_rounds.cc:107] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:02:51.264902  2349 raft_consensus.cc:2889] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:51.224628  8910 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:51.268024  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.268541  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.246981  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:51.268786  1631 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:51.269222  1635 raft_consensus.cc:2368] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21.
W20260709 13:02:51.236523  8860 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:02:51.272998  1629 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:02:51.195906  8903 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:51.236794  8945 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:51.277606  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:51.251708  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:51.252941  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.257053  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.278795  1627 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:51.279241  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:51.279758  1627 pending_rounds.cc:85] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:02:51.279901  1710 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:02:51.280056  1627 pending_rounds.cc:107] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:02:51.280328  1627 raft_consensus.cc:2889] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:51.280509  8664 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:51.282140  8852 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:51.282940  2347 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:51.283354  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.284070  8852 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:51.284825  8653 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.236304  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:51.219691  8752 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:51.224705  8616 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:51.194532  8878 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:51.217248  8848 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.266608  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:51.358062  1705 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:51.230201  8909 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:51.268772  1707 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:51.363032  1707 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:51.364457  1707 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:02:51.364948  1707 pending_rounds.cc:85] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 13
I20260709 13:02:51.269673  1628 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:51.245517  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:51.269942  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.271811  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.369421  1632 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:51.370486  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:51.371097  1632 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:51.267611  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:51.272058  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:51.372215  1556 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:51.142288  8925 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.245316  8906 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.274489  8809 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:51.276019  8474 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:51.276532  1703 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:51.277017  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:51.396301  1704 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:51.278216  1708 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:51.285148  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:51.237334  8783 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.294703  8822 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:51.295542  8903 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.230324  8914 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:02:51.296645  8910 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:51.298063  8948 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:51.252303  8792 consensus_queue.cc:1048] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:02:51.299594  8807 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:51.251435  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:51.304953  1554 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:51.249415  8843 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:51.305817  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.422900  1635 raft_consensus.cc:2368] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17.
W20260709 13:02:51.305742  1454 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.757s	user 0.007s	sys 0.025s
I20260709 13:02:51.312644  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:51.423736  1629 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:51.313740  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.425091  1525 leader_election.cc:400] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20.
I20260709 13:02:51.384819  8906 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:51.272789  8149 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18.
I20260709 13:02:51.272764  8943 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:51.271441  8793 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:51.320075  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.441105  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.374063  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:51.309460  8784 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:02:51.374651  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.361451  8752 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.446820  1710 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:51.395675  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 119028us
I20260709 13:02:51.386504  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.447975  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:51.398123  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 112970us
I20260709 13:02:51.448515  1628 raft_consensus.cc:2376] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17. Re-sending same reply.
I20260709 13:02:51.407649  8862 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:51.448915  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.365202  1707 pending_rounds.cc:107] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 18.14
I20260709 13:02:51.449869  1707 raft_consensus.cc:2917] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Aborting config change with OpId 18.14: Aborted: Op aborted by new leader
I20260709 13:02:51.449805  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:51.450584  1707 raft_consensus.cc:2889] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:51.450781  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:51.400106  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 120872us
I20260709 13:02:51.452958  2317 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:02:51.400239  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 132613us
W20260709 13:02:51.404845  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 159312us
I20260709 13:02:51.421154  8850 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:02:51.361915  8909 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.412386  1632 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:51.470570  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.314530  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:51.373539  8878 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:02:51.432152  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:51.475574  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54785us
I20260709 13:02:51.426427  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:51.427439  1454 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.879s	user 0.008s	sys 0.025s
I20260709 13:02:51.481230  8793 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.482658  8381 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:51.355697  8616 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.400173  8875 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:02:51.459090  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:51.492636  1630 raft_consensus.cc:2376] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21. Re-sending same reply.
I20260709 13:02:51.492899  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.493985  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.422735  8871 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:51.449312  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.467772  1629 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:51.422083  8919 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:51.472486  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.455248  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:51.499030  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43862.9us
I20260709 13:02:51.460600  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.398692  8817 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.047s
W20260709 13:02:51.476524  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50155.1us
I20260709 13:02:51.427356  8843 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.427763  8474 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:51.478794  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.513708  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.458410  8862 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.468618  8882 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:51.477998  8909 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.373672  8848 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.537532  8875 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.538015  8875 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.538360  8752 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:51.474460  8855 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:02:51.538904  8848 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:51.422516  8904 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:02:51.538937  8752 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:51.541153  8904 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:51.541640  8904 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:51.541767  8910 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.542565  8904 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.540000  8953 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:51.543681  8953 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:51.543802  8904 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.544107  8953 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:51.481539  8561 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:51.439404  8822 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:51.476493  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:51.550100  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:51.550740  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 74333.8us
I20260709 13:02:51.406641  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:51.457265  8943 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:51.457821  8698 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.496870  1630 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:51.565938  8698 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.566964  1630 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:51.504281  1635 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:51.505884  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:51.504452  8792 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:51.569787  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 63970.9us
I20260709 13:02:51.571655  1634 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:51.540093  8875 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:51.561723  8848 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:51.524204  8862 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:51.549899  8856 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:51.573849  8856 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.575939  8856 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:51.577958  8878 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.475646  8880 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:51.488426  8616 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.533895  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:51.591852  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:51.524303  8882 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.535027  8387 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:51.536213  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.536764  8916 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:51.540268  8799 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:51.548005  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:51.600785  1628 raft_consensus.cc:2368] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21.
I20260709 13:02:51.546554  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:51.502650  8871 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.501796  8919 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.562435  1525 leader_election.cc:400] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21.
I20260709 13:02:51.604974  1525 leader_election.cc:403] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:02:51.473275  8942 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.054s
W20260709 13:02:51.565084  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.497046  8784 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:51.568574  1630 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:02:51.525465  8664 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:51.521654  8843 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:51.582666  8792 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:51.598340  8947 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:02:51.590692  1634 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:02:51.585172  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:51.541658  8903 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:51.603645  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 67450us
I20260709 13:02:51.643015  1703 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 24: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:51.605193  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:51.643389  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.608916  1628 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:51.609341  1635 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:51.611924  1630 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:51.569928  8561 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:51.584335  1631 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:51.650264  1628 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:51.653612  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.597605  8882 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:51.612438  8785 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.000s
I20260709 13:02:51.601589  8855 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:51.662549  1630 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:51.662837  1631 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:51.610562  8808 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.002s
W20260709 13:02:51.635076  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 182087us
W20260709 13:02:51.636603  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:51.638232  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.568570  8698 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:51.659917  1629 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:51.664556  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.604601  8635 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:51.610787  8871 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:51.634323  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 185956us
I20260709 13:02:51.668650  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:51.610816  8943 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:51.610481  8919 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.611225  8664 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:51.647661  8792 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.685966  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:51.616997  8784 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:51.634438  8856 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:02:51.663221  8903 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21.
I20260709 13:02:51.666800  8764 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:51.674185  8635 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.702399  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:51.718343  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.718791  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.702555  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:51.719367  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 258705us
I20260709 13:02:51.697369  1705 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:51.719835  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.708325  1707 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:51.711750  1557 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:51.720840  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:51.719849  2345 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:51.666754  8653 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:51.714426  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:51.667695  8855 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.666728  8704 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:51.707271  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:51.720054  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:51.659734  8909 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.734045  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:51.692641  8752 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:51.699424  8783 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:51.799821  2345 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:51.704278  8925 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:51.802469  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.729283  1701 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:51.821913  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 250855us
I20260709 13:02:51.724612  8878 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:51.673995  8808 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.673398  8785 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.825874  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:51.827047  1704 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:02:51.830358  1701 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:02:51.747640  8855 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:51.837468  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 117615us
I20260709 13:02:51.728629  8807 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:51.729812  8635 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:51.845188  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 126378us
I20260709 13:02:51.845531  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:51.850683  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 298797us
I20260709 13:02:51.848414  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.815001  8880 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:51.815578  8910 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:51.819331  8822 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:51.862538  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:51.843935  8944 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:02:51.867689  2350 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:51.868108  2351 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:51.699777  8474 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.870959  1556 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:51.729872  8809 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.118s
I20260709 13:02:51.837298  8653 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:51.728813  8960 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.141s
I20260709 13:02:51.844826  8387 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:51.880755  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:51.882019  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:51.891582  1525 leader_election.cc:400] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17.
I20260709 13:02:51.848012  8878 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:51.859015  8880 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.880659  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:51.844882  8804 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:51.854874  8884 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.002s
W20260709 13:02:51.900305  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.859174  8910 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.864866  8783 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:51.861923  8807 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:51.904328  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:51.902617  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:51.905047  1553 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:51.906762  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 58272.4us
W20260709 13:02:51.910359  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.818116  8561 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.917407  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:51.918116  1556 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.919814  1525 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:51.897552  8903 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:51.906128  8952 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 13:02:51.897794  8958 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:51.901089  8880 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:51.904152  8910 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:51.941949  8817 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:51.919903  8884 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:51.946069  1525 leader_election.cc:400] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21.
I20260709 13:02:51.890056  8972 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 13:02:51.917210  8653 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:51.921610  8856 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:51.905118  8804 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:51.947852  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.885896  8784 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:51.955204  1556 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:02:51.930426  8903 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:51.872599  8664 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:51.960049  8950 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.317s	user 0.000s	sys 0.001s
I20260709 13:02:51.968297  8922 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:51.963474  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.966989  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:51.973073  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.975855  2318 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:51.981923  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:51.920084  8925 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
W20260709 13:02:51.984702  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 161360us
W20260709 13:02:51.980615  8956 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:02:52.004819  2353 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:52.008560  2318 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:52.019989  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 53067.2us
I20260709 13:02:52.004200  2345 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:52.007788  1675 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:52.022608  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:52.023396  2348 raft_consensus.cc:2368] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16.
I20260709 13:02:51.983662  8903 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:51.971856  8884 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:52.006438  8977 raft_consensus.cc:3055] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:52.018955  8592 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:02:51.970021  8809 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:02:52.049607  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:52.013248  8817 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:51.960853  8981 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 18.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.077s
W20260709 13:02:52.038146  8955 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
W20260709 13:02:52.039968  8962 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:02:52.015834  8698 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:02:51.959473  8387 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:52.078951  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 127018us
I20260709 13:02:52.069547  8785 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:51.960745  8878 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.028635  8984 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:52.036504  8922 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:52.156096  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 352469us
I20260709 13:02:52.047968  8977 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:52.117012  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1278) took 1050 ms. Trace:
W20260709 13:02:52.154026  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 146241us
I20260709 13:02:52.166440  2351 rpcz_store.cc:276] 0709 13:02:51.035795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.035902 (+   107us) service_pool.cc:224] Handling call
0709 13:02:52.086217 (+1050315us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b54004b7154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:52.086427 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1612045568}
I20260709 13:02:52.133239  1557 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:02:52.028242  8664 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:52.077359  8961 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
W20260709 13:02:52.081171  8848 consensus_meta.cc:287] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.508s	user 0.006s	sys 0.006s
I20260709 13:02:52.038511  8909 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:52.155344  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 78639.7us
I20260709 13:02:52.077250  8808 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:52.148113  8964 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
W20260709 13:02:52.170523  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 161933us
I20260709 13:02:52.024149  8807 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.187775  1634 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:52.176863  1701 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:52.146603  8852 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:52.185964  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:52.195719  8926 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:02:52.196240  8979 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:02:52.196588  8986 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:52.197190  8969 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:02:52.197592  8988 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
W20260709 13:02:52.198365  8967 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:02:52.199007  8988 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.199479  8988 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.184065  8848 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:52.183034  8972 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.174280  8793 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:02:52.143517  8653 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:52.240056  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.241026  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.173655  8922 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.243014  1701 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:52.190439  8984 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:52.247709  1704 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:52.249905  1704 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:02:52.255533  8635 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.259296  8635 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:52.266141  8988 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:52.266518  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 109435us
W20260709 13:02:52.267362  8794 consensus_meta.cc:287] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.020s	user 0.006s	sys 0.006s
I20260709 13:02:52.199492  8943 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:52.273293  1555 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:02:52.206902  8909 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:52.275799  1555 pending_rounds.cc:85] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 13
I20260709 13:02:52.279685  1555 pending_rounds.cc:107] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:02:52.204665  8926 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:52.280319  1555 raft_consensus.cc:2889] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:52.246431  8964 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:52.287371  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2576) took 1294 ms. Trace:
I20260709 13:02:52.289764  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:52.296897  1557 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 20: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:52.290874  1556 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:52.298705  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 441125us
I20260709 13:02:52.202366  8852 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.294675  1709 rpcz_store.cc:276] 0709 13:02:50.993085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.993501 (+   416us) service_pool.cc:224] Handling call
0709 13:02:51.016883 (+ 23382us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:51.038751 (+ 21868us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:51.104459 (+ 65708us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.322820 (+218361us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:51.323507 (+   687us) log.cc:844] Serialized 52 byte log entry
0709 13:02:51.323696 (+   189us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:51.324241 (+   545us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.385466 (+ 61225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.385572 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:52.287088 (+901516us) raft_consensus.cc:1685] finished
0709 13:02:52.287118 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.287299 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46680,"prepare.queue_time_us":58300,"prepare.run_cpu_time_us":947,"prepare.run_wall_time_us":31494,"raft.queue_time_us":83829,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":80008,"thread_start_us":204883,"threads_started":1,"wal-append.queue_time_us":436437}
W20260709 13:02:52.304335  1525 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.504s (SENT) [suppressed 14 similar messages]
I20260709 13:02:52.281176  8875 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.194032  8884 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.279853  8958 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:52.277199  8848 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:52.337790  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:52.283201  8804 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:52.299388  8793 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:52.239204  8972 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:52.343719  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 50916.7us
I20260709 13:02:52.273756  8808 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:02:52.203572  8817 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.353840  8991 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:02:52.354821  8989 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.172s
W20260709 13:02:52.360659  8919 consensus_meta.cc:287] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.662s	user 0.008s	sys 0.000s
W20260709 13:02:52.360999  8792 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.646s	user 0.007s	sys 0.000s
I20260709 13:02:52.395993  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:52.397351  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:52.398118  1630 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
W20260709 13:02:52.363652  8882 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.685s	user 0.007s	sys 0.004s
I20260709 13:02:52.300585  8988 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:52.365352  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:52.300019  8909 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:52.366298  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:52.322659  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:52.284049  8943 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.372730  8804 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:02:52.390431  8959 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
W20260709 13:02:52.392340  8970 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.739s	user 0.002s	sys 0.002s
I20260709 13:02:52.392789  8852 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:52.361272  8984 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.337083  8998 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:52.404260  2349 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:02:52.407617  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:52.461630  1554 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:02:52.352314  8848 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.411990  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:52.466507  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:52.467072  1629 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:52.351699  8793 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:02:52.429205  1676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.018s	sys 0.026s
W20260709 13:02:52.469307  1676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.019s	sys 0.026s
W20260709 13:02:52.469713  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 592276us
I20260709 13:02:52.431216  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:52.471321  1627 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
W20260709 13:02:52.431782  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 135606us
I20260709 13:02:52.473189  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:52.435638  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:52.440907  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 257805us
I20260709 13:02:52.443398  1556 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:52.372722  8958 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:52.442763  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:52.348297  8875 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.398386  8635 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:52.399717  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:52.403246  8773 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:52.468003  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:52.500500  1629 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:52.443066  8983 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:02:52.456900  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:52.502796  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:52.399080  8919 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.394335  8387 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:02:52.491828  2345 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEARNER]: Advancing to term 19
I20260709 13:02:52.432314  9001 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 13:02:52.468637  8998 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.518714  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:52.492686  8992 consensus_queue.cc:1059] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.123s
I20260709 13:02:52.493121  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:52.493347  8987 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.498929  2352 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:52.505175  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:52.510111  2346 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:52.521190  2349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:38616
I20260709 13:02:52.293269  8977 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.543133  1705 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:52.534129  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:52.430171  8988 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.538996  2349 raft_consensus.cc:606] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:52.412323  8882 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.549687  1674 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:52.410466  8792 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.551177  9015 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:52.552295  8792 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:52.552568  9015 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:52.494159  8976 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:02:52.552973  9015 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:52.553401  9015 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:52.553619  1704 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:52.553572  1674 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:52.562603  1629 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:52.534855  8387 consensus_queue.cc:1243] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:02:52.584448  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 113528us
I20260709 13:02:52.584762  1633 raft_consensus.cc:2368] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17.
I20260709 13:02:52.587118  2345 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:52.504815  8635 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.430634  8926 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.541949  9003 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:52.589608  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 285123us
I20260709 13:02:52.517827  8773 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.586284  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:52.590437  8387 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:52.590224  8922 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:52.543969  8987 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:52.608129  8862 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.014s	user 0.012s	sys 0.000s
W20260709 13:02:52.620625  8843 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.961s	user 0.011s	sys 0.002s
I20260709 13:02:52.682600  8843 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.683113  8919 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:52.683673  8919 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:52.684167  8919 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.684784  8843 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:52.620261  8995 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:52.590260  8835 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:02:52.685768  8995 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.546490  8909 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.687675  8882 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:52.688071  8995 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:52.688868  1702 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:52.607172  8852 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:52.689709  8843 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:52.611074  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:52.689746  8852 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:52.691329  8852 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:52.690912  8919 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:52.691381  8843 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.694554  8843 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:52.681915  8972 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:52.699998  1627 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:52.701165  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:52.714491  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 114078us
W20260709 13:02:52.681260  8616 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.076s	user 0.009s	sys 0.004s
I20260709 13:02:52.698848  8862 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:52.717698  9018 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:02:52.718400  9011 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:02:52.719203  8664 consensus_meta.cc:287] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.002s
I20260709 13:02:52.719378  8987 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:52.723727  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:52.736543  1554 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:52.725299  9020 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:52.725829  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 189188us
I20260709 13:02:52.727187  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:52.593146  8958 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.679371  8986 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:52.740000  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:52.744396  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:52.745203  1552 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:52.710508  8843 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.744346  1554 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
W20260709 13:02:52.735108  8953 consensus_meta.cc:287] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.190s	user 0.013s	sys 0.000s
I20260709 13:02:52.631246  9024 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.130s
I20260709 13:02:52.726570  8793 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:52.745137  8986 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:52.785087  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3209) took 1428 ms. Trace:
I20260709 13:02:52.755292  8943 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:52.785559  1634 rpcz_store.cc:276] 0709 13:02:51.356430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.642830 (+286400us) service_pool.cc:224] Handling call
0709 13:02:52.187420 (+544590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.188350 (+   930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.784906 (+596556us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b5400349ed0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:52.785032 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238528384}
I20260709 13:02:52.735852  8616 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.788529  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:52.742230  8922 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:02:52.788764  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:52.792744  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.771502  8848 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:52.780149  8878 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.794081  1676 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:52.778955  8909 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.802486  1552 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
W20260709 13:02:52.788349  8904 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.242s	user 0.014s	sys 0.000s
I20260709 13:02:52.866333  9003 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:52.809556  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:52.863173  1634 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:02:52.794044  8773 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:52.923408  8764 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.192s	user 0.011s	sys 0.002s
W20260709 13:02:52.915156  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 121092us
W20260709 13:02:52.938225  8910 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.981s	user 0.009s	sys 0.002s
W20260709 13:02:52.942271  8906 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.496s	user 0.010s	sys 0.003s
I20260709 13:02:52.773157  8882 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 13:02:52.768744  8953 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.921607  1559 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 20: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:52.924840  9006 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:52.926227  9025 log.cc:927] Time spent T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 13:02:52.926438  9023 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:02:52.926915  8616 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:52.928498  8855 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.071s	user 0.011s	sys 0.000s
W20260709 13:02:52.929524  9028 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:52.985342  2344 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 23. Current term is 24. Ops: []
W20260709 13:02:52.913381  8783 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.040s	sys 0.038s
I20260709 13:02:52.933812  1552 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
I20260709 13:02:52.795328  8995 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 13:02:52.948935  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:52.952082  1632 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 14.
I20260709 13:02:52.923000  8998 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:52.866657  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:52.766460  8843 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:52.984433  8871 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.286s	user 0.006s	sys 0.007s
I20260709 13:02:52.944204  9003 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:52.961587  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:52.970948  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:52.980355  9009 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.333s	user 0.000s	sys 0.001s
I20260709 13:02:52.950405  8635 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:52.923704  8878 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:53.009224  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47745.4us
W20260709 13:02:52.996616  9033 log.cc:927] Time spent T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:02:53.010267  1525 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:52.987972  9006 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.011032  8998 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:53.011814  8998 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:53.012212  8998 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:52.997215  9032 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.945515  8793 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:53.000766  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3262) took 1730 ms. Trace:
W20260709 13:02:53.001197  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 52348.5us
I20260709 13:02:52.956730  8910 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:52.976570  8804 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:53.008926  1704 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:52.942417  8909 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:52.990391  8783 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.047s	user 0.041s	sys 0.039s
I20260709 13:02:52.987846  8855 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.012612  8998 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:52.999826  8916 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.960548  8906 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.021083  1702 rpcz_store.cc:276] 0709 13:02:51.270255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.270383 (+   128us) service_pool.cc:224] Handling call
0709 13:02:51.298583 (+ 28200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.689307 (+1390724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.000484 (+311177us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54002a67d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:53.000698 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3194957056}
I20260709 13:02:53.021672  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:53.004328  8387 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:53.026031  9036 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.028012  8843 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:53.029712  8804 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:53.030830  8880 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.078s	user 0.009s	sys 0.003s
I20260709 13:02:53.037840  8910 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.038439  1707 raft_consensus.cc:2368] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18.
W20260709 13:02:53.039149  9016 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.384s	user 0.000s	sys 0.002s
I20260709 13:02:53.038854  9032 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.018430  9003 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:53.099498  8903 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.044s	user 0.005s	sys 0.007s
W20260709 13:02:53.041216  2350 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1276) took 2009 ms (client timeout 2375 ms). Trace:
I20260709 13:02:53.004163  8919 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:53.134127  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1480) took 1079 ms. Trace:
I20260709 13:02:53.134404  9014 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:53.134637  2353 rpcz_store.cc:276] 0709 13:02:52.054463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.080430 (+ 25967us) service_pool.cc:224] Handling call
0709 13:02:52.080717 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.080918 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:52.142945 (+ 62027us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.130735 (+987790us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b18004284a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287
0709 13:02:53.133624 (+  2889us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:53.133691 (+    67us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:53.133753 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.133868 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.134050 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24300,"prepare.queue_time_us":62098,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":97333,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":36284,"spinlock_wait_cycles":2820992,"thread_start_us":91621,"threads_started":1}
W20260709 13:02:53.135823  2350 rpcz_store.cc:269] 0709 13:02:51.032119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.032221 (+   102us) service_pool.cc:224] Handling call
0709 13:02:53.040982 (+2008761us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b5400762d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:53.041149 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1909136640}
I20260709 13:02:53.136740  8785 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:53.071626  8953 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:53.137295  8878 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.130127  8903 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.139494  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:53.140270  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 118629us
I20260709 13:02:53.141129  1702 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:53.143214  1636 raft_consensus.cc:2368] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18.
I20260709 13:02:53.046897  1704 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:02:53.018659  8871 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.147915  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:53.150074  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:53.102988  9041 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:53.107060  9006 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.165390  1708 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:53.101629  8783 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:53.165804  1676 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:53.143411  8878 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:02:53.149559  9035 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:02:53.178247  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:53.179114  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:53.180188  8966 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.256s	user 0.000s	sys 0.015s
I20260709 13:02:53.180289  1702 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:53.180367  9014 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:53.152516  8852 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.164812  1704 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 16: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:53.111027  8880 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:53.169399  8822 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.301s	user 0.015s	sys 0.006s
I20260709 13:02:53.194202  1706 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:02:53.193117  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:53.198148  1676 leader_election.cc:400] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16.
I20260709 13:02:53.198618  1676 leader_election.cc:403] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:53.201788  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:53.181104  8966 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.257s	user 0.000s	sys 0.016s
I20260709 13:02:53.263477  1631 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:53.187139  8903 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.186610  8852 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.137310  8387 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.187332  8910 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:53.184623  8909 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.206991  8822 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:53.208762  8919 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:53.263623  9015 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.710s	user 0.012s	sys 0.000s
I20260709 13:02:53.214496  8799 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:53.181320  8878 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.210754  9014 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 15. Current term is 16.
I20260709 13:02:53.139808  9036 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:53.277966  9034 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.223s	user 0.000s	sys 0.002s
I20260709 13:02:53.174001  9041 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.139582  8916 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.182899  8783 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.276684  8910 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:53.281644  8799 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:53.299165  9032 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.292910  8855 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.284092  9015 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.314049  8906 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.304992  8880 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.322517  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2588) took 1604 ms. Trace:
I20260709 13:02:53.322983  1707 rpcz_store.cc:276] 0709 13:02:51.717733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.733714 (+ 15981us) service_pool.cc:224] Handling call
0709 13:02:53.313733 (+1580019us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54000cbcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:53.322445 (+  8712us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2827133824}
I20260709 13:02:53.320945  8561 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:53.292783  8799 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:53.334044  8561 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.291642  8910 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:53.331952  8871 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:53.324481  8980 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:53.340740  9036 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:53.342283  9015 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:53.355923  8963 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.702s	user 0.002s	sys 0.002s
I20260709 13:02:53.354029  8980 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:53.363631  8985 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:53.364976  8878 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.366329  9041 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.365134  8985 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:53.369176  8980 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.369694  8985 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.370368  8980 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:53.370882  8985 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:53.373353  8793 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.366405  8875 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.374217  8793 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.376472  1705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2586) took 1671 ms. Trace:
I20260709 13:02:53.376852  1705 rpcz_store.cc:276] 0709 13:02:51.705122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.721675 (+ 16553us) service_pool.cc:224] Handling call
0709 13:02:51.721922 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.543664 (+821742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.376184 (+832520us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b540016ba54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:53.376397 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1886843264}
I20260709 13:02:53.379063  8561 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.382316  1675 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:53.383508  8990 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:53.384057  8990 raft_consensus.cc:697] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:53.384728  8998 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.385667  1557 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:02:53.386101  1557 pending_rounds.cc:85] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 10
I20260709 13:02:53.386305  1557 pending_rounds.cc:107] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:02:53.386664  1557 raft_consensus.cc:2889] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:53.385033  8990 consensus_queue.cc:237] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.385686  8973 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:02:53.391055  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.394614  1557 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:53.395678  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:53.397034  1560 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:53.376767  8793 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.397940  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1642) took 1046 ms. Trace:
I20260709 13:02:53.398739  2344 rpcz_store.cc:276] 0709 13:02:52.351330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.351671 (+   341us) service_pool.cc:224] Handling call
0709 13:02:52.399293 (+ 47622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.991148 (+591855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.397678 (+406530us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b540002b2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:53.397865 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1346438400}
I20260709 13:02:53.401036  8988 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.401404  9003 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:53.418898  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.403570  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3218) took 1006 ms. Trace:
I20260709 13:02:53.420030  1627 rpcz_store.cc:276] 0709 13:02:52.399309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.472104 (+ 72795us) service_pool.cc:224] Handling call
0709 13:02:53.403379 (+931275us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54002db9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:53.403506 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414790528}
I20260709 13:02:53.403685  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:53.406649  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.416033  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:53.418218  9015 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:53.427851  9015 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.429204  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.429756  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.425457  1557 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:53.429945  1554 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 24.
I20260709 13:02:53.418771  8988 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.436159  1560 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:53.439981  1553 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:53.447934  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:53.435834  8880 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.442270  8999 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.459646  8972 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:02:53.466625  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:53.467018  8977 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:53.467226  8852 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.468807  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:53.494305  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:53.445535  8998 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.467532  8906 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:53.445075  8875 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:02:53.501411  1634 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 22: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:53.506739  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:53.471020  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:53.467408  8878 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:53.467770  8903 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:53.510517  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:53.517431  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:53.521021  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:53.439734  8988 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.493026  8784 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:53.493299  9003 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:53.535899  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:53.537453  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:53.501466  8664 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:53.544129  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.544936  1557 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:53.499101  8767 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.545974  1634 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:53.503492  9041 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:53.515615  8592 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:53.566751  8592 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.547161  8871 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:53.549896  8903 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.569922  8903 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.572371  8903 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.503280  8852 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.569046  8799 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.573583  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.576550  8799 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.546653  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:53.577126  8664 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:53.577831  1629 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:53.578459  8664 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.578934  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:53.580283  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:53.582448  9054 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:02:53.586050  8875 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.587217  8875 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.576241  1553 raft_consensus.cc:1240] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 13:02:53.591945  8988 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.522413  8878 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.596135  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:53.597616  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:53.601408  1553 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:53.569195  8592 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.584165  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:53.604425  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:53.604969  8996 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:53.605562  8996 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:53.606690  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:53.607817  8799 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:53.606722  8996 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.610607  8996 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:53.555855  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2577) took 3423 ms. Trace:
I20260709 13:02:53.493571  8880 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.500777  8999 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.629609  8875 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.558719  8916 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.597025  8878 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.639073  8999 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:53.518929  8971 consensus_queue.cc:1059] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:02:53.643353  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:53.518430  9015 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.523231  8906 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:53.530445  8984 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.572662  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:53.612460  1555 rpcz_store.cc:276] 0709 13:02:50.132867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.357906 (+225039us) service_pool.cc:224] Handling call
0709 13:02:50.370539 (+ 12633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.281016 (+1910477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.555465 (+1274449us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b54002530d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:53.555768 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4372510976}
I20260709 13:02:53.612985  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:53.554540  1636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3199) took 2238 ms (client timeout 1504 ms). Trace:
I20260709 13:02:53.624416  8880 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.645061  1675 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:53.662848  1675 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:53.657214  1636 rpcz_store.cc:269] 0709 13:02:51.316648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.318489 (+  1841us) service_pool.cc:224] Handling call
0709 13:02:53.554144 (+2235655us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b540017b454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:53.554420 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3910242176}
I20260709 13:02:53.575687  8852 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.592959  8767 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.679111  1629 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:53.670408  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:53.654568  8878 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:53.654989  8916 consensus_queue.cc:1059] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:02:53.695758  1675 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:53.669509  8843 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:53.696805  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.704792  1560 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:53.704855  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:53.671839  9015 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:02:53.706422  1636 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:53.653107  8999 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:02:53.707461  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.672032  8561 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:53.716259  8906 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.658921  8882 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:02:53.731496  1560 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:53.732949  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:53.692130  8878 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:53.754827  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:53.755477  1629 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:53.708045  9015 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.668314  8984 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.759394  1636 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:53.759172  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.761294  1560 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:02:53.762040  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:53.730018  8880 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.766112  1636 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 13:02:53.756016  8843 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:53.753109  8878 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:53.761448  9015 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:53.776093  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:53.777784  1636 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
I20260709 13:02:53.727887  8910 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.764014  8916 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:53.750247  8561 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.789414  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.796281  1636 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:02:53.797240  1554 raft_consensus.cc:2410] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 18 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:02:53.797111  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:53.780223  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.798385  8882 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:53.798501  8996 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:53.799340  8800 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:53.799672  1561 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:02:53.799979  8800 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:53.801074  8800 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:53.782265  9063 log.cc:927] Time spent T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:02:53.784149  9055 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 13:02:53.786301  9059 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:02:53.779508  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:53.758250  8906 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:02:53.772751  8880 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.793154  8984 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.794878  8871 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:53.787577  8843 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:53.845467  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 403627us
I20260709 13:02:53.827808  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.830777  1558 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:53.844280  8843 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.782287  8767 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.780628  8916 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:53.836656  8999 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:53.834769  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 55423.1us
W20260709 13:02:53.856841  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 80781.6us
I20260709 13:02:53.858721  1634 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:53.860343  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:53.862478  8698 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:53.864261  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.862507  9041 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:53.852649  8916 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:53.874567  8804 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.860936  8880 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.888998  1708 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 13:02:53.882843  9006 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.883145  8804 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.894249  1634 raft_consensus.cc:1240] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:02:53.897832  1561 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:53.899044  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:53.899621  1561 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:53.900368  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:53.907805  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44036us
I20260709 13:02:53.921451  8664 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:02:53.922230  8968 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 2.267s	user 0.002s	sys 0.003s
I20260709 13:02:53.903784  9006 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:53.925299  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3272) took 2435 ms. Trace:
I20260709 13:02:53.925779  1710 rpcz_store.cc:276] 0709 13:02:51.489693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.491335 (+  1642us) service_pool.cc:224] Handling call
0709 13:02:51.504823 (+ 13488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:51.505157 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.505183 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:51.549091 (+ 43908us) log.cc:844] Serialized 564 byte log entry
0709 13:02:51.655893 (+106802us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:51.655932 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.655997 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.920361 (+2264364us) spinlock_profiling.cc:243] Waited 8.77 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:53.924808 (+  4447us) raft_consensus.cc:1685] finished
0709 13:02:53.924846 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.925181 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20168064,"thread_start_us":106451,"threads_started":1,"wal-append.queue_time_us":105515}
I20260709 13:02:53.937983  8910 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.901914  8804 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:53.922259  8664 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:53.947728  1633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3200) took 2628 ms (client timeout 2105 ms). Trace:
W20260709 13:02:53.948253  9047 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.689s	user 0.005s	sys 0.000s
W20260709 13:02:53.948319  1633 rpcz_store.cc:269] 0709 13:02:51.319786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.375789 (+ 56003us) service_pool.cc:224] Handling call
0709 13:02:53.947460 (+2571671us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400343d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:53.947628 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2345551232}
I20260709 13:02:53.945626  8804 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.947908  1628 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:53.905987  8919 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:02:53.925450  9006 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.955019  8664 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:53.965344  8971 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:53.977150  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:53.975533  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:53.980053  1707 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:53.982563  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:53.984210  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:53.984923  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:53.985651  1710 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:53.985857  8989 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:53.986758  8804 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:53.987784  8804 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:53.988160  8804 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:53.989332  1702 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:53.992938  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:53.995110  1556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3366) took 1513 ms. Trace:
I20260709 13:02:53.995616  1556 rpcz_store.cc:276] 0709 13:02:52.481997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.530149 (+ 48152us) service_pool.cc:224] Handling call
0709 13:02:52.530443 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.559292 (+ 28849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.559341 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:52.559401 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:52.559447 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.994869 (+1435422us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b5400208354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:53.994922 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.995049 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65781504}
I20260709 13:02:53.996416  1702 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:02:53.996888  1702 pending_rounds.cc:85] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 14
I20260709 13:02:53.997112  1702 pending_rounds.cc:107] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:02:53.997357  1702 raft_consensus.cc:2889] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:53.999136  1707 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:53.957417  8916 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:54.004555  8474 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:54.005075  8474 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:54.006176  8474 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.007498  8474 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:53.989125  8804 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:54.012198  8916 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:53.980727  8387 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:02:54.014612  2317 connection.cc:256] RPC call timeout handler was delayed by -1.3337506070000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21610000000000001s for a response
I20260709 13:02:54.022984  8817 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:54.023844  9037 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:54.024434  9037 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.025442  8976 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.026011  8976 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:54.029224  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.979620  8971 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:54.014926  8960 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:54.050928  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:54.051916  2352 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:54.051919  2349 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:54.053687  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.054610  2344 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:54.063530  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:54.028591  8976 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:54.065294  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:54.043568  9048 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.012s	sys 0.023s
I20260709 13:02:54.068769  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.028841  8910 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:54.066977  1705 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:02:54.069134  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.040913  8959 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.069844  2353 raft_consensus.cc:2410] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:02:54.077903  9072 log.cc:927] Time spent T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:02:54.028565  9037 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:54.079190  9075 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:02:54.040998  8954 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.081935  8954 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.071777  8910 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.084848  8954 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:54.063416  8992 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.085651  8910 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:54.085889  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1214) took 1491 ms. Trace:
I20260709 13:02:54.077605  8959 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.086136  8992 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.086385  1708 rpcz_store.cc:276] 0709 13:02:52.594748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.176487 (+581739us) service_pool.cc:224] Handling call
0709 13:02:53.176996 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:53.889761 (+712765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.085579 (+195818us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b540036e2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:54.085819 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1635517696}
I20260709 13:02:54.088616  8926 raft_consensus.cc:3055] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:54.089217  8926 raft_consensus.cc:740] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:54.052526  8804 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:54.063856  8960 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.090313  8926 consensus_queue.cc:260] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:54.091750  8926 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:54.092131  8960 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:54.093551  2350 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:02:54.072359  9048 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.012s	sys 0.023s
I20260709 13:02:54.082674  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:54.095702  8971 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:54.042729  8878 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:54.097177  8971 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:54.036084  8387 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.100160  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 1134) took 2199 ms. Trace:
I20260709 13:02:54.100603  1559 rpcz_store.cc:276] 0709 13:02:51.901118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.901257 (+   139us) service_pool.cc:224] Handling call
0709 13:02:54.099913 (+2198656us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b5400143054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:54.100097 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2213755136}
I20260709 13:02:54.034904  9006 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 13:02:54.102382  9006 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:02:54.102730  9006 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:54.103682  9006 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.105105  9006 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:02:54.106765  9076 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:02:54.106613  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:02:54.095171  9071 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:02:54.095038  8959 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:54.116169  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:54.117184  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.088768  8992 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:54.122678  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:54.123720  1708 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:54.123608  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.125448  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.126907  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.127696  1705 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:54.127731  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.128782  2350 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:54.137442  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.135748  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.140468  2350 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:54.162827  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.160585  1527 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:54.172245  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.184540  1527 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:54.215389  2347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "946eea23a2334d678895281941dd9926"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:38616
I20260709 13:02:54.216499  2347 raft_consensus.cc:606] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:54.216981  2347 raft_consensus.cc:614] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:54.219274  8862 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:54.181416  9040 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:54.237969  1557 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:54.224767  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:54.239583  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:54.240991  1556 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:54.203148  8387 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:54.161612  8817 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:02:54.161823  8875 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:02:54.243525  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.243695  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.257438  2350 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:54.248593  2347 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:54.259871  8954 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.260439  8954 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.261734  8848 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:54.264632  8561 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:54.265231  8878 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:54.239511  9040 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.266263  9040 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:54.267210  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:54.268008  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3523) took 1654 ms. Trace:
I20260709 13:02:54.268860  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:54.269812  1558 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:54.275731  8799 raft_consensus.cc:3055] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:54.276211  8799 raft_consensus.cc:740] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:54.273144  9085 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:54.215335  9001 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:54.256260  1560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198"
dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:41400
I20260709 13:02:54.280447  1560 raft_consensus.cc:606] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:54.280819  1560 raft_consensus.cc:614] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:54.253027  1554 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: [14.11-15.12]
I20260709 13:02:54.282873  1553 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:54.279284  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:54.285909  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:54.285166  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.239629  8982 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:54.268375  1630 rpcz_store.cc:276] 0709 13:02:52.613430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.298264 (+684834us) service_pool.cc:224] Handling call
0709 13:02:53.409568 (+111304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:53.410156 (+   588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.410205 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:53.410935 (+   730us) log.cc:844] Serialized 26 byte log entry
0709 13:02:53.518727 (+107792us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:53.518794 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.518901 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.839037 (+320136us) raft_consensus.cc:1685] finished
0709 13:02:53.839064 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.267721 (+428657us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54002db9d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:54.267945 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13237,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":4724,"spinlock_wait_cycles":254650752,"thread_start_us":107355,"threads_started":1,"wal-append.queue_time_us":257966}
W20260709 13:02:54.283013  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.289750  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:54.291040  2349 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:02:54.291430  2349 pending_rounds.cc:85] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 11
I20260709 13:02:54.291661  2349 pending_rounds.cc:107] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:02:54.292068  2349 raft_consensus.cc:2889] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:54.292197  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:54.253021  9002 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:54.253588  8959 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:54.294539  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:54.296430  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:54.289940  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.298470  2344 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
W20260709 13:02:54.302637  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.288967  8976 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 18.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.016s
I20260709 13:02:54.277189  8799 consensus_queue.cc:260] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:54.304708  1710 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:02:54.304823  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.305590  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.305727  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.339661  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.339879  1702 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:54.339995  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:54.310037  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.287771  9037 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:02:54.311034  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:54.341462  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:54.297561  8981 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:54.284080  8997 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:02:54.279580  8960 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.346539  8809 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 18.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:02:54.346479  8960 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.283937  9001 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:54.349593  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.293286  8664 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.335999  8958 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.355737  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.293254  8977 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:54.342018  1701 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:54.264662  9015 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:54.341311  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:54.367525  2344 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:54.343950  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.344386  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.344866  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.349715  9001 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:54.342873  8784 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:02:54.371575  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.372429  1704 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:02:54.373821  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.374193  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.374569  1701 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:54.374902  1705 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 14.
I20260709 13:02:54.375566  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.376061  1705 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:54.378715  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1335) took 1977 ms. Trace:
I20260709 13:02:54.379330  1631 rpcz_store.cc:276] 0709 13:02:52.401363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.536590 (+135227us) service_pool.cc:224] Handling call
0709 13:02:52.562165 (+ 25575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.264587 (+702422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.378460 (+1113873us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b5400197b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:54.378647 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1611362176}
I20260709 13:02:54.382170  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3270) took 3007 ms. Trace:
I20260709 13:02:54.381785  8958 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.382576  1706 rpcz_store.cc:276] 0709 13:02:51.374818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.375109 (+   291us) service_pool.cc:224] Handling call
0709 13:02:51.405813 (+ 30704us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:53.196362 (+1790549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.381843 (+1185481us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54002d87d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:54.382103 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4109502208}
I20260709 13:02:54.336309  8953 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:54.289196  8980 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:54.351934  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.357915  2346 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:54.358167  8664 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.332767  8783 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:54.329766  9002 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:54.287621  9085 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.391144  2352 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:02:54.369802  1702 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:54.349280  8960 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:54.398218  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.350646  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:54.409509  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.318015  8959 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.298710  8982 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:54.432446  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 203521us
W20260709 13:02:54.425336  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.424752  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.430688  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:54.425107  1707 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:02:54.435599  2349 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:54.436869  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.438511  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:54.438871  2347 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 14.
I20260709 13:02:54.439868  2346 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:54.439934  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.441323  2347 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:54.441751  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.442453  1709 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:54.446141  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:54.447467  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:54.448436  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.422250  8992 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:54.453222  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.454752  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:54.455472  2346 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:54.450313  8875 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:54.467713  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.421553  8953 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:54.450917  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 201336us
I20260709 13:02:54.469803  2344 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:02:54.469264  2349 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:54.473954  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:54.477567  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.478528  2344 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:54.480034  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.480826  2349 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:54.482187  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:54.486487  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.487473  2344 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 14.
I20260709 13:02:54.472152  8958 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:54.489889  2349 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:02:54.471575  8875 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:54.493662  2344 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:54.482015  9014 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.500521  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:54.503384  9083 log.cc:927] Time spent T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:02:54.508782  9087 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:02:54.493760  8875 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:54.512799  9090 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:02:54.526057  9089 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:02:54.529743  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1478) took 2484 ms. Trace:
I20260709 13:02:54.531057  8664 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:54.532580  8953 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:54.530308  2348 rpcz_store.cc:276] 0709 13:02:52.043233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.043419 (+   186us) service_pool.cc:224] Handling call
0709 13:02:52.078319 (+ 34900us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:52.137536 (+ 59217us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:52.273192 (+135656us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.530089 (+256897us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.998084 (+1467995us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b18004c1ec0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:02:53.998250 (+   166us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:53.998760 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:02:54.168335 (+169575us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:54.168402 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.168517 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.174646 (+  6129us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b18004c1ec0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:54.527344 (+352698us) raft_consensus.cc:1685] finished
0709 13:02:54.527371 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.527534 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":130757,"prepare.queue_time_us":33653,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":1619835,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":11195,"spinlock_wait_cycles":18460288,"thread_start_us":506450,"threads_started":3,"wal-append.queue_time_us":680164,"wal-append.run_cpu_time_us":2969,"wal-append.run_wall_time_us":1202217}
I20260709 13:02:54.537735  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:54.539223  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:54.540436  1676 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:54.543859  1561 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:54.543994  2346 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:54.546159  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:54.548435  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:54.551503  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:54.552528  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:54.555846  2348 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:54.521200  9040 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.549409  8914 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:54.578464  8914 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:54.579043  8914 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.579427  8914 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:54.490229  9002 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.549835  8387 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:54.544502  8381 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:54.507501  9014 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:54.585919  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.554109  9015 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:54.594574  1525 leader_election.cc:336] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.504s (SENT)
I20260709 13:02:54.600383  1559 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 24.
I20260709 13:02:54.549606  8981 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 13:02:54.602784  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 54390.5us
I20260709 13:02:54.603840  1676 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:54.605146  1552 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 17. Ops: []
W20260709 13:02:54.606695  9097 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:54.611024  9040 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:54.614436  9015 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:54.618597  8875 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:54.627285  9014 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:54.605645  8959 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:54.636238  9049 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.376s	user 0.002s	sys 0.003s
I20260709 13:02:54.638856  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3521) took 2026 ms. Trace:
I20260709 13:02:54.639310  1635 rpcz_store.cc:276] 0709 13:02:52.612854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.144464 (+531610us) service_pool.cc:224] Handling call
0709 13:02:53.148813 (+  4349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:53.149428 (+   615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.149504 (+    76us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:53.150372 (+   868us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.260112 (+109740us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:53.260161 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.260251 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.998329 (+738078us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b48000318d8. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2c603aa1 00007fda2c5fab37 00007fda2c5fa9ca 00007fda2c6f226f 00007fda2c6f1f37 00007fda2c70655c 00007fda2c706508 00007fda2c706488 00007fda2c706443 00007fda2c7056ec 00007fda2b62fa96
0709 13:02:54.635506 (+637177us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:02:54.638603 (+  3097us) raft_consensus.cc:1685] finished
0709 13:02:54.638629 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.638790 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1923,"raft.run_cpu_time_us":1880,"raft.run_wall_time_us":847170,"spinlock_wait_cycles":22996992,"thread_start_us":109179,"threads_started":1,"wal-append.queue_time_us":109648}
I20260709 13:02:54.646610  1676 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:54.643828  9001 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.652546  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:54.658802  8993 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:54.669819  8986 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.672115  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:54.672884  8997 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:54.676658  9004 raft_consensus.cc:3055] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:02:54.680769  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.676096  1555 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:02:54.682297  8997 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:54.634020  8875 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:54.658357  1676 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:54.654677  8910 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:54.684407  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.692744  1558 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:54.689253  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:54.690693  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:54.687844  9001 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:54.686343  9050 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.422s	user 0.011s	sys 0.004s
I20260709 13:02:54.687248  8986 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:54.686009  8997 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.702478  8997 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:54.693145  1558 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:54.707891  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.711556  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.714224  1709 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:54.716933  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.718528  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.719417  1709 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:54.681823  8381 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:54.718302  1558 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:54.730330  1558 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:54.730738  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 72845.1us
I20260709 13:02:54.731631  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:54.732551  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.734992  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:54.683944  9004 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:54.739177  1710 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:54.737757  1702 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:54.740800  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:54.751961  1676 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:54.728330  8982 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:54.735795  9015 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:54.774611  8884 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:54.775355  8884 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
W20260709 13:02:54.769246  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 859529us
I20260709 13:02:54.778774  8987 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:54.779301  8987 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.780391  8862 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:02:54.781617  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.784595  9085 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:54.790165  9100 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:02:54.791965  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3214) took 2395 ms. Trace:
I20260709 13:02:54.792527  1628 rpcz_store.cc:276] 0709 13:02:52.396318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.398882 (+  2564us) service_pool.cc:224] Handling call
0709 13:02:53.947658 (+1548776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.950150 (+  2492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.791684 (+841534us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400343d50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:54.791836 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3558879616}
I20260709 13:02:54.800468  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:54.804069  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:54.807376  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:54.808357  2346 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:54.810182  1552 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:02:54.753192  8387 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:54.749891  9099 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:54.825718  1560 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:54.814503  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:54.831437  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:54.833359  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:54.835572  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:54.836956  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:54.838371  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:54.839673  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:54.840528  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.842064  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:54.767565  9014 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:54.851907  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:54.863035  1552 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:54.866367  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3548) took 1065 ms. Trace:
I20260709 13:02:54.867126  1710 raft_consensus.cc:2368] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18.
I20260709 13:02:54.866794  1636 rpcz_store.cc:276] 0709 13:02:53.801296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.886301 (+ 85005us) service_pool.cc:224] Handling call
0709 13:02:53.886913 (+   612us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:53.887393 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:53.898427 (+ 11034us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.671464 (+773037us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b1800386500. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:02:54.812766 (+141302us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:54.813335 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:02:54.813686 (+   351us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:54.813726 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.813800 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.840127 (+ 26327us) raft_consensus.cc:1685] finished
0709 13:02:54.866132 (+ 26005us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.866301 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":514975,"prepare.queue_time_us":10663,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":40,"raft.queue_time_us":516736,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":19473,"spinlock_wait_cycles":17136000,"thread_start_us":356754,"threads_started":1,"wal-append.queue_time_us":506214}
W20260709 13:02:54.871821  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1324) took 3494 ms (client timeout 2161 ms). Trace:
W20260709 13:02:54.872334  1632 rpcz_store.cc:269] 0709 13:02:51.376936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.470267 (+ 93331us) service_pool.cc:224] Handling call
0709 13:02:54.871618 (+3401351us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400196754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:54.871745 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3403420544}
I20260709 13:02:54.814710  8987 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:54.879614  2346 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
W20260709 13:02:54.876392  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.881525  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:54.883801  1628 raft_consensus.cc:1275] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:02:54.884235  1628 pending_rounds.cc:85] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 10
I20260709 13:02:54.884784  1628 pending_rounds.cc:107] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:02:54.884876  1560 raft_consensus.cc:1240] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:02:54.885033  1628 raft_consensus.cc:2889] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:54.815325  8882 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:54.881719  1527 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:54.888418  1628 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 13:02:54.884131  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 59019us
I20260709 13:02:54.898361  8884 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:54.900341  8794 raft_consensus.cc:1064] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: attempting to promote NON_VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb to VOTER
I20260709 13:02:54.874994  8381 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:02:54.832118  9099 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:54.927733  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:54.911394  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.927150  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:54.931710  1631 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:02:54.936044  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:54.930068  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 179431us
I20260709 13:02:54.865643  9014 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.890230  8882 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:54.867554  8989 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:02:54.960253  1635 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:02:54.934681  9099 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:54.990800  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2603) took 1003 ms. Trace:
I20260709 13:02:54.943308  9014 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:54.994735  1703 rpcz_store.cc:276] 0709 13:02:53.987538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.987667 (+   129us) service_pool.cc:224] Handling call
0709 13:02:53.988000 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.098563 (+110563us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:02:54.098870 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.351691 (+252821us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:54.351746 (+    55us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:54.351803 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.990563 (+638760us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b540014d2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:54.990614 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.990744 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":605,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":103,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":4363,"spinlock_wait_cycles":253485824,"thread_start_us":252433,"threads_started":1,"wal-append.queue_time_us":348743}
I20260709 13:02:54.957537  8704 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.996999  8704 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:54.988445  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.999416  8704 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.001696  2352 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac931d9355284a168be64d7debd009ae"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:38616
I20260709 13:02:55.002499  2352 raft_consensus.cc:606] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:55.003057  2352 raft_consensus.cc:614] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:54.970472  9101 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:02:54.888764  8784 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 13:02:54.959270  8441 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.975030  8989 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:54.975816  9093 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:54.942804  8880 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:54.980499  8914 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:54.949400  8862 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.984674  9069 log.cc:927] Time spent T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.158s	user 0.003s	sys 0.003s
I20260709 13:02:54.904870  8809 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 13:02:54.965232  8561 raft_consensus.cc:2955] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Committing config change with OpId 18.11: config changed from index 3 to 11, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:02:55.042936  9098 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.297s	user 0.002s	sys 0.000s
I20260709 13:02:54.906471  9095 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:54.991484  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:54.768273  9004 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.012341  1558 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 13:02:55.068107  1558 pending_rounds.cc:85] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 8
I20260709 13:02:55.068430  1558 pending_rounds.cc:107] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 13:02:55.068722  1558 raft_consensus.cc:2889] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:55.015036  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:55.009248  2350 raft_consensus.cc:2955] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index 3 to 11, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:02:55.026024  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 265545us
I20260709 13:02:55.001572  9070 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.047900  1553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2620) took 1450 ms. Trace:
I20260709 13:02:54.990758  1710 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:55.074275  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.019250  8989 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:55.073333  1553 rpcz_store.cc:276] 0709 13:02:53.597147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.266029 (+668882us) service_pool.cc:224] Handling call
0709 13:02:54.282425 (+ 16396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.283405 (+   980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.047594 (+764189us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b54002ffb50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:55.047790 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36710656}
I20260709 13:02:55.044700  9106 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:02:55.076508  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61520.8us
I20260709 13:02:55.080305  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:55.073438  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:55.085289  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:55.022254  9006 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:02:55.077610  9106 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:55.055126  8862 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.021898  8441 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.090611  8664 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:55.111944  8664 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:55.112594  8664 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:55.115047  9106 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.116676  9106 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:55.116894  8441 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.078562  9070 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.118957  8954 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:55.119750  9070 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:55.094655  8848 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:55.120527  8848 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.120949  8848 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:55.122697  8914 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.123458  8914 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:55.044179  9093 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.125679  8855 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:55.126200  8855 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:55.125330  8664 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.126729  8855 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:55.127352  8862 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:55.127758  9093 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.048636  8880 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.128626  8664 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.130192  8880 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:55.075500  9085 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:02:55.139343  9015 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:55.083142  9001 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:55.083726  9095 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:55.141719  8954 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.141973  9095 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.142500  8954 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.147156  8954 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:55.153142  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:55.154126  1553 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:02:55.155516  8954 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.158982  2352 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:55.160604  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:55.156617  8954 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.163483  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:55.166321  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:55.170236  8809 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:55.179738  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.213361  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.231700  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.222029  8882 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.177732  8809 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.285817  8848 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.289239  8848 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:55.258980  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 319701us
I20260709 13:02:55.314029  1602 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:55.314898  8862 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:55.325650  1558 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:55.325923  8862 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.327771  9015 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:55.329020  8862 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.370414  8992 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.371100  8992 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.378311  8884 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:55.379398  8817 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:55.379848  8817 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:55.380774  8817 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.383226  8817 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:55.180963  8664 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.139s
I20260709 13:02:55.329417  9015 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:55.387005  8989 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.388974  8989 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:55.389580  8987 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:02:55.320606  9087 log.cc:927] Time spent T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:02:55.333869  8954 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:55.259318  8809 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:55.350193  9111 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
W20260709 13:02:55.356336  9107 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.274s	user 0.003s	sys 0.000s
I20260709 13:02:55.341238  9004 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:55.393976  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:55.394665  1704 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:55.395113  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:55.395226  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 81254.3us
I20260709 13:02:55.395340  1707 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:55.396063  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:55.393648  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:55.406389  1710 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:55.409494  1703 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:55.414803  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.407821  1602 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:55.398025  8878 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:55.420766  1602 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:55.373338  8992 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:55.428850  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 340084us
I20260709 13:02:55.408641  8664 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:55.428843  8878 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.430387  8987 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:55.344873  8794 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.431057  8441 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:02:55.455144  8878 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:55.439522  8871 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
W20260709 13:02:55.523149  9099 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.000s
I20260709 13:02:55.506875  9014 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.538332  9014 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.527818  9099 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.548805  9099 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.600730  1558 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:55.606098  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.005s	sys 0.016s
W20260709 13:02:55.606823  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.005s	sys 0.017s
W20260709 13:02:55.607448  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 521866us
I20260709 13:02:55.608875  1526 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:55.609968  1601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.009s	sys 0.013s
W20260709 13:02:55.611413  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 190520us
W20260709 13:02:55.611660  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 174504us
W20260709 13:02:55.614262  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.450394  8664 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:55.638123  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.636443  1601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.009s	sys 0.013s
W20260709 13:02:55.641359  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.012s	sys 0.024s
I20260709 13:02:55.647727  9113 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:55.649024  9113 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.652451  9113 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:55.656471  9114 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:55.671720  1602 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:55.674809  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.678622  9014 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.679579  9099 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.679958  9014 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.680744  9099 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.686571  8704 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:55.687485  8704 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.687956  9099 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.668601  9114 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.688887  8871 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:55.689644  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.689989  8871 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.691166  9114 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:55.694120  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.013s	sys 0.024s
W20260709 13:02:55.646371  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 680993us
I20260709 13:02:55.702981  9119 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.695390  8871 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.724040  9014 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:55.686643  1702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97"
dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:43592
I20260709 13:02:55.727128  1702 raft_consensus.cc:606] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:55.727715  1702 raft_consensus.cc:614] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:55.728636  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.729801  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.732477  8871 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:55.734093  9099 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:02:55.740695  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 132113us
I20260709 13:02:55.704702  9119 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:55.752751  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 48066.1us
W20260709 13:02:55.753991  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.687981  8704 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:55.742334  9099 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:55.759963  9099 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.761951  9121 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:55.762851  9121 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.763613  9121 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:55.764019  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:55.766081  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:55.767753  2351 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc"
dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
mode: GRACEFUL
new_leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
 from {username='slave'} at 127.0.0.1:38616
I20260709 13:02:55.768556  2351 raft_consensus.cc:606] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Received request to transfer leadership to TS ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:55.768754  1559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42562 (request call id 1143) took 1532 ms (client timeout 1779 ms). Trace:
W20260709 13:02:55.761947  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.762763  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:55.774672   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:02:55.775094  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:55.746417  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 801039us
W20260709 13:02:55.781139  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.782145  1557 raft_consensus.cc:2410] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:02:55.785923  2351 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 15. (term mismatch)
I20260709 13:02:55.786535  2351 pending_rounds.cc:85] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 14
I20260709 13:02:55.786864  2351 pending_rounds.cc:107] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 16.15
I20260709 13:02:55.785825  8977 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:55.788443  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:55.788132  8977 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.790052  9001 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.787206  2351 raft_consensus.cc:2889] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:55.793360  2350 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:55.796520  8855 consensus_meta.cc:287] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.669s	user 0.014s	sys 0.000s
I20260709 13:02:55.798276  8664 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:55.791036  9001 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.796986  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:55.829319  8977 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:55.812199  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:55.832887  2349 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:55.833244  1635 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:55.835376  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:55.838828  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:55.840299  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:55.840498  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 169013us
I20260709 13:02:55.848675  1557 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:55.851102  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:55.812354  9122 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.799070  8799 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:55.816635  9103 log.cc:927] Time spent T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.984s	user 0.000s	sys 0.007s
W20260709 13:02:55.853394  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.797350  8855 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:55.873873  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 77057.8us
I20260709 13:02:55.881680  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:55.874693  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:55.864221  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:55.899652  1702 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:55.865322  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:55.886511  1602 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:55.872906  1560 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:55.821947  8882 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:55.875878  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1665) took 1393 ms. Trace:
I20260709 13:02:55.883999  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:55.877230  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:55.870102  8848 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:55.777307  1559 rpcz_store.cc:269] 0709 13:02:54.236179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.237262 (+  1083us) service_pool.cc:224] Handling call
0709 13:02:55.768068 (+1530806us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b5400312254. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:55.768334 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":745321344}
I20260709 13:02:55.925959  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:55.855871  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:55.928447  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 72797.1us
I20260709 13:02:55.930052  9015 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:55.930990  9108 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:55.931690  9106 consensus_queue.cc:1048] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:02:55.932363  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 78826.1us
I20260709 13:02:55.856442  9104 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:55.874246  9127 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:55.933784  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:55.933569  9104 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.926172  8954 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:02:55.935643  8664 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.936378  9104 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.936286  8664 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.937367  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:55.939137  8882 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.932088  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:55.939812  8882 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.930110  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:55.941592  1561 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:55.943379  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:55.943671  9004 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:55.865033  9122 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:02:55.868180  9001 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.946020  9004 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:55.946427  9122 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:55.946678  9001 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:55.947319  9001 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:55.947486  2352 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:55.948539  9001 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:55.884346  9119 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:55.956682  2348 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:02:55.931161  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:55.979871  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:55.964771  8848 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:55.982234  8848 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:55.982000  9116 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:55.989004  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:55.996070   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:55.997653  9127 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:55.998265  9127 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:55.998955  9127 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:56.002893  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 116485us
I20260709 13:02:56.003010  8882 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:56.003989  8882 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:56.004510  8882 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.935655  8954 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:56.005374  8954 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:56.005831  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.006533  1636 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:56.006700  8954 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.918445  2347 rpcz_store.cc:276] 0709 13:02:54.482511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.486718 (+  4207us) service_pool.cc:224] Handling call
0709 13:02:54.992108 (+505390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:54.992595 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.992627 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:54.993162 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:02:55.438466 (+445304us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:55.438520 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.438624 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:55.614815 (+176191us) raft_consensus.cc:1685] finished
0709 13:02:55.614857 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.875355 (+260498us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b54007701d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:55.875659 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1799,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":10022,"spinlock_wait_cycles":1160350080,"thread_start_us":444838,"threads_started":1,"wal-append.queue_time_us":533507}
I20260709 13:02:56.011139  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:56.012228  1702 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:56.013872  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:56.017544  8916 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.017952  8664 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:56.019191  8989 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.012s
W20260709 13:02:56.022454  9137 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.097s	user 0.002s	sys 0.000s
I20260709 13:02:56.023435  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.024905  1636 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:02:55.954030  9006 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:02:55.977110  8441 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:56.004987  8882 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:56.016695  8704 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:55.953451  9099 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:55.984762  9116 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:56.048420  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:56.051920  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:56.052284  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.058465  2350 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:56.059244  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:56.012255  8809 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.611s	user 0.014s	sys 0.003s
I20260709 13:02:56.036769  8441 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:56.060632  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:56.073148  1627 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:56.074052  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:56.076912  1628 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:56.077502  1632 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:56.082410  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.084328  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.086190  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.087821  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.088376  1634 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:56.089995  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:56.063369  8855 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.090776  1710 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:56.091426  8704 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.092775  1635 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:56.093138  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.096808  1709 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:56.097012  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:56.098935  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:56.099807  1707 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:56.100646  8989 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:56.101599  1704 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:56.103183  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:56.104020  1705 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
W20260709 13:02:56.106784  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 175805us
I20260709 13:02:56.108485  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.109714  9116 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:56.109112  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 59669.4us
I20260709 13:02:56.111104  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.113214  8149 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:56.113827  8149 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:56.115103  8149 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:56.118222  9084 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.931264  9015 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.118896  9084 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.101150  8989 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:55.931747  8871 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.143s
I20260709 13:02:56.149425  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.061802  9122 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:02:56.156908  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 129176us
I20260709 13:02:56.155064  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:56.150220  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:56.146100  8916 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:56.161921  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 209020us
I20260709 13:02:56.162844  8916 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.071233  9114 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:56.165512  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.167024  8855 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:02:56.167860  1627 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:02:56.168664  9006 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:56.167991  8441 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.169112  9006 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:56.169636  9006 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:56.153657  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:56.166033  9114 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.172987  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:56.179831  1628 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 24.
W20260709 13:02:56.179417  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 57999us
W20260709 13:02:56.187322  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.192531  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:56.193331  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.217515  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:56.223431  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.144317  9119 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:02:56.143752  8809 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.236315  8809 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:56.185830  8664 consensus_queue.cc:1059] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:02:56.239732  8664 raft_consensus.cc:3055] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:56.240149  8664 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:56.240546  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:56.241638  8972 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:56.242101  8972 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:56.242331  8794 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:56.242954  8794 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.243379  8794 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:56.245925  9093 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:56.246263  8817 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.247056  8817 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.242491  8972 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:56.252434  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 64993.9us
I20260709 13:02:56.256194  2349 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
W20260709 13:02:56.256510  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.205703  8855 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:56.268572  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 323237us
W20260709 13:02:56.275439  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.262266  8855 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.278735  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:56.280737  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.283234  2350 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:56.294792  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:56.295859  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.241216  8664 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.300472  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:56.305182  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.287432  8855 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:56.285792  9015 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.282342  9084 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:56.318331  2349 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:56.323086  8809 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:56.328490  1628 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:56.330114  9040 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 13:02:56.331655  8793 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:56.332196  8793 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:56.334137  8793 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.336253  8793 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:56.344754  8954 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:56.346639  8848 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.348948  9040 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:56.349848  9040 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:56.352625  8848 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:56.350459  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.354138  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 73727.9us
I20260709 13:02:56.356809  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:56.359908  1526 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:56.362703  8794 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.369122  8809 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.371091  8794 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.371738  8809 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.289098  8989 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.321522  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.366256  8848 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:56.387516  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.388092  8809 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:56.353397  9040 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:56.398769  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:56.399418  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:56.429124  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 455008us
I20260709 13:02:56.398416  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.397953  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.399094  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:56.437305  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.440062  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.440812  1628 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:56.448359  1710 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
W20260709 13:02:56.450106  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 93436.5us
W20260709 13:02:56.450940  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.437268  1634 raft_consensus.cc:2376] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 24. Re-sending same reply.
W20260709 13:02:56.408914  9121 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.007s
I20260709 13:02:56.441196  9040 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.466368  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:56.468245  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 100050us
W20260709 13:02:56.468748  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 81433.2us
I20260709 13:02:56.455739  9116 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:56.473896  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:56.470192  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.474645  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:56.397383  8855 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.478130  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:56.478160  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:56.471439  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:56.473076  1627 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:56.480646  1630 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:56.528142  1704 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:56.553725  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.473583  8989 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:56.472024  8817 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:56.561960  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.472967  8387 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.557828  1705 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 25: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:56.446266  9127 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.479481  9084 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:02:56.598248  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 123770us
W20260709 13:02:56.600503  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.612388  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.614861  1635 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:56.604099  1525 leader_election.cc:336] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.506s (SENT)
I20260709 13:02:56.465744  9040 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:56.620728  8882 consensus_meta.cc:287] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.003s
I20260709 13:02:56.621229  8882 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.623694  8882 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.625605  8635 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.626395  8635 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.632356  9145 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:56.634279  9140 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.309s	user 0.000s	sys 0.000s
I20260709 13:02:56.635529  9095 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:56.636368  9095 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:56.637645  1628 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:02:56.637728  9095 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:56.638708  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.639519  1627 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:56.640405  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.641587  8855 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.641757  1703 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:56.642561  9115 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:02:56.642970  8878 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.168s	user 0.011s	sys 0.010s
I20260709 13:02:56.643464  8878 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.638129  1633 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:56.644964  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:56.645032  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 184724us
I20260709 13:02:56.595643  8794 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.648488  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:56.654271  1704 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:56.654748  1636 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:56.655493  1632 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:56.610232  9084 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.617074  9106 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:56.666057  1526 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:56.672223  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 60783us
W20260709 13:02:56.675462  9148 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:56.685799  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3418) took 1021 ms. Trace:
I20260709 13:02:56.686384  1554 rpcz_store.cc:276] 0709 13:02:55.597426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.598430 (+  1004us) service_pool.cc:224] Handling call
0709 13:02:55.939075 (+340645us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b5800002798. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5f8c 00007fda245b5ef6 0000000000375939 00007fda2c6291e0 00007fda2c6f20c4 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:02:55.939264 (+   189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.939753 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.939793 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:55.940498 (+   705us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.274470 (+333972us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:56.274537 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.274636 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.598001 (+323365us) raft_consensus.cc:1685] finished
0709 13:02:56.598026 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.618559 (+ 20533us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14488,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":210,"spinlock_wait_cycles":61991424,"thread_start_us":328324,"threads_started":1,"wal-append.queue_time_us":449259}
I20260709 13:02:56.653241  8794 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.689139  8817 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:56.687011  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 103443us
W20260709 13:02:56.649344  8992 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.014s	sys 0.005s
I20260709 13:02:56.675585  9145 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.673460  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:56.690068  8817 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:56.463732  9121 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.637039  9006 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:56.671738  9084 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:56.649153  8387 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.660660  8793 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:56.716385  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:56.733398  1554 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:56.696172  8992 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.015s	sys 0.006s
I20260709 13:02:56.737066  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:56.738591  2350 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:56.661180  9114 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.705511  8871 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.738029  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.700876  8878 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.744140  9015 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:56.744436  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:56.709262  8817 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:56.754652  1702 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:56.758142  8809 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:56.759053  8635 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.759856  1555 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:56.753636  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.762014  8664 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:56.746423  8381 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.764341  8381 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.767088  8381 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.768051  1703 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:56.734431  9108 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.770054  9108 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.770795  9015 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.708174  8954 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.735025  8793 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.780886  1627 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:56.781720  9145 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.794368  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:56.796991  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.794723  1635 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:56.800882  2318 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.801205  2348 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:56.801949  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.802775  1628 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:56.804821  1631 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 13:02:56.806001  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 99555.2us
I20260709 13:02:56.806304  9001 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:56.807786  9001 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:56.808200  9001 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:56.810344  9015 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.811208  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.811822  9006 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:56.812260  1630 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:56.812494  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:56.812712  9085 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.813248  9085 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.813686  8954 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.815512  1559 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 20. Current term is 21. Ops: []
W20260709 13:02:56.801177  1525 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.754s (SENT)
I20260709 13:02:56.809935  8817 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.809058  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.816181  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:56.821192  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.755277  8794 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:56.793378  8799 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.925s	user 0.012s	sys 0.004s
W20260709 13:02:56.793673  8972 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.004s
I20260709 13:02:56.827669  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:56.826592  1702 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:56.832819  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:56.827908  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:02:56.827777  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47242.7us
I20260709 13:02:56.828701  9105 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.829341  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:56.828747  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:56.831342  2350 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:56.810355  9040 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:56.833791  2347 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:56.834213  2352 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:56.798310  8793 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:56.838951  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:56.840346  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:56.840075  8972 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:56.840994  2345 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:56.841190  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:56.842197  1710 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:56.836915  8977 raft_consensus.cc:2764] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:02:56.844230  8972 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:56.834946  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 567147us
I20260709 13:02:56.816144  9004 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:56.849002  2352 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:02:56.849494  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:56.853803  1703 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:02:56.815801  8954 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:56.856985  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:56.857818  1552 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:56.859045  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:56.859350  1708 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:56.859939  1558 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:56.860222  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:56.862109  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:56.862895  1633 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:56.863739  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:56.864511  1710 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:56.869135  9040 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:56.861166  9105 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.869699  9040 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.870448  9040 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:56.870010  8855 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.872028  9105 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:56.872239  9143 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.259s	user 0.001s	sys 0.001s
I20260709 13:02:56.872864  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:56.873623  1703 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:02:56.874392  9145 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:56.877378  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.877672  8875 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:56.872237  8855 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.878777  2348 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:56.880514  8855 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.815755  9085 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:56.907269  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51173.5us
I20260709 13:02:56.911741  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:56.912920  1707 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
W20260709 13:02:56.914099  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 105109us
I20260709 13:02:56.915208  1704 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:02:56.915191  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.915915  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:56.916776  8972 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:56.917351  8972 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:56.918646  8972 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.919632  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:56.919950  8916 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:56.920463  1561 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:56.922125  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3300) took 2587 ms (client timeout 1736 ms). Trace:
W20260709 13:02:56.922653  1708 rpcz_store.cc:269] 0709 13:02:54.334308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.344123 (+  9815us) service_pool.cc:224] Handling call
0709 13:02:56.921828 (+2577705us) spinlock_profiling.cc:243] Waited 2.48 s on lock 0x7b5400223b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:56.921984 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5713529088}
I20260709 13:02:56.925421  1705 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:56.926043  9127 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.926795  9121 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:56.856108  8878 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:56.809331  9001 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:56.935253  9001 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:56.936113  1554 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:02:56.936635  9084 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:56.937928  9099 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.879s	user 0.012s	sys 0.004s
I20260709 13:02:56.938885  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.939693  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:56.940507  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.940811  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.941740  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.941860  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.941982  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.942610  1628 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:56.943044  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.943276  9121 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:56.943492  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.943838  9121 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:56.944196  1628 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:56.944475  1627 raft_consensus.cc:1240] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:56.945292  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:56.946202  1635 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:56.939945  1631 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:56.952067  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.943071  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:56.957763  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 104432us
I20260709 13:02:56.944964  9121 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:56.963728  1635 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:56.915107  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:56.917313  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.970800  1558 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
W20260709 13:02:56.979028  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 69429us
I20260709 13:02:56.968389  1674 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:56.983804  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:56.984200  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 72909us
I20260709 13:02:56.986162  1710 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:56.988202  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:56.847129  8871 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:56.996562  1561 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
W20260709 13:02:56.997807  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 160129us
I20260709 13:02:56.980002  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:57.009091  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:57.007236  1675 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:56.988791  8878 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:57.020128  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:57.020714  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.021078  1561 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:57.021194  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.995431  8794 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:57.030496  2318 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:57.002838  9158 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:57.034318  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:57.039583  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 59660.8us
I20260709 13:02:57.041917  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:57.043926  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.989528  9106 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:02:57.044621  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:56.982208  9119 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:02:57.051260  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:57.032316  9099 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:57.051703  1629 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
W20260709 13:02:57.059724  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 142329us
I20260709 13:02:57.060906  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:57.049245  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:57.063354  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:57.013029  8799 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.048779  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.066929  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.053854  1554 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:57.041024  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:57.043135  9105 raft_consensus.cc:2764] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:02:57.069547  1631 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:57.067860  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.046264  8794 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:57.078943  1630 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:57.082857  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:57.051208  8992 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:57.096946  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.094071  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:57.068231  8875 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.100185  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:57.100273  2348 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:57.097007  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:57.104598  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:57.067910  9104 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:57.101042  2345 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:57.074445  9158 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:57.099668  1629 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:57.074690  9099 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:57.087266  8878 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:57.111204  1710 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:02:57.063938  9085 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:02:57.094632  9145 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:57.119050  9145 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.119505  9145 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:57.114183  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.120401  1631 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:57.103798  8387 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:57.115166  2317 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.161s (SENT)
I20260709 13:02:57.116086  8875 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.127014  8794 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:57.093393  8809 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:57.131505  1527 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:57.098095  9015 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:57.171015  1525 leader_election.cc:336] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.652s (SENT)
I20260709 13:02:57.172570  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:57.175177  1633 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:57.080477  8799 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:57.108250  9158 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.178517  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:02:57.180002  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 50131.2us
I20260709 13:02:57.179721  8954 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:57.183008  9006 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:57.183274  8954 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:57.183856  9006 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:57.184495  9006 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:57.185379  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:57.180457  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 91678us
I20260709 13:02:57.099301  9122 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:02:57.183828  1631 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:02:57.184597  8954 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.130673  8992 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:57.196410  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.196487  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.197764  1631 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:02:57.192567  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 131721us
I20260709 13:02:57.200733  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:57.202370  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.202533  1633 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
W20260709 13:02:57.203646  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.204427  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 163370us
I20260709 13:02:57.198858  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.205991  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:57.186748  9158 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:57.208148  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.218562  8804 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:57.198582  8635 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:02:57.220001  9006 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:57.192163  8817 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:57.224388  2347 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:02:57.225069  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54079.7us
I20260709 13:02:57.187332  9085 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:57.227759  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:57.228084  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.230233  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.200821  9162 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:57.237046  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.221715  8977 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:57.224788  8954 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:57.227620  8381 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:57.227989  8817 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.245405  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:57.247746  1525 leader_election.cc:336] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.702s (SENT)
I20260709 13:02:57.247568  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.229763  8809 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:57.186233  9040 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:57.249732  9119 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:57.239600  8977 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.259081  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:57.236586  9162 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:57.267205  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.267935  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:57.250084  1631 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:57.272960  1676 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:57.253165  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1340) took 1247 ms. Trace:
I20260709 13:02:57.275202  9084 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:57.247982  8381 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.275377  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.250623  9085 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.277413  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:57.278309  8875 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:57.279603  8793 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.279376  8809 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.280200  9115 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:02:57.285813  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.290122  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.289914  1636 rpcz_store.cc:276] 0709 13:02:56.028317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.036100 (+  7783us) service_pool.cc:224] Handling call
0709 13:02:57.252821 (+1216721us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b54001b7454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:57.253049 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1366207232}
I20260709 13:02:57.249459  8817 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:57.291656  9162 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:57.296418  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:57.318542  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:57.300212  8381 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:57.293164  8992 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.306608  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.287729  8977 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:57.308732  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.325356  1627 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:57.322888  2353 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
W20260709 13:02:57.298388  9165 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:57.314169  1631 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:57.312038  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:57.305785  9169 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:02:57.314172  2317 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.079s (SENT) [suppressed 37 similar messages]
W20260709 13:02:57.306645  9166 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:57.330528  1557 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:57.325722  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.307894  8989 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:57.304180  9119 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:57.334406  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:02:57.333570  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:57.266410  9099 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.346907  1629 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:57.349264  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.315655  9127 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:57.355190  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.357978  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:57.364609  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.365340  9180 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:57.365995  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:57.337214  8387 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:57.297533  9145 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.385609  9145 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:57.387964  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.388298  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.389122  2347 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:57.389647  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.389830  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:57.390334  1552 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:57.375567  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.392997  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:57.376756  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 99472.6us
I20260709 13:02:57.329633  9104 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:57.338045  9106 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:57.315488  9114 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:02:57.401063  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.406514  1631 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:57.357012  9004 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:57.354475  9115 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:57.410702  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.416921  1629 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 24.
I20260709 13:02:57.418411  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.419447  1629 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:57.412853  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.420420  1636 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:57.421847  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.422917  1636 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:57.424000  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.424798  1636 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
W20260709 13:02:57.413787  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 175071us
W20260709 13:02:57.426143  1525 leader_election.cc:336] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.726s (SENT)
W20260709 13:02:57.427573  1525 leader_election.cc:336] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.000s (SENT)
I20260709 13:02:57.428126  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:57.429363  8875 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:57.430907  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.432704  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.416695  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.434135  1628 raft_consensus.cc:2376] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18. Re-sending same reply.
I20260709 13:02:57.380340  9169 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:57.394712  9180 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:57.378942  8799 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:57.396818  9104 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.441936  1636 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 15: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:57.442041  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.442886  2351 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:57.444196  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.407330  9122 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:57.451176  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 116669us
I20260709 13:02:57.472671  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.441253  9169 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:57.459944  9104 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:57.474433  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.475553  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:57.450711  9180 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:57.479278  1601 leader_election.cc:400] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18.
I20260709 13:02:57.479622  1601 leader_election.cc:403] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:02:57.485648  1525 leader_election.cc:336] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 2.105s (SENT)
I20260709 13:02:57.486264  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:57.490402  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.497623  9040 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:57.498627  9174 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:57.499348  9174 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.501974  9168 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:57.502533  9168 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:57.503014  9168 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:57.503479  9179 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18.
W20260709 13:02:57.506114  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 67961.6us
I20260709 13:02:57.509217  8793 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:57.509836  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:57.510439  9172 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:02:57.461319  9115 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:57.511363  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:57.511785  9167 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:57.512557  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.513007  9115 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:57.491132  1630 raft_consensus.cc:2376] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18. Re-sending same reply.
I20260709 13:02:57.515542  1630 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:57.515888  1630 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:57.516738  1630 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.517999  1630 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:57.523362  8977 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.524156  1631 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:57.526082  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.484346  9119 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.527850  1629 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:02:57.553030  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 66827.1us
I20260709 13:02:57.528297  2349 raft_consensus.cc:2410] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:02:57.529323  9108 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:57.530818  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.556137  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.476857  9106 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:02:57.544467  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.552047  1630 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:02:57.558468  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.552377  1552 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:57.476486  9169 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:57.560506  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.560494  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.566414  2346 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:57.573158  8381 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.559170  8799 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:57.583861  1561 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:57.557408  9106 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:57.584151  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.589174  2347 raft_consensus.cc:2368] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19.
I20260709 13:02:57.581629  9014 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.577452  8381 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:57.592293  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:57.596979  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:57.597492  1704 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:57.597026  1701 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:57.577307  9104 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.599472  9174 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:57.599381  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:57.602631  9015 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:57.585650  8799 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.606330  9106 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:57.629181  8799 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:57.628620  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.629402  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.607501  9014 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:57.630421  1629 raft_consensus.cc:2410] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:57.631186  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.631932  1634 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:57.631877  9014 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:57.609104  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 126864us
I20260709 13:02:57.629441  9106 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:57.633863  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.629492  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.635542  1635 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:02:57.625658  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.639283  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:57.640015  2344 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:57.640636  8799 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.641750  2347 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20. Ops: [19.15-19.15]
W20260709 13:02:57.613611  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.644284  9015 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:57.645056  8799 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:57.645409  9015 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:57.646898  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.646942  9015 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:57.614612  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:57.648219  2352 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:57.620586  9158 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:57.622776  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:57.649892  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3420) took 1895 ms. Trace:
I20260709 13:02:57.624465  9180 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.624969  9085 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:57.626698  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.650610  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:57.658555  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.659330  1633 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:57.626648  1703 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:57.627036  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.627259  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.661062  1630 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:02:57.606973  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:57.649343  2352 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:57.651986  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.664142  1635 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:57.653081  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.654671  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.650271  1555 rpcz_store.cc:276] 0709 13:02:55.754071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.762145 (+  8074us) service_pool.cc:224] Handling call
0709 13:02:57.649684 (+1887539us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b5400389ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:57.649832 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2245713152}
I20260709 13:02:57.667683  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:57.668341  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.671010  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:57.671868  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.672773  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:57.675329  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:57.676251  2353 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:57.675534  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:57.685397  1633 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:57.687587  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.688055  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.696120  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:02:57.697955  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.699628  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.673187  9158 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:57.705183  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:57.705503  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:57.706200  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:57.708799  1554 raft_consensus.cc:2368] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19.
I20260709 13:02:57.708885  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:57.708014  1629 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:57.709735  1630 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:57.711599  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:57.712255  1627 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:57.713025  1634 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:57.716930  1633 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:57.719053  1557 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:57.727604  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:57.730088  2348 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 13:02:57.714802  1601 connection.cc:256] RPC call timeout handler was delayed by -1.3583606290000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 13:02:57.731462  1602 leader_election.cc:400] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19.
I20260709 13:02:57.731982  1602 leader_election.cc:403] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:57.731840  1601 connection.cc:256] RPC call timeout handler was delayed by -1.134260629000039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
I20260709 13:02:57.733565  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:57.734699  1703 raft_consensus.cc:2410] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:02:57.737778  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:57.738659  1557 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:57.715102  8977 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:57.714529  9040 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.705328  8793 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:02:57.746451  9040 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.742128  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:57.749099  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:57.750010  2353 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:57.745298  1630 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
I20260709 13:02:57.735601  8804 consensus_queue.cc:1059] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:57.754079  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:57.755792  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:57.741828  9115 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19.
I20260709 13:02:57.721864  8817 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.759687  1703 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:57.762609  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.765244  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:57.756678  8793 raft_consensus.cc:3055] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:57.805783  8793 raft_consensus.cc:740] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:57.806298  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:57.807711  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:57.808357  9115 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:57.809581  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:57.810002  1704 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 13:02:57.811120  9187 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:02:57.811229  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:57.812095  1704 raft_consensus.cc:2368] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20.
I20260709 13:02:57.753378  9127 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:57.757339  9162 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.815390  1703 raft_consensus.cc:1240] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 19. Ops: []
I20260709 13:02:57.817870  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:57.822310  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:57.815248  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 59525.2us
I20260709 13:02:57.808827  1708 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:57.839227  1710 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:02:57.820005  9162 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:57.807618  8817 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:57.813433  9122 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:57.816960  9179 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:57.873986  1706 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:57.873998  9179 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:57.875872  9179 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.817168  9040 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:57.884636  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1342) took 1805 ms. Trace:
I20260709 13:02:57.815809  9180 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:57.825757  8804 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:57.885408  1632 rpcz_store.cc:276] 0709 13:02:56.079124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.144511 (+ 65387us) service_pool.cc:224] Handling call
0709 13:02:57.884280 (+1739769us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b540010bdd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:57.884540 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1115328256}
I20260709 13:02:57.806952  8793 consensus_queue.cc:260] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.899204  8793 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:57.899971  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:57.902740  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 84878.7us
I20260709 13:02:57.904392  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:57.871313  9127 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:57.814719  9168 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:57.905946  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 155092us
I20260709 13:02:57.905968  8804 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:57.914435  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 1168) took 1087 ms. Trace:
W20260709 13:02:57.918295  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.909343  8817 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:57.910689  8664 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:57.912228  8916 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:57.936813  9169 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:57.913851  8804 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:57.945067  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.915855  9104 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:57.954005  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54198.6us
I20260709 13:02:57.972896  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:57.964314  8794 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.987684  8664 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:02:57.988400  8664 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:57.994529  9168 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:58.002300  8871 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:57.982790  8794 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:57.914939  1559 rpcz_store.cc:276] 0709 13:02:56.821434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.829132 (+  7698us) service_pool.cc:224] Handling call
0709 13:02:56.829467 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.829773 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:57.093219 (+263446us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.742743 (+649524us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b1800493060. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:02:57.908046 (+165303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.908200 (+   154us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:57.908267 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.908351 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.908486 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":584,"prepare.queue_time_us":267870,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":780,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":922,"spinlock_wait_cycles":106960256,"thread_start_us":396060,"threads_started":2}
I20260709 13:02:58.040670  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:58.017021  9169 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:58.053562  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:58.055292  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:58.056640  1557 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:58.056281  1559 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:58.058642  1554 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 13:02:58.059099  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.014036  9197 consensus_queue.cc:1059] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:02:58.055789  9162 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:58.062918  9162 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:02:58.068969  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.068765  9162 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.072264  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:58.072901  1709 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:58.073989  1704 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:58.075237  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:58.076182  1704 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:58.076802  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:58.078886  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:58.079738  2353 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:58.081935  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:58.073045  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:58.073686  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:58.093962  1703 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:58.095031  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:58.095674  1703 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:58.096493  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:02:58.099402  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.100281  9205 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.101307  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:58.101046  9205 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.102705  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:58.056249  9201 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:02:58.094540  1701 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:58.109169  1708 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:02:58.108392  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:58.112514  9145 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:58.113559  9145 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:58.114064  1703 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:58.114220  8878 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:02:58.114413  8992 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:02:58.114645  8878 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:58.117280  8878 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:58.123414  8878 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:58.137758  9204 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:58.138595  9204 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:58.121254  8794 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:58.088771  9206 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:58.114112  9145 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:58.146929  9193 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.333s	user 0.002s	sys 0.005s
I20260709 13:02:58.161284  1704 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:02:58.106315  8989 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:02:57.128813   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:58.152372  9015 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.164513  9015 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.141141  9204 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:58.145036  9197 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:02:58.176242  9179 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:58.200928  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:58.202431  8809 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:58.202960  8809 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.176425  2352 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
I20260709 13:02:58.183143  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:58.184799  1557 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:58.185235  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:58.205042  1633 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:58.189785  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:58.185487  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 168903us
I20260709 13:02:58.184037  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:58.203372  8809 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:58.208386  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 240015us
I20260709 13:02:58.224237  1633 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:02:58.210570  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:58.237123  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:58.238741  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:58.239607  1708 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:58.240600  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:58.241513  1708 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:58.247004  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:58.247941  2349 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:58.249444  1629 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:58.249886  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:58.210820  9145 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.251554  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:58.224345  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:58.105468  8793 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.254292  2344 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:58.250000  1629 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:58.254848  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:02:58.272344  1601 leader_election.cc:400] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19.
I20260709 13:02:58.274531  2345 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:02:58.276140  2345 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
W20260709 13:02:58.292210  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 196007us
W20260709 13:02:58.297838  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 94236.9us
I20260709 13:02:58.193668  8804 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.294272  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:58.311604  1552 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:58.290951  1629 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.282194  1675 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:58.318701  1629 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:58.319749  9145 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:58.320760  9145 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:02:58.321630  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 97331us
I20260709 13:02:58.324288  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:02:58.325807  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43723.8us
I20260709 13:02:58.284015  8387 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.328517  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:58.334378  9040 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:58.336650  8817 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:02:58.278815  8793 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.312177  8794 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:02:58.355244  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 77745.2us
I20260709 13:02:58.358808  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:58.315567  9169 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.307763  2349 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:58.337949  9215 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:58.335310  9162 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:02:58.381626  9169 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:58.385923  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:58.397523  1708 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:58.391815  8804 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:58.399430  1707 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:02:58.403950  1709 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:58.410127  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:58.404431  1675 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:58.338975  8817 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.338524  9040 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:58.342124  9214 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:58.416523  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:02:58.434279  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 86287.7us
I20260709 13:02:58.398427  9215 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.389195  9127 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:58.343737  8387 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:58.456722  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52407.4us
I20260709 13:02:58.466176  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:02:58.461114  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 141458us
W20260709 13:02:58.470785  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 54329.9us
I20260709 13:02:58.405743  9162 consensus_queue.cc:1243] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:02:58.434980  8817 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:58.420910  9205 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:58.478056  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:02:58.481081  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.481009  1675 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:58.484730  1710 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:58.436952  9040 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:58.455240  9214 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:58.435221  9207 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:58.411621  9197 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:02:58.491012  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:58.461591  9215 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:58.499102  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:58.499469  9213 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:02:58.499799  8871 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:58.500824  8804 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:58.501812  9205 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:58.502534  8793 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:58.503016  9168 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:58.518232  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:58.473454  9105 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.523947  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:58.524712  1710 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:02:58.526768  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:58.488162  8799 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:58.550571  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:58.550731  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:58.551980  1559 raft_consensus.cc:2410] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:58.495438  9214 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:58.559154  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:58.564831  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:58.565951  1556 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 17: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:58.560707  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:58.566987  1559 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 23: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:58.568193  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:58.574375  8794 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:58.574465  2350 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:58.588697  8387 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:58.562325  9197 consensus_queue.cc:1243] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:02:58.592110  9223 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:02:58.602670  9212 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:02:58.603776  9105 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.604624  9105 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:58.548171  9222 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:02:58.608788  9219 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.563540  8871 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:58.628571  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.650676  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1503) took 1048 ms. Trace:
I20260709 13:02:58.651350  2347 rpcz_store.cc:276] 0709 13:02:57.602572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.642807 (+ 40235us) service_pool.cc:224] Handling call
0709 13:02:57.643156 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:57.643609 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.643642 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:58.142538 (+498896us) spinlock_profiling.cc:243] Waited 9.54 ms on lock 0x7b2c0001d508. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda25fd3a11 00007fda25fe3372 00007fda27715a01 00007fda276da43a 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2
0709 13:02:58.143151 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 13:02:58.281196 (+138045us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:58.281256 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.281330 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.650309 (+368979us) raft_consensus.cc:1685] finished
0709 13:02:58.650333 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.650598 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":144,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"spinlock_wait_cycles":21952256,"thread_start_us":137253,"threads_started":1,"wal-append.queue_time_us":262430}
I20260709 13:02:58.658206  1557 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:02:58.669376  1559 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:02:58.670251  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 172591us
I20260709 13:02:58.668888  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:58.678421  2353 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.598s	user 0.020s	sys 0.000s
I20260709 13:02:58.679174  2353 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
W20260709 13:02:58.665792  9187 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:02:58.573410  8799 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:58.609438  9219 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.707991  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1700) took 1421 ms. Trace:
I20260709 13:02:58.710335  2346 rpcz_store.cc:276] 0709 13:02:57.282210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.282458 (+   248us) service_pool.cc:224] Handling call
0709 13:02:58.703513 (+1421055us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540054a454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:58.703712 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583037824}
I20260709 13:02:58.697045  9162 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:02:58.724974  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 257287us
I20260709 13:02:58.739852  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:58.741549  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:02:58.748451  9206 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.605s	user 0.003s	sys 0.010s
I20260709 13:02:58.725709  9162 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.749720  9221 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:58.750036  9222 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:58.750208  9213 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.752302  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:58.765543  2317 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:58.731446  9169 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:02:58.790772  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 309696us
W20260709 13:02:58.797101  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 120227us
I20260709 13:02:58.803226  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:58.803932  1559 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
W20260709 13:02:58.790184  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 431185us
I20260709 13:02:58.770828  9158 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.810398  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:58.784480  8664 consensus_meta.cc:287] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.796s	user 0.016s	sys 0.015s
I20260709 13:02:58.766850  9206 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.806787  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:58.808089  1629 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:02:58.808161  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:58.813207  1559 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:58.814190  1629 pending_rounds.cc:85] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 11
I20260709 13:02:58.814538  1629 pending_rounds.cc:107] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:02:58.814898  1629 raft_consensus.cc:2889] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:58.816325  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:58.819159  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:58.800716  8809 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.003s
W20260709 13:02:58.825419  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.789417  9227 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:58.827901  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:58.828279  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:58.829679  1709 raft_consensus.cc:2393] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 15: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:58.796936  9220 raft_consensus.cc:2764] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:02:58.828631  1554 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
W20260709 13:02:58.829084  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 64071us
I20260709 13:02:58.830163  9115 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:58.827138  8387 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.831817  9226 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:58.832118  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.818081  8664 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.787619  9225 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:02:58.833330  1710 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:58.853046  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:58.828258  8809 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.834993  9015 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:58.837971  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2636) took 1030 ms. Trace:
I20260709 13:02:58.842810  8871 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:58.857844  1556 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:58.858709  8809 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:58.768679  8799 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.788684  9213 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.845932  1708 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
W20260709 13:02:58.844178  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 91910us
I20260709 13:02:58.855665  8387 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.866878  1525 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:58.868449  9221 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:58.858855  2348 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:02:58.849316  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:58.869926  8387 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:58.869676  9206 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:02:58.869254  9222 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:02:58.854635  1707 rpcz_store.cc:276] 0709 13:02:57.807912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.809245 (+  1333us) service_pool.cc:224] Handling call
0709 13:02:58.837724 (+1028479us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b540036ddd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:58.837872 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458596736}
I20260709 13:02:58.878029  8664 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:58.865278  9213 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:58.823927  9127 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:58.869424  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:58.877907  1702 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:02:58.869797  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:58.891587  1552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2640) took 1327 ms. Trace:
W20260709 13:02:58.887017  9179 consensus_meta.cc:287] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.677s	user 0.008s	sys 0.011s
I20260709 13:02:58.820988  9232 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:58.905051  1552 rpcz_store.cc:276] 0709 13:02:57.564131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.564445 (+   314us) service_pool.cc:224] Handling call
0709 13:02:57.579545 (+ 15100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.312377 (+732832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.891276 (+578899us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b54002706d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:58.891513 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681886848}
W20260709 13:02:58.921777  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 114996us
W20260709 13:02:58.908459  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 55539us
I20260709 13:02:58.827683  9162 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:58.924551  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:58.928604  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:58.930469  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 81207.7us
I20260709 13:02:58.894564  9207 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:58.817900  9158 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:58.932832  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:58.848845  9234 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:02:58.933382  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 64009.2us
I20260709 13:02:58.934185  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:58.937261  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:58.893882  9198 raft_consensus.cc:3055] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:58.903376  9229 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:58.941105  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:58.935081  1709 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:58.935770  1555 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:58.934718  2352 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:02:58.946825  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:58.949177  9179 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:58.949592  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:58.950088  9213 raft_consensus.cc:2764] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:58.949940  9179 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:58.950402  2353 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:58.952482  9179 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:58.942032  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:58.923218  9232 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:58.960274  1704 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:58.933087  9206 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:58.963519  1557 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:02:58.933856  9207 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:58.964092  1557 pending_rounds.cc:85] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 9
I20260709 13:02:58.968772  1557 pending_rounds.cc:107] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:02:58.969056  1557 raft_consensus.cc:2889] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:58.966257  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:02:58.939947  9113 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:58.891223  8817 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:58.939055  9198 raft_consensus.cc:740] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:58.961149  2317 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:58.957851  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:58.938953  9197 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.023321  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.000125  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:58.943107  9226 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:02:59.037035  1707 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:58.942025  9225 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:59.001380  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1327) took 1612 ms. Trace:
I20260709 13:02:59.013895  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:58.997735  1601 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:59.019040  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:59.035723  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 89708.3us
I20260709 13:02:58.956130  9115 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:58.959859  9127 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:59.018651  9229 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:59.059607  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:58.959437  9227 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:59.062280  1706 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:59.051800  2351 rpcz_store.cc:276] 0709 13:02:57.388598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.400616 (+ 12018us) service_pool.cc:224] Handling call
0709 13:02:59.001146 (+1600530us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b54009902d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:59.001282 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93255040}
I20260709 13:02:59.044082  1526 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:58.963846  9232 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:59.068459  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.068553  1557 raft_consensus.cc:1217] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Deduplicated request from leader. Original: 16.11->[17.12-17.12]   Dedup: 17.12->[]
I20260709 13:02:59.090358  1557 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 13:02:59.070792  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 71002.6us
I20260709 13:02:59.092662  1525 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:02:59.053320  8793 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:59.054126  9240 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:02:59.061841  9113 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:02:59.080245  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 119109us
W20260709 13:02:59.082669  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.048429  9230 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:59.074327  9225 raft_consensus.cc:697] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:58.968132  9040 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:59.074059  9205 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.548s	user 0.005s	sys 0.010s
W20260709 13:02:59.075214  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 77550.3us
I20260709 13:02:59.088444  9127 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:59.103278  9214 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.513s	user 0.013s	sys 0.003s
I20260709 13:02:58.971266  8916 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.117s
I20260709 13:02:59.072024  9229 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.091516  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:59.045958  9197 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:59.136886  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45442.6us
I20260709 13:02:59.136277  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:59.113282  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:59.039083  9233 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:02:59.054046  8878 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:02:59.082216  9119 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.097254  9226 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:59.092792  9206 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:59.120915  8977 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.099968  9213 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:59.134352  9229 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:59.128671  9015 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.128413  9236 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:59.133055  9113 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.133445  9158 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:59.134357  9217 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:59.135598  9127 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.100528  9219 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.137784  1526 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:02:59.215982  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 78277.6us
I20260709 13:02:59.074261  9115 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.112857  9230 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.205644  9226 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:59.126701  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.172713  1561 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:02:59.064880  9162 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:02:59.234030  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.174284  9119 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.161576  8664 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:59.077067  9207 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.135993  9214 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.185310  9206 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:59.071210  9234 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:02:59.195323  9213 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:02:59.225561  9222 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:02:59.249517  9239 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:02:59.251799  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.234097  8989 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:59.130455  9205 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:59.123883  9198 consensus_queue.cc:260] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.253844  8989 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.255366  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2635) took 1447 ms. Trace:
I20260709 13:02:59.256242  8989 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.258153  9119 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:59.258998  9105 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:59.218238  9127 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.261323  9219 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:59.261711  9127 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.263154  9115 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.263535  9245 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:59.263818  9115 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.263986  9127 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.264160  9245 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:02:59.240130  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 113456us
I20260709 13:02:59.268250  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.189199  8977 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.203183  9015 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.223582  8809 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:59.227234  9230 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.226775  9220 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:59.245322  1709 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:02:59.245761  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 182420us
W20260709 13:02:59.247921  9209 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.015s	user 0.007s	sys 0.000s
I20260709 13:02:59.247242  9213 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.248449  9234 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:59.255760  1702 rpcz_store.cc:276] 0709 13:02:57.807597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.852957 (+ 45360us) service_pool.cc:224] Handling call
0709 13:02:57.867036 (+ 14079us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:58.900745 (+1033709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.255055 (+354310us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400077e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:02:59.255279 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2322483968}
I20260709 13:02:59.239292  8664 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:59.243753  9235 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
W20260709 13:02:59.294961  9242 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:02:59.297616  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:59.315282  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:59.318701  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 205368us
W20260709 13:02:59.319387  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51185.2us
I20260709 13:02:59.185110  9227 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.329442  2317 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.323119  9233 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:59.331632  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.236488  9226 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:59.338289  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.339840  9226 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.340068  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.341200  1702 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:59.345654  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.290832  9220 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.293390  9207 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:59.362792  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:59.365685  1634 raft_consensus.cc:2393] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:02:59.367449  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:59.295214  9015 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:59.323696  8664 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.374884  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.375458  9226 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:59.375542  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46229.4us
I20260709 13:02:59.376147  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:59.378700  1553 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:59.315692  9234 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:59.382067  1556 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:59.374112  9220 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:59.383541  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:59.417738  1635 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.390398  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:59.392019  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:59.393731  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.398906  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.399744  1628 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 23: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:02:59.397212  1632 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:59.408555  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:59.410751  1630 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
W20260709 13:02:59.423354  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.421362  1633 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:59.373436  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:59.238425  9225 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.442597  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:02:59.417150  9235 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.065s	user 0.002s	sys 0.000s
I20260709 13:02:59.447038  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:59.344871  9127 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:02:59.453605  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.443408  1630 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:59.445468  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.450162  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:59.455431  1633 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
W20260709 13:02:59.457631  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 48628.8us
I20260709 13:02:59.458313  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.459067  1633 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:02:59.455099  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.454480  1632 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:02:59.459887  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:59.460474  1633 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:59.456151  1634 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:59.427039  9040 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.428748  9197 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:02:59.464731  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 65874.2us
I20260709 13:02:59.465549  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.443413  8916 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:59.446645  9113 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.480731  1635 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:59.482784  1630 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:02:59.485292  9245 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.485826  9015 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:02:59.489995  9015 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.492264  1630 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:59.492690  1630 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:59.473397  8809 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:59.432605  9206 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.491626  9127 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:59.504722  9246 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.516175  1601 leader_election.cc:400] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20.
I20260709 13:02:59.516737  1601 leader_election.cc:403] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:02:59.519129  1633 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:02:59.526957  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:59.548172  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 95055.8us
I20260709 13:02:59.549863  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:59.499416  9255 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:59.506318  9249 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:02:59.557363  9250 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:02:59.510116  9214 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.524750  8878 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:02:59.564045  9115 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:59.565006  1627 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:02:59.567032  9219 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:02:59.567691  9219 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.570359  9015 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:02:59.572091  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:59.572604  9249 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:59.573565  9249 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.570376  9219 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.574841  9249 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:59.575896  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.475994  9234 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.576782  9255 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.577832  9254 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:59.578258  9127 consensus_queue.cc:1059] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:59.578670  8977 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.578881  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:59.580552  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.491426  9113 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.589164  1701 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:02:59.601164  1554 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
W20260709 13:02:59.603826  9215 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.057s	user 0.012s	sys 0.004s
I20260709 13:02:59.548404  9251 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20.
W20260709 13:02:59.609956  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 210545us
I20260709 13:02:59.620048  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:02:59.603422  9232 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.005s
W20260709 13:02:59.635499  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 81580.2us
I20260709 13:02:59.638809  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.631944  2317 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.645347  9245 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:59.629973  8977 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:59.650068  8977 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:59.650683  8977 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:59.650911  9248 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:02:59.616509  9215 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.653458  1703 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:02:59.659052  9113 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:59.658588  9215 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:59.661356  9219 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:59.662747  1630 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.663230  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:59.664158  9219 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:59.664158  1630 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:59.669827  9251 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:59.671789  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.625363  8793 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.516s	user 0.012s	sys 0.000s
I20260709 13:02:59.610571  9254 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 21 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:59.689795  1630 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:59.526172  9246 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:59.688071  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45843.6us
I20260709 13:02:59.693782  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:59.688334  8793 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:59.708992  8878 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:02:59.728654  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.730657  1630 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
W20260709 13:02:59.731930  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 57067.6us
I20260709 13:02:59.633719  9232 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.724613  9183 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
W20260709 13:02:59.722651  9236 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.510s	user 0.012s	sys 0.000s
I20260709 13:02:59.739369  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.720914  8817 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:02:59.748292  1706 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:59.750820  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.733098  9105 consensus_queue.cc:1059] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:02:59.762492  9105 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:59.762974  9105 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:02:59.764187  9105 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.722520  9127 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:59.765969  9127 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:59.765686  9254 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.767120  1560 raft_consensus.cc:2410] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 13:02:59.767073  9127 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.763013  9201 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:59.770059  9201 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:59.771046  9201 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:02:59.775374  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 1243) took 2693 ms (client timeout 2374 ms). Trace:
W20260709 13:02:59.775902  1705 rpcz_store.cc:269] 0709 13:02:57.082344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.082594 (+   250us) service_pool.cc:224] Handling call
0709 13:02:59.775124 (+2692530us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54000cb7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:59.775284 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4497789184}
I20260709 13:02:59.778748  9246 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:02:59.783819  9221 consensus_meta.cc:287] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.915s	user 0.013s	sys 0.000s
I20260709 13:02:59.784406  9221 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:59.785099  1558 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:59.785362  9105 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.786303  9255 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:02:59.786600  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:02:59.789274  9247 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:59.789767  9247 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:59.789975  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.790679  9247 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.792303  9247 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:59.799595  9198 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:59.800606  9226 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:59.801819  9258 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.311s	user 0.002s	sys 0.000s
I20260709 13:02:59.802557  8916 raft_consensus.cc:3055] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:59.802980  8916 raft_consensus.cc:740] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:59.803865  8916 consensus_queue.cc:260] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.807240  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.808015  1634 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:02:59.808981  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.749372  1635 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:59.767620  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.812463  1627 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:02:59.819459  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.749719  9236 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:59.746096  9245 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 13:02:59.823251  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.825182  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:59.827340  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:59.829811  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.830543  1703 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:02:59.835271  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:59.836333  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.837152  1635 raft_consensus.cc:2410] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 13:02:59.838253  1556 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:59.841248  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.842037  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:02:59.845422  1627 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.842752  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.843113  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.847275  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:02:59.847903  1630 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:02:59.849880  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.851214  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.851354  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:59.852403  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:02:59.821408  9230 consensus_meta.cc:287] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.003s
I20260709 13:02:59.822893  9245 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:02:59.841562  9229 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.639s	user 0.003s	sys 0.010s
I20260709 13:02:59.823333  9168 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.857092  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:59.863214  1674 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:59.878332  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.887683  1634 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:59.848806  8916 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:02:59.892243  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.850860  9224 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:02:59.904268  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 149669us
I20260709 13:02:59.906695  2317 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:59.908624  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:59.854933  9219 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:59.898218  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.896312  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.900835  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:02:59.849583  9255 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:59.886124  9226 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:59.942641  2317 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:59.936441  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:02:59.885497  9245 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:02:59.948212  1556 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:02:59.931758  8664 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.001s
I20260709 13:02:59.875151  9197 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:02:59.959086  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:59.972548  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:02:59.974468  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:59.959945  1555 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 13:02:59.904443  9224 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:02:59.961700  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:02:59.877884  9230 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.984933  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:02:59.944970  9145 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:59.899533  9168 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:02:59.958573  9215 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:02:59.975706  9207 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.001s
I20260709 13:02:59.957430  9236 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:00.008633  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:02:59.992602  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3458) took 1443 ms. Trace:
I20260709 13:03:00.003913  1634 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:59.999195  1674 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:00.024663  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:02:59.895903  9229 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.028021  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:00.014094  1561 rpcz_store.cc:276] 0709 13:02:58.549348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.550464 (+  1116us) service_pool.cc:224] Handling call
0709 13:02:59.992385 (+1441921us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54003c0fd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:02:59.992526 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426823424}
I20260709 13:03:00.031699  1704 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:02:59.993395  9220 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.001s
I20260709 13:03:00.035967  1629 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:00.030645  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:02:59.946794  9219 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.058264  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:00.059188  1634 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:59.996857  9015 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:03:00.053608  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 157288us
W20260709 13:03:00.055480  9262 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:03:00.056088  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47572.6us
I20260709 13:03:00.076050  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:00.056437  8387 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:00.011832  9215 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:02:59.978281  8664 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:02:59.987221  9278 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:00.007921  9113 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:00.051275  9217 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:03:00.063539  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 104971us
I20260709 13:03:00.064992  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:00.076699  1708 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 13:03:00.016582  9232 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:00.013134  9183 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:00.056589  9205 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:00.023077  9099 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:00.051981  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:00.006064  8804 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.072257  8989 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:00.096163  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:00.112596  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 47710.5us
I20260709 13:03:00.153928  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:00.092024  8387 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:00.158826  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:00.164192  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 165017us
I20260709 13:03:00.166293  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:00.166903  1553 raft_consensus.cc:2368] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20.
I20260709 13:03:00.100616  9278 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.168232  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:00.168535  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:00.169128  2351 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:03:00.098367  9215 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:00.172498  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:00.171773  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:00.170399  9249 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.005s
I20260709 13:03:00.162367  9183 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:00.161146  9232 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.094022  9245 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.104192  9217 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:00.173735  9278 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:00.164582  8387 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:00.108942  9206 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:00.152233  9224 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.187625  9206 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:00.187988  9224 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:00.188163  9206 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:00.162628  9225 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:00.184537  9232 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.190296  9217 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.191828  2348 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:03:00.191346  9183 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.195855  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:00.196834  2346 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:03:00.190512  9225 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.200275  9232 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:00.202311  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:00.204864  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:00.205084  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:00.205631  2344 raft_consensus.cc:2376] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19. Re-sending same reply.
I20260709 13:03:00.206105  2346 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:00.152499  9197 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:00.212716  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46466.9us
I20260709 13:03:00.213682  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:00.058665  9220 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:00.215831  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:00.218403  2345 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:03:00.152643  9226 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:03:00.230628  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3473) took 1400 ms. Trace:
I20260709 13:03:00.231652  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:00.200439  9225 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:00.232179  1560 rpcz_store.cc:276] 0709 13:02:58.829885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.831046 (+  1161us) service_pool.cc:224] Handling call
0709 13:03:00.228703 (+1397657us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b540038a4d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:00.230551 (+  1848us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2097745664}
I20260709 13:03:00.208845  9157 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:00.228421  9205 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:00.154520  8804 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.214820  9266 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:00.234977  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:00.216825  9015 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:00.255663  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:00.267235  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 171079us
I20260709 13:03:00.242147  9099 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:00.297956  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:00.270829  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:00.300851  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:00.257910  9205 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:00.305238  1674 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:00.269694  9157 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:00.312927  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 207180us
I20260709 13:03:00.300345  9099 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.244889  9113 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:00.329011  9113 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:00.311517  9157 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.336149  9157 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:00.339636  9270 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.370s	user 0.002s	sys 0.000s
W20260709 13:03:00.342372  9276 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:03:00.343802  9158 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:00.344312  9158 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:00.344761  9158 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:00.346495  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:00.347496  9229 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:00.348311  8664 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:00.350242  9274 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:03:00.353834  1627 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:03:00.355609  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:00.356951  1627 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:00.305461  9284 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:00.372402  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:00.373312  1627 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:00.325945  9099 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:00.324785  9230 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:00.372772  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 57512.4us
I20260709 13:03:00.335263  9168 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:00.383440  9285 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:03:00.384479  1634 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 25.
I20260709 13:03:00.384688  9220 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:00.388849  1634 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:00.392163  1634 raft_consensus.cc:1217] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Deduplicated request from leader. Original: 12.11->[13.12-17.14]   Dedup: 13.12->[15.13-17.14]
I20260709 13:03:00.392714  1634 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: [13.12-17.14]
I20260709 13:03:00.394126  1634 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:00.395314  1634 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 21. Current term is 22. Ops: [19.13-21.14]
I20260709 13:03:00.395769  9220 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:03:00.397063  8989 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:00.397603  9220 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:03:00.398169  8989 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:00.395203  9215 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:00.400734  9215 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:00.401427  1709 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:00.401890  9205 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.321391  9265 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.288s
I20260709 13:03:00.404678  9289 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:00.411660  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:00.375881  9219 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:00.419148  1527 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:00.394312  9278 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:00.453732  9273 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.285s	user 0.002s	sys 0.000s
I20260709 13:03:00.439033  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:00.455011  2344 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:00.457962  8804 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:00.458989  1705 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:03:00.424886  9288 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:00.461944  9288 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:00.462962  9288 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.410087  8664 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:03:00.463976  8664 consensus_queue.cc:1243] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:00.437991  9265 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:00.464392  1558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3474) took 1560 ms (client timeout 2067 ms). Trace:
W20260709 13:03:00.465888  9247 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.673s	user 0.012s	sys 0.004s
W20260709 13:03:00.453555  9286 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:03:00.466687  9201 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:00.464612  9265 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.469820  9265 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:00.417627  9289 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:00.470312  1704 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:03:00.470942  9289 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.471625  9223 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:00.472350  9223 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.441102  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:00.447594  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3343) took 1421 ms. Trace:
I20260709 13:03:00.475244  1708 rpcz_store.cc:276] 0709 13:02:59.026117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.047523 (+ 21406us) service_pool.cc:224] Handling call
0709 13:02:59.070317 (+ 22794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.110116 (+1039799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.447204 (+337088us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540016c954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:00.447502 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2311990528}
I20260709 13:03:00.477525  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:00.478381  1554 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:03:00.439708  9295 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:03:00.464993  1558 rpcz_store.cc:269] 0709 13:02:58.903971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.904187 (+   216us) service_pool.cc:224] Handling call
0709 13:03:00.464194 (+1560007us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54000bd454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:00.464325 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1966791552}
I20260709 13:03:00.479032  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:00.479950  2352 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:03:00.480171  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:00.483565  2347 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:03:00.485077  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:00.485930  1703 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:00.487679  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:00.496177  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:00.497644  1561 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:00.498396  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:00.450331  8977 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.799s	user 0.014s	sys 0.000s
W20260709 13:03:00.452427  9281 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.190s	user 0.002s	sys 0.000s
W20260709 13:03:00.447186  9105 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.001s
I20260709 13:03:00.511370  1708 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:03:00.500087  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 199732us
I20260709 13:03:00.433401  9219 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:00.518013  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:00.518707  1628 raft_consensus.cc:2368] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22.
I20260709 13:03:00.523725  1706 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:00.482422  9295 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.529492  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:00.518285  8977 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.535778  8977 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:00.537604  9278 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:00.538184  9223 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:00.528752  9295 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:00.538919  9099 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:00.443609  9206 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:00.542537  9157 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:00.544426  9206 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:00.544788  9157 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:00.546368  2351 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:00.526893  9219 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:00.558765  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 253466us
I20260709 13:03:00.565811  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:00.567992  2344 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:00.569155  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:00.569830  2344 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:00.550393  9206 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.548857  9099 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:00.575002  2318 leader_election.cc:400] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20.
I20260709 13:03:00.549795  8817 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:03:00.576551  9226 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.003s
I20260709 13:03:00.562042  9221 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:00.580364  2318 leader_election.cc:403] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:03:00.580616  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:00.587996  2352 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:00.582967  8817 consensus_queue.cc:1243] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:00.604384  9219 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:00.605631  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.606998  9215 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:00.618866  9099 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:00.623643  9099 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:00.582309  9206 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.625270  9168 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.625710  9298 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.625900  9168 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:03:00.626298  9298 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.626801  9294 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:03:00.627720  9198 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.006s
I20260709 13:03:00.625128  9157 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:00.628659  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:00.628732  9157 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.629324  2346 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:03:00.629433  1560 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:00.627388  9294 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:00.636713  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 61817us
I20260709 13:03:00.639149  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:00.618867  9223 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:00.662961  9223 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:00.664439  9223 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.687127  9223 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:00.688213  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3288) took 1302 ms. Trace:
W20260709 13:03:00.690524  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.691213  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3594) took 1326 ms. Trace:
I20260709 13:03:00.691630  1631 rpcz_store.cc:276] 0709 13:02:59.365020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.370155 (+  5135us) service_pool.cc:224] Handling call
0709 13:03:00.690426 (+1320271us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b5800135698. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c6f20ab 00007fda2c6f2131 00007fda2c6b3fc0 00007fda2c69e929 00007fda2c69e3d2 00007fda25c15bd4
0709 13:03:00.690664 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.690864 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.690894 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.690936 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:00.690966 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.691029 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.691153 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8814464}
I20260709 13:03:00.693495  9127 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.675598  8809 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:00.694157  8809 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.694649  8809 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:00.694094  9127 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.694792  9197 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.678164  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:00.696152  1560 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 16.
I20260709 13:03:00.675774  9226 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:00.679147  1553 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:00.680084  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:00.637845  9219 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149) with higher term. Message: Invalid argument: T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20.
I20260709 13:03:00.695780  9197 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.690384  9277 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:00.700278  1527 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:00.685678  8664 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:00.710595  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:00.725603  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.757200  9040 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.757649  9226 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:00.758051  9040 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:00.769904  9267 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.779s	user 0.002s	sys 0.003s
I20260709 13:03:00.771116  8664 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:00.758903  9303 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:03:00.798314  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3292) took 1411 ms. Trace:
I20260709 13:03:00.798839  1629 rpcz_store.cc:276] 0709 13:02:59.387284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.466552 (+ 79268us) service_pool.cc:224] Handling call
0709 13:02:59.484560 (+ 18008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.036392 (+551832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.798021 (+761629us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b540017b954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:00.798241 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1266491648}
I20260709 13:03:00.816287  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:00.790010  9303 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:03:00.792451  9277 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.792430  9213 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 18.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 13:03:00.742576  9230 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.073s
I20260709 13:03:00.821110  9220 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.827049  9288 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:03:00.862560  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46385.9us
W20260709 13:03:00.865640  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 711341us
I20260709 13:03:00.866860  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:00.867699  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 157150us
W20260709 13:03:00.868383  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.875828  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:00.877952  2318 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:00.885773  1554 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:00.874851  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 174602us
I20260709 13:03:00.888844  9294 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:00.894805  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:00.896968  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:00.894825  1525 leader_election.cc:400] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22.
I20260709 13:03:00.898864  2352 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:03:00.900872  2344 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
W20260709 13:03:00.902673  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.849440  9288 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:00.912056  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:00.912827  1628 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:03:00.688711  1636 rpcz_store.cc:276] 0709 13:02:59.385886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.388809 (+  2923us) service_pool.cc:224] Handling call
0709 13:02:59.389430 (+   621us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:02:59.408255 (+ 18825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.408293 (+    38us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:02:59.414068 (+  5775us) log.cc:844] Serialized 104 byte log entry
0709 13:02:59.795821 (+381753us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b1800307c60. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:02:59.953824 (+158003us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:00.115750 (+161926us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.152542 (+ 36792us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.351616 (+199074us) spinlock_profiling.cc:243] Waited 9.64 ms on lock 0x7b1800307c60. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1
0709 13:03:00.459284 (+107668us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b1800307c60. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:03:00.686649 (+227365us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b1800307c60. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287 00007fda276f5208
0709 13:03:00.687164 (+   515us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.687542 (+   378us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.687702 (+   160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.687832 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:00.687890 (+    58us) raft_consensus.cc:1685] finished
0709 13:03:00.687905 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.688107 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56194,"prepare.queue_time_us":177612,"prepare.run_cpu_time_us":161,"prepare.run_wall_time_us":166,"raft.queue_time_us":598346,"raft.run_cpu_time_us":601,"raft.run_wall_time_us":8127,"spinlock_wait_cycles":144645760,"thread_start_us":294277,"threads_started":2,"wal-append.queue_time_us":661566}
I20260709 13:03:00.904522  1631 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:03:00.906575  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:00.914376  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:00.914659  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:00.915304  1634 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:00.915259  1636 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:03:00.825426  9277 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:00.890985  8989 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:03:00.911072  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44251.1us
I20260709 13:03:00.883677  9221 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.873818  9213 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:00.935896  1601 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:00.917585  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:00.936774  1628 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:00.917977  2345 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:03:00.862902  9303 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:00.893916  9214 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:00.911577  9215 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.924027  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:00.877573  9217 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:00.929340  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:00.925661  9219 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.931599  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:00.923326  1631 pending_rounds.cc:85] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 14
I20260709 13:03:00.903515  9095 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:00.858196  9220 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.947533  1629 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:00.945760  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:00.950793  1635 pending_rounds.cc:85] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:03:00.958838  1635 pending_rounds.cc:107] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:03:00.959120  1635 raft_consensus.cc:2889] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:00.931118  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:00.916694  9179 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:00.920560  9278 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:00.921466  9288 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:00.953516  1631 pending_rounds.cc:107] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:03:00.971004  1631 raft_consensus.cc:2889] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:00.928126  8992 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:00.956586  1676 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:00.941009  9303 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:00.970386  9243 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:00.968266  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:00.944590  9214 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.928015  9225 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 13:03:00.963070  8387 consensus_meta.cc:287] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.777s	user 0.009s	sys 0.013s
I20260709 13:03:00.941504  9226 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:03:00.976755  9157 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:00.955487  9227 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:00.983206  2345 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:00.971104  9280 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.002151  9168 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:00.966766  9219 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:00.988821  9099 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:00.966668  9233 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:01.027004  1703 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.531s	user 0.017s	sys 0.002s
I20260709 13:03:01.117220  1703 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:01.028280  9225 consensus_queue.cc:1243] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:00.951848  9221 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:00.969228  9095 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 25 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:01.029510  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:01.141721  2352 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:01.025364  1629 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
W20260709 13:03:01.074882  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 187025us
W20260709 13:03:01.075884  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 189259us
I20260709 13:03:00.998615  9245 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:01.079254  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:00.949664  9213 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.144411  1527 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:00.974860  9278 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:01.085860  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.963471  9236 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:03:00.968421  1634 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:03:01.009382  9214 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:01.014300  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:00.985771  9206 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.226640  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:01.226959  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:01.109616  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:01.228085  2344 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:03:01.228005  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 81569.2us
I20260709 13:03:00.995711  9243 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:01.112552  1628 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:00.955794  9215 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:01.126084  2378 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:43857 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:43857 timed out after 15.000s (SENT)
I20260709 13:03:01.131422  9220 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.035079  9197 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.113494  8992 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:01.339156  1708 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:03:01.137796  9179 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:01.036319  8387 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.132829  9303 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:01.134063  9266 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.866s	user 0.010s	sys 0.009s
I20260709 13:03:01.161423  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:03:01.082029  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 113799us
I20260709 13:03:01.111560  9280 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:01.386317  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.082490  9296 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:03:01.210575  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.351845  8916 consensus_meta.cc:287] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.447s	user 0.009s	sys 0.014s
I20260709 13:03:01.190565  1703 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:01.219760  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 205497us
I20260709 13:03:01.223191  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:00.980916  9040 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.113857  8664 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:01.338549  1706 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:01.341792  9127 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.350448  1709 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:01.216861  8989 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:03:01.328505  9158 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.983s	user 0.024s	sys 0.007s
I20260709 13:03:01.361647  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:01.406776  1702 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:03:01.339010  9206 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.323837  9243 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:01.338663  9197 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.365932  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:01.412696  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:01.366298  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 455813us
I20260709 13:03:01.315321  9183 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:01.379937  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:01.352773  9265 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.882s	user 0.025s	sys 0.002s
W20260709 13:03:01.383973  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 222542us
I20260709 13:03:01.391789  1630 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:01.419730  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:01.218305  8793 consensus_queue.cc:1059] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.145s
I20260709 13:03:01.186558  9249 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.177s
W20260709 13:03:01.393237  1524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.080s	sys 0.084s
W20260709 13:03:01.393947  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.425655  1524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.080s	sys 0.085s
I20260709 13:03:01.351537  8878 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.349979  9319 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:01.186151  9233 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:01.366797  9307 log.cc:927] Time spent T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:03:01.351372  8992 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:01.363982  9179 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.408752  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:01.325744  9314 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:03:01.367115  9289 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:01.417748  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:01.423800  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:01.394670  9266 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:01.414062  9158 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.428824  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:01.465987  8809 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.771s	user 0.018s	sys 0.006s
I20260709 13:03:01.532174  9246 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.602860  9246 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.476990  8387 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:01.603518  1453 catalog_manager.cc:5697] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
W20260709 13:03:01.604897  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 196141us
I20260709 13:03:01.603922  9205 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:01.605523  9205 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:01.606352  9205 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.607592  9205 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:01.464944  9215 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.608260  1453 catalog_manager.cc:5697] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 3 to 2, term changed from 11 to 15, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from VOTER to NON_VOTER, now has a pending config: VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194), VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 15 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:01.611199  9246 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:01.611547  9179 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.428517  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:01.431675  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:01.479017  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:01.416991  9303 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:01.425709  9206 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:01.443154  9295 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.903s	user 0.018s	sys 0.003s
I20260709 13:03:01.694033  9295 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.694706  9206 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.428565  9226 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:01.451002  9326 raft_consensus.cc:3055] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:01.431918  9290 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.185s
I20260709 13:03:01.429037  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:01.723582  1634 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
I20260709 13:03:01.725235  1634 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:03:01.725840  9220 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.728587  8793 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:01.729332  8793 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.732120  8793 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.733726  9290 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.734493  9290 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.735054  9316 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:01.444221  9278 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.453933  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:01.738090  9290 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:01.738318  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:01.739408  9316 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:01.739444  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:01.740458  1705 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:01.741353  9316 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:01.423040  9318 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.203s
I20260709 13:03:01.744168  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:01.413903  8989 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:01.745962  9316 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:01.746083  8989 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.428519  2344 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 13:03:01.751288  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:01.460144  9219 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:01.419025  9095 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.444171  9249 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:01.410813  9296 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:01.477195  9221 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:01.411935  9277 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:01.426573  9197 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.452428  9213 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:01.454679  1560 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:03:01.409931  8664 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:01.617444  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 1199) took 1132 ms. Trace:
I20260709 13:03:01.463711  9265 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.645246  1706 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:01.623195  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:01.453744  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:01.469420  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:01.772887  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:01.773226  1561 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:01.773857  2344 raft_consensus.cc:2393] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:01.621528  1704 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:01.428232  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:01.453187  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:01.774907  9328 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.776085  9219 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:01.775734  9328 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.776643  9219 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:01.777137  9219 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:01.612339  1453 catalog_manager.cc:5697] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 12 to 14, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:01.471086  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:01.779630  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:01.780081  1555 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:01.780718  9329 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:01.781545  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:01.775182  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:01.772156  1632 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:03:01.782203  1560 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:01.782781  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:01.471603  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:01.471513  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:01.783563  1632 raft_consensus.cc:1217] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Deduplicated request from leader. Original: 12.11->[13.12-17.14]   Dedup: 13.12->[15.13-17.14]
I20260709 13:03:01.612373  1629 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:01.772325  8664 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.786012  8664 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:01.542590  1674 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:01.775000  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 346675us
W20260709 13:03:01.787619  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 245006us
I20260709 13:03:01.788475  1674 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:01.627452  2345 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:01.486047  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:01.677448  2347 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 15.
W20260709 13:03:01.760329  9217 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.808s	user 0.013s	sys 0.000s
I20260709 13:03:01.678700  1702 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:01.784294  1632 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: [13.12-17.14]
I20260709 13:03:01.678097  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:01.530279  9099 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.681468  9315 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:01.681993  8817 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.793915  9321 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:01.806160  8817 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.806756  9321 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:01.681905  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:01.807964  9321 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.808954  8817 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:01.601087  9319 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.810307  9298 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.812008  9319 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:01.623473  9289 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.792842  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:01.814687  9289 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:01.683679  9233 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:01.490005  9230 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:01.793715  9266 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:03:01.815598  9230 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.816246  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1379) took 1709 ms. Trace:
I20260709 13:03:01.781311  9329 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.817875  9230 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:01.429571  9183 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:01.819128  9183 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.819865  1555 raft_consensus.cc:3060] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:01.820168  1601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.821799  9282 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.673769  8804 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:03:01.483471  9288 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.000s
I20260709 13:03:01.466567  9280 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.822338  9282 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.822490  8804 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.822906  9280 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.823518  9280 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.824602  9282 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:01.824990  8804 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.687427  9234 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:01.698367  9295 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.674690  8992 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:01.678927  8878 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.679054  9314 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:01.702065  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:01.602777  8809 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.622910  9158 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:01.718930  9214 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.768162  1554 rpcz_store.cc:276] 0709 13:03:00.485505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.486199 (+   694us) service_pool.cc:224] Handling call
0709 13:03:00.499807 (+ 13608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.893490 (+393683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.617169 (+723679us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b5400313154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:01.617373 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":886632320}
I20260709 13:03:01.782227  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:01.675465  1675 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:01.795203  1633 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:03:01.789465  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:01.919150  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 129699us
I20260709 13:03:01.920735  1559 raft_consensus.cc:2410] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 21 because replica has last-logged OpId of term: 20 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 17.
I20260709 13:03:01.819629  9329 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:01.426988  9127 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.833770  1556 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:03:01.829428  9260 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:03:01.761343  9245 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.002s
I20260709 13:03:01.831522  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:01.804047  1705 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:03:01.801196  1600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.028s	sys 0.028s
W20260709 13:03:01.931059  1600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.028s	sys 0.028s
I20260709 13:03:01.805014  9322 raft_consensus.cc:2804] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:01.932371  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:03:01.932915  1628 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.051s	sys 0.033s
W20260709 13:03:01.934041  1628 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.051s	sys 0.033s
I20260709 13:03:01.726392  9220 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:01.860692  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.907184  9225 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.825201  9282 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:01.835364  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:01.760462  9326 raft_consensus.cc:740] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:01.871058  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:01.816628  1627 rpcz_store.cc:276] 0709 13:03:00.107216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.374224 (+267008us) service_pool.cc:224] Handling call
0709 13:03:00.374703 (+   479us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:00.697330 (+322627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.697367 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:00.698117 (+   750us) log.cc:844] Serialized 52 byte log entry
0709 13:03:00.888602 (+190485us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:00.890485 (+  1883us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.815339 (+924854us) spinlock_profiling.cc:243] Waited 6.27 ms on lock 0x7b1800233140. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda24567c0a 00007fda2dca7219 00007fda245efa40 00007fda245ea4e4 00007fda245ea455 00007fda2b0d23c7 00007fda276e49a7 00007fda276f40bf 00007fda276f4038
0709 13:03:01.815582 (+   243us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.815821 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.815896 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.816019 (+   123us) raft_consensus.cc:1685] finished
0709 13:03:01.816033 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.816187 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":359699,"prepare.queue_time_us":1163,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":340629,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":11971,"spinlock_wait_cycles":14416640,"thread_start_us":189965,"threads_started":1,"wal-append.queue_time_us":190345}
I20260709 13:03:01.915886  1553 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:01.923915  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:01.925084  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 249553us
I20260709 13:03:01.758102  9249 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.964097  2350 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:01.966912  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:01.933881  1554 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:03:01.839260  1629 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
W20260709 13:03:01.864092  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.850488  1561 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:03:01.805634  9226 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.965276  2346 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:03:01.765061  9243 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:01.968225  1560 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:01.970359  1627 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:01.977703  1555 raft_consensus.cc:2468] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:03:01.982712  1632 raft_consensus.cc:2410] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 13:03:01.984637  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 113638us
I20260709 13:03:01.983615  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:01.805779  9315 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:01.990849  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.925602  9280 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.018380  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:02.016050  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 184512us
I20260709 13:03:01.932325  9230 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:02.027734  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:01.987807  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:01.986989  9282 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.028575  1552 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:03:02.021991  9303 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:01.928580  9127 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.986567  9225 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.928810  9260 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:02.022917  9296 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.024928  9219 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.020260  9289 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:02.090874  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:01.954702  9214 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:01.805310  9099 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:01.936604  9290 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:02.037096  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:02.028537  9331 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:02.102514  1561 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:03:02.041597  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:02.104082  2351 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:03:02.105571  2353 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:01.965263  9314 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.014645  9322 raft_consensus.cc:697] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:02.036720  9280 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:02.027722  9245 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:02.044155  1636 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:03:01.865466  9288 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.134373  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3363) took 1239 ms. Trace:
I20260709 13:03:02.054270  9225 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.055132  2346 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:03:02.026301  9266 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:03:02.137398  1558 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:02.058053  9328 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.066385  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1388) took 1125 ms. Trace:
I20260709 13:03:02.143452  1629 rpcz_store.cc:276] 0709 13:03:00.940648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.143503 (+202855us) service_pool.cc:224] Handling call
0709 13:03:02.066137 (+922634us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b54001b6054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:02.066286 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":884419840}
I20260709 13:03:02.067656  9265 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.014776  9158 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
W20260709 13:03:02.073076  1633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1384) took 1960 ms (client timeout 1893 ms). Trace:
I20260709 13:03:02.040565  8387 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:02.106554  9252 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.076638  8809 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.078840  1559 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 13:03:02.156965  9040 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.007s	sys 0.022s
W20260709 13:03:02.035490  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51971.5us
I20260709 13:03:01.933684  9298 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:02.041168  9303 consensus_queue.cc:1243] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:01.933326  9233 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.016959  9226 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:02.225390  9040 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.008s	sys 0.023s
I20260709 13:03:02.108861  2344 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:02.131723  1702 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:02.042742  1554 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:03:02.093127  9289 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.134967  1701 rpcz_store.cc:276] 0709 13:03:00.894680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.895045 (+   365us) service_pool.cc:224] Handling call
0709 13:03:00.930722 (+ 35677us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:01.087046 (+156324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.087089 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:01.177141 (+ 90052us) log.cc:844] Serialized 52 byte log entry
0709 13:03:01.834436 (+657295us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:01.834477 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.834550 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.134113 (+299563us) raft_consensus.cc:1685] finished
0709 13:03:02.134139 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.134308 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8082,"raft.queue_time_us":44086,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":22722,"thread_start_us":618148,"threads_started":2,"wal-append.queue_time_us":795972}
I20260709 13:03:02.019095  9247 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.266736  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.267529  1554 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
W20260709 13:03:02.268297  1527 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.003s	sys 0.024s
W20260709 13:03:02.268770  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 514411us
W20260709 13:03:02.137960  1707 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.183s	user 0.014s	sys 0.027s
I20260709 13:03:02.098503  9282 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.155124  1627 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:02.275417  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.276749  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:02.170357  1633 rpcz_store.cc:269] 0709 13:03:00.112278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.346201 (+233923us) service_pool.cc:224] Handling call
0709 13:03:02.072819 (+1726618us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400343854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:02.072976 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3327726336}
I20260709 13:03:02.045156  9230 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:02.183225  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:02.277969  2351 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:03:02.278134  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.279264  1633 raft_consensus.cc:2410] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:02.139742  9168 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.081086  9220 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.192397  9099 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.196161  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.198050  1453 catalog_manager.cc:5697] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 6 to 18. New cstate: current_term: 18 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:02.156101  9266 consensus_queue.cc:1243] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:02.199113  8664 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.199877  9127 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.201288  9158 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:02.170661  9225 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.201771  9215 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.202375  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:02.283569  9299 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.214680  9219 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.214200  8992 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.140189  9314 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.111563  9331 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.217726  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1346) took 1705 ms. Trace:
I20260709 13:03:02.221619  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:02.344852  1636 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:02.221805  9280 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:02.146287  9288 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.222007  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:02.224308  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:02.224835  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:02.229873  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.230918  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.234402  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:02.236023  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.237205  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:02.183825  8387 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.229274  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.244688  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:02.107801  9290 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.257000  1710 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
W20260709 13:03:02.107303  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 71172.7us
W20260709 13:03:02.272472  1707 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.321s	user 0.014s	sys 0.027s
I20260709 13:03:02.109266  9326 consensus_queue.cc:260] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.185956  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.281025  1627 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:02.243330  9298 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:02.303504  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.189891  9252 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.306896  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.267441  9289 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:02.303014  9168 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.315116  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:02.316398  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.322005  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:02.396095  2351 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:03:02.327775  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:02.329785  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:02.330715  1557 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
W20260709 13:03:02.341850  1703 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.852s	user 0.014s	sys 0.035s
W20260709 13:03:02.397680  1703 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.015s	sys 0.035s
I20260709 13:03:02.343106  1675 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:02.260907  9233 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.331872  9215 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.274367  9247 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.403218  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:02.345767  9249 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.344651  2350 rpcz_store.cc:276] 0709 13:03:00.512481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.512663 (+   182us) service_pool.cc:224] Handling call
0709 13:03:00.697639 (+184976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.965514 (+1267875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.216641 (+251127us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54006310d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:02.217521 (+   880us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54006310d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:02.217664 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3302679552}
W20260709 13:03:02.411453  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 81741us
I20260709 13:03:02.412199  9288 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:02.412647  9288 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:02.413055  9288 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:02.419139  9331 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.421103  9336 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:02.421587  9336 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:02.421975  9336 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:02.426357  9226 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.389657  2352 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:02.390863  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:02.327816  9015 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:02.428277  8992 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:02.428612  9015 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.345981  9198 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:02.430512  8664 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.431313  9099 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:02.396193  1705 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 15.
I20260709 13:03:02.203754  9322 consensus_queue.cc:237] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.433846  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:02.434480  2351 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:03:02.434715  8878 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.431380  9015 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.394840  9298 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:02.400225  1633 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:02.332010  9225 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.383736  8387 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:02.410887  1556 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:02.466004  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.466352  1632 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:02.426496  9290 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.379395  9095 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:02.393893  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.403558  9315 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.411055  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:02.386811  1552 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:02.439874  1636 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:02.474829  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.477253  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:02.475292  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.478544  1636 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 22: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:02.478605  1633 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:02.403182  9326 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:02.435586  9197 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:02.441885  2353 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1713) took 2269 ms (client timeout 1993 ms). Trace:
I20260709 13:03:02.446842  1557 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:03:02.447357  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.485369  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.334419  9158 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.450093  2349 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:02.451154  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:02.484400  2353 rpcz_store.cc:269] 0709 13:03:00.172154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.172275 (+   121us) service_pool.cc:224] Handling call
0709 13:03:02.441679 (+2269404us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54007bff54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:02.441797 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4347938560}
I20260709 13:03:02.440267  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:02.347776  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3627) took 1438 ms. Trace:
I20260709 13:03:02.462558  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.345981  9299 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.398870  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:03:02.464174  9318 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.016s	sys 0.010s
I20260709 13:03:02.378953  9280 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.432013  9099 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:02.480376  1559 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:03:02.474227  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 483359us
I20260709 13:03:02.481105  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.510360  1636 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:03:02.426324  9288 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.486038  1633 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:03:02.486763  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 75752us
I20260709 13:03:02.490065  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.492992  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.523895  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.524880  1552 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 21: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:02.497182  1629 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
W20260709 13:03:02.473940  9321 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.004s	sys 0.024s
I20260709 13:03:02.494256  2346 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:03:02.500190  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.401209  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:02.502085  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 89933.1us
W20260709 13:03:02.480840  9115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.005s	sys 0.022s
I20260709 13:03:02.495710  1631 rpcz_store.cc:276] 0709 13:03:00.909252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.922750 (+ 13498us) service_pool.cc:224] Handling call
0709 13:03:00.923157 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.974374 (+ 51217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.974412 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:00.974926 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.325568 (+350642us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:02.344975 (+1019407us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.345807 (+   832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.346661 (+   854us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.346720 (+    59us) raft_consensus.cc:1685] finished
0709 13:03:02.346731 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.346859 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":504440,"prepare.queue_time_us":593345,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":734,"raft.queue_time_us":1916,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8836,"thread_start_us":780633,"threads_started":2,"wal-append.queue_time_us":430400}
I20260709 13:03:02.401891  9230 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.494117  8989 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:02.515913  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.516410  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:02.518640  1627 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
W20260709 13:03:02.502180  9318 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.016s	sys 0.010s
I20260709 13:03:02.509896  9314 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:02.507939  8664 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.527415  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:02.542825  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 141594us
W20260709 13:03:02.530509  2317 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:37097 timed out after 1.893s (SENT) [suppressed 11 similar messages]
I20260709 13:03:02.617698  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.538646  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:02.618479  1629 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 15.
I20260709 13:03:02.618762  1633 raft_consensus.cc:2393] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:02.544088  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:02.537318  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:03:02.545713  9321 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.004s	sys 0.024s
I20260709 13:03:02.569311  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.534209  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:02.614095  1636 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:02.613432  1559 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 25.
W20260709 13:03:02.528262  2348 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.004s	sys 0.026s
I20260709 13:03:02.621454  1631 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: []
W20260709 13:03:02.624783  2348 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.004s	sys 0.026s
I20260709 13:03:02.625661  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1355) took 1597 ms. Trace:
I20260709 13:03:02.625922  1631 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:02.622617  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.622815  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.627882  1552 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
W20260709 13:03:02.628118  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 100773us
I20260709 13:03:02.628712  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.628811  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:02.493409  9095 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.510058  9105 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
W20260709 13:03:02.567922  9115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.005s	sys 0.022s
I20260709 13:03:02.526060  8992 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.623503  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 79463.7us
W20260709 13:03:02.630373  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 96211.5us
I20260709 13:03:02.631762  1633 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:02.567386  9249 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:02.566721  9265 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.632751  9332 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:02.632947  9330 consensus_queue.cc:1059] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:02.624400  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.638792  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.633169  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:02.538146  9197 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:02.537650  9290 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:02.617441  9329 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:02.624508  9314 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.625991  2348 rpcz_store.cc:276] 0709 13:03:01.028298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.028463 (+   165us) service_pool.cc:224] Handling call
0709 13:03:01.110585 (+ 82122us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:01.111135 (+   550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.111169 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:01.186284 (+ 75115us) log.cc:844] Serialized 52 byte log entry
0709 13:03:01.868048 (+681764us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:02.625172 (+757124us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.625367 (+   195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.625440 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.625495 (+    55us) raft_consensus.cc:1685] finished
0709 13:03:02.625505 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.625612 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60311,"raft.queue_time_us":2944,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":23850,"thread_start_us":1214557,"threads_started":2,"wal-append.queue_time_us":864286}
I20260709 13:03:02.630143  9219 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:03:02.688752  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 55621.3us
I20260709 13:03:02.706236  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:02.653407  1636 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:03:02.637147  9319 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:02.689019  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:02.645247  9266 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:02.690186  9265 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.735440  9249 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:02.695832  9197 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.696448  9099 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.697768  9331 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:02.681610  9243 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.735446  1629 raft_consensus.cc:1240] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:02.695206  9322 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:02.699990  9290 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.738672  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1266) took 1063 ms. Trace:
I20260709 13:03:02.787379  1708 rpcz_store.cc:276] 0709 13:03:01.674822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.794383 (+119561us) service_pool.cc:224] Handling call
0709 13:03:01.805252 (+ 10869us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.805717 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.805766 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.806392 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.306653 (+500261us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:02.306712 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.306803 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.738486 (+431683us) raft_consensus.cc:1685] finished
0709 13:03:02.738504 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.738623 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3981,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":160,"thread_start_us":499745,"threads_started":1,"wal-append.queue_time_us":866874}
I20260709 13:03:02.730777  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:02.739523  2344 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:02.739264  9219 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.613562  9315 consensus_queue.cc:1059] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.118s
I20260709 13:03:02.757684  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:02.763350  9351 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.769899  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1391) took 1341 ms. Trace:
I20260709 13:03:02.796473  9243 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:02.805389  9243 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.803978  1634 rpcz_store.cc:276] 0709 13:03:01.428100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.726133 (+298033us) service_pool.cc:224] Handling call
0709 13:03:01.726508 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.758565 (+ 32057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.758720 (+   155us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.767724 (+  9004us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.381519 (+613795us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:02.403554 (+ 22035us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.409460 (+  5906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.409625 (+   165us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.769595 (+359970us) raft_consensus.cc:1685] finished
0709 13:03:02.769625 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.769818 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70071,"prepare.queue_time_us":21965,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":45904,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":189,"thread_start_us":450068,"threads_started":1,"wal-append.queue_time_us":857569}
I20260709 13:03:02.805882  9243 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:02.807173  1634 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:02.808619  1634 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:02.770054  9215 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.827885  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:02.828868  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:02.829519  2345 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:02.832617  1632 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:03:02.637845  8664 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.835665  8664 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.636869  9183 consensus_queue.cc:1059] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.117s
W20260709 13:03:02.776311  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 554040us
I20260709 13:03:02.836941  9183 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:02.837291  9183 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:02.838207  9183 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.839671  9183 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:02.734184  9314 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:02.840858  9095 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.841619  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1387) took 1916 ms. Trace:
I20260709 13:03:02.842084  1628 rpcz_store.cc:276] 0709 13:03:00.925732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.331249 (+405517us) service_pool.cc:224] Handling call
0709 13:03:01.331769 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.016764 (+684995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.016802 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.029620 (+ 12818us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.568405 (+538785us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:02.568449 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.568526 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.841362 (+272836us) raft_consensus.cc:1685] finished
0709 13:03:02.841397 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.841558 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111512,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":129,"thread_start_us":1150318,"threads_started":2,"wal-append.queue_time_us":710012}
I20260709 13:03:02.843194  9280 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.803112  9298 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.844977  9105 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.790897  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 60201.5us
I20260709 13:03:02.772110  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:02.845934  1601 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:02.798148  1557 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:02.772065  9319 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:02.684219  8992 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.813005  9321 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:02.813932  9168 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.793262  9247 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.817893  9197 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:02.818657  9343 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
I20260709 13:03:02.868680  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:03:02.819348  9342 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
I20260709 13:03:02.819214  9158 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:02.820057  9315 raft_consensus.cc:3055] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:02.818650  9351 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.826128  9299 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.833446  2344 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
W20260709 13:03:02.798106  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 454801us
I20260709 13:03:02.858901  9249 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:02.857467  8387 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:02.878976  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:03:02.863368  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 234486us
I20260709 13:03:02.789178  9322 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:02.842181  9288 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.888305  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.872468  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3373) took 1385 ms. Trace:
I20260709 13:03:02.876513  1675 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:02.891814  1675 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:02.893924  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:02.883738  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.014s	sys 0.013s
I20260709 13:03:02.888077  8387 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:02.846295  2346 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
W20260709 13:03:02.895670  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.014s	sys 0.013s
I20260709 13:03:02.889505  1703 rpcz_store.cc:276] 0709 13:03:01.487643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.487905 (+   262us) service_pool.cc:224] Handling call
0709 13:03:01.488429 (+   524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.488795 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:02.398143 (+909348us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.871642 (+473499us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.871906 (+   264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.871962 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:02.872031 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.872204 (+   173us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.872371 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":204375,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":160,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":3995,"spinlock_wait_cycles":13867264,"thread_start_us":1381531,"threads_started":2}
I20260709 13:03:02.849751  9105 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:02.873775  9298 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:02.902151  1675 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:02.902966  8387 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:02.879498  9249 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.873231  9315 raft_consensus.cc:740] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:02.879956  9303 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:02.910122  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:02.913964  8387 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:02.916013  8387 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.916554  8387 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:02.916633  9298 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:02.917486  9298 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:02.917833  9298 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:02.898449  9168 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:02.952153  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1528) took 1449 ms. Trace:
I20260709 13:03:02.953212  2347 rpcz_store.cc:276] 0709 13:03:01.502417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.805125 (+302708us) service_pool.cc:224] Handling call
0709 13:03:01.817944 (+ 12819us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.818368 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.818400 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.818992 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.317041 (+498049us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:02.317098 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.317187 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.951845 (+634658us) raft_consensus.cc:1685] finished
0709 13:03:02.951875 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.952071 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8001,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"thread_start_us":497610,"threads_started":1,"wal-append.queue_time_us":680214}
I20260709 13:03:02.883350  9243 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:02.954038  1707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.010s	sys 0.012s
I20260709 13:03:02.912569  9249 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:02.956970  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 426222us
W20260709 13:03:02.956689  1707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.010s	sys 0.012s
I20260709 13:03:02.936307  8387 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:02.966616  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 282198us
W20260709 13:03:02.971149  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 69035.5us
I20260709 13:03:02.904891  9234 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:03:02.952968  9294 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:02.973856  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:02.976343  2317 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:02.994338  2317 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:02.952507  9168 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:03.001847  2346 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:03.042964  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.043056  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.044389  1705 raft_consensus.cc:2410] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:03:03.044517  1709 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:03.046689  8387 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:03.049391  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:03.050156  2346 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEARNER]: Advancing to term 21
W20260709 13:03:03.046475  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 206471us
I20260709 13:03:02.919252  9266 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
W20260709 13:03:03.047708  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 48841.7us
I20260709 13:03:03.062248  1703 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:03.065919  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:02.992970  9324 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:03.066992  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.068171  1710 raft_consensus.cc:2410] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:03:03.071760  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.072433  1709 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 25.
I20260709 13:03:03.018997  9234 consensus_queue.cc:1243] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer d62ce354d09a4be59c29094425d09bc1 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:03:03.019624  8664 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:03:02.990566  9158 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:03.082955  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.083592  1702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52646 (request call id 1265) took 1409 ms. Trace:
I20260709 13:03:03.074896  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.082895  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1724) took 1660 ms. Trace:
I20260709 13:03:03.086127  1702 rpcz_store.cc:276] 0709 13:03:01.674547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.794361 (+119814us) service_pool.cc:224] Handling call
0709 13:03:01.794715 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.261208 (+466493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.083367 (+822159us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b5400077954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:03.083534 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725400448}
I20260709 13:03:02.997866  9321 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.042332  9168 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:03.085412  1705 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:03:03.041356  9350 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:02.957002  9322 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:03.092934  8387 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:03.050822  8809 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:03.086321  2352 rpcz_store.cc:276] 0709 13:03:01.422973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.424103 (+  1130us) service_pool.cc:224] Handling call
0709 13:03:01.429180 (+  5077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.427337 (+998157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.082676 (+655339us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b54004ecad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:03.082839 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2207097600}
I20260709 13:03:03.092105  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:03.094645  9289 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.681s	user 0.009s	sys 0.002s
W20260709 13:03:03.105798  9363 log.cc:927] Time spent T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:03:03.150527  1674 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:03.106884  9356 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 13:03:03.189342  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61286.2us
I20260709 13:03:03.100139  9105 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:03.115883  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:03.190727  2352 raft_consensus.cc:2410] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:03:02.990995  9315 consensus_queue.cc:260] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:03.133775  9329 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:03.138424  9233 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:03.140805  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.204913  1705 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:03:02.993157  9226 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.142026  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1362) took 1303 ms. Trace:
I20260709 13:03:03.206250  2350 rpcz_store.cc:276] 0709 13:03:01.839271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.450113 (+610842us) service_pool.cc:224] Handling call
0709 13:03:02.467880 (+ 17767us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.468274 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.468308 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.474496 (+  6188us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.860657 (+386161us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:02.860704 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.860773 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.141813 (+281040us) raft_consensus.cc:1685] finished
0709 13:03:03.141833 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.141969 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1211,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":14410,"thread_start_us":349764,"threads_started":1,"wal-append.queue_time_us":542174}
I20260709 13:03:03.142585  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.142840  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.143831  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.144243  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.145509  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:03.075174  9368 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:03.147899  9168 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:03.148270  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.226812  1706 raft_consensus.cc:2393] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:03.149659  8992 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:02.919826  9319 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:03.152839  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:03.074527  9099 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:03.161046  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3333) took 1005 ms. Trace:
I20260709 13:03:03.193020  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3641) took 1213 ms. Trace:
W20260709 13:03:03.220239  9371 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:03:03.202183  9289 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:03.228889  9336 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.806s	user 0.013s	sys 0.000s
W20260709 13:03:03.222312  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 246082us
I20260709 13:03:03.222256  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:03.275207  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:03.136592  9260 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:03.094139  9321 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:03.265420  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:03.150208  9243 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 26 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:03.204198  9359 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:03:03.270594  1627 rpcz_store.cc:276] 0709 13:03:02.155765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.450276 (+294511us) service_pool.cc:224] Handling call
0709 13:03:03.139731 (+689455us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b5400139cd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:03.160966 (+ 21235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93873408}
W20260709 13:03:03.259207  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 55684.1us
I20260709 13:03:03.270717  1634 rpcz_store.cc:276] 0709 13:03:01.979594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.811863 (+832269us) service_pool.cc:224] Handling call
0709 13:03:02.812164 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.812395 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:03:02.863471 (+ 51076us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.192411 (+328940us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:03.192486 (+    75us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:03.192547 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.192640 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.192942 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39301,"prepare.queue_time_us":51315,"prepare.run_cpu_time_us":165,"prepare.run_wall_time_us":1646,"raft.queue_time_us":230,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":3612,"thread_start_us":328351,"threads_started":1}
I20260709 13:03:03.271375  1703 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:03.210036  1704 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:03:03.221824  9230 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:03.221998  9329 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:03.311004  1561 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:03.202823  9266 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.275279  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:03.314980  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:03.259975  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:03.317067  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3639) took 1338 ms (client timeout 1605 ms). Trace:
W20260709 13:03:03.317637  1632 rpcz_store.cc:269] 0709 13:03:01.979085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.039545 (+ 60460us) service_pool.cc:224] Handling call
0709 13:03:03.316854 (+1277309us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54003cd7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:03.316999 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860062592}
I20260709 13:03:03.319255  1708 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:03.320886  1633 raft_consensus.cc:1240] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: [15.8-18.10]
I20260709 13:03:03.278548  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:03.344241  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.225487  9158 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:03:03.301921  2350 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:03.282871  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3310) took 2369 ms. Trace:
I20260709 13:03:03.310104  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:03.310910  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 101767us
I20260709 13:03:03.312677  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.273762  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.350509  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:03.301074  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:03.227912  9226 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:03.362674  1630 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:03.362762  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:03.364682  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:03.364769  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.369055  1704 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 25.
I20260709 13:03:03.371381  1704 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:03:03.359144  1635 rpcz_store.cc:276] 0709 13:03:00.907261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.914904 (+  7643us) service_pool.cc:224] Handling call
0709 13:03:00.915230 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:00.983534 (+ 68304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.983569 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:00.984362 (+   793us) log.cc:844] Serialized 52 byte log entry
0709 13:03:02.070498 (+1086136us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b180010db00. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:03:02.502719 (+432221us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:02.502763 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.276671 (+773908us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b540010f9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:03.276728 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.276797 (+    69us) raft_consensus.cc:1685] finished
0709 13:03:03.276809 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.276926 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60703,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":11514,"spinlock_wait_cycles":84812928,"thread_start_us":432009,"threads_started":1,"wal-append.queue_time_us":1743481}
I20260709 13:03:03.372705  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:03.372980  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3364) took 2452 ms. Trace:
I20260709 13:03:03.373880  1632 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:03:03.298768  9197 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.304322  9370 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.394408  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:03.274562  9099 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:03.403096  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:03.283481  9233 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:03.412850  1602 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:03.426249  1635 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:03:03.282505  9315 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:03.304319  9230 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:03.375608  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 100353us
I20260709 13:03:03.437562  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:03.361277  9260 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:03.436692  9243 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:03.421031  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:03.441663  9243 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.442112  9243 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:03.442584  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:03.443732  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:03.422143  2353 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:03.373340  1707 rpcz_store.cc:276] 0709 13:03:00.904613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.904840 (+   227us) service_pool.cc:224] Handling call
0709 13:03:00.937711 (+ 32871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.393282 (+1455571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.393314 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.423402 (+ 30088us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.974844 (+551442us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:02.974904 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.975509 (+   605us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.356561 (+381052us) raft_consensus.cc:1685] finished
0709 13:03:03.356592 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.356778 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14604,"raft.queue_time_us":48719,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":25738,"thread_start_us":1984823,"threads_started":2,"wal-append.queue_time_us":796444}
I20260709 13:03:03.427281  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:03.445864  1635 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
W20260709 13:03:03.430292  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.427441  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:03.447263  1602 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:03.447556  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:03.448196  1602 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:03.448575  2350 raft_consensus.cc:2376] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22. Re-sending same reply.
I20260709 13:03:03.449275  9230 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:03.449728  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:03.451215  9280 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:03.451687  9280 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.452071  9280 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:03.375878  9371 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:03:03.358119  8809 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:03:03.411345  9359 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:03:03.285712  9336 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:03.470919  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:03.471622  1634 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:03:03.482189  1630 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22.
I20260709 13:03:03.483528  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:03.438994  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:03.434382  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 129220us
I20260709 13:03:03.427872  9350 raft_consensus.cc:697] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:03.440400  9247 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.440322  9233 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:03.440768  1675 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:03.464082  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:03:03.416313  9314 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.002s
I20260709 13:03:03.486312  1632 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:03:03.365289  9266 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:03.475446  8809 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:03.447747  9198 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:03.491067  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:03.501874  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3323) took 1585 ms. Trace:
I20260709 13:03:03.502322  1636 rpcz_store.cc:276] 0709 13:03:01.916547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.709414 (+792867us) service_pool.cc:224] Handling call
0709 13:03:02.709894 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.739866 (+ 29972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.739899 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.740389 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.959344 (+218955us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:02.994033 (+ 34689us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.994304 (+   271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.994407 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.501631 (+507224us) raft_consensus.cc:1685] finished
0709 13:03:03.501654 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.501801 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29515,"prepare.queue_time_us":59356,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":34409,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":43290,"thread_start_us":218573,"threads_started":1,"wal-append.queue_time_us":368097}
I20260709 13:03:03.435571  9370 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:03.496026  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:03.498080  1709 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:03:03.412050  9197 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:03.487248  9326 consensus_meta.cc:287] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.994s	user 0.008s	sys 0.004s
I20260709 13:03:03.507817  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3642) took 1372 ms. Trace:
I20260709 13:03:03.508101  2346 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEARNER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:03:03.509199  9373 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:03:03.510123  9350 consensus_queue.cc:237] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:03.562634  1632 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
W20260709 13:03:03.871070  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.947379  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3331) took 1829 ms. Trace:
I20260709 13:03:03.947804  1632 rpcz_store.cc:276] 0709 13:03:02.117941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.501115 (+1383174us) service_pool.cc:224] Handling call
0709 13:03:03.562466 (+ 61351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.947115 (+384649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.947325 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:03.948452  1634 raft_consensus.cc:2410] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:03.475253  9282 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:03.579223  2352 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:03.652245  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:03.658357  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 194208us
W20260709 13:03:03.570611  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 122400us
I20260709 13:03:03.942471  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:03.946013  1636 rpcz_store.cc:276] 0709 13:03:02.135867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.504720 (+1368853us) service_pool.cc:224] Handling call
0709 13:03:03.505086 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.505259 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.505285 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:03.505331 (+    46us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:03.505365 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.505420 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.507729 (+  2309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:03.951031  1602 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:03.953635  1676 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:03:04.101615  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 150543us
W20260709 13:03:04.102035  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 148387us
I20260709 13:03:04.101217  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:04.104349  2346 raft_consensus.cc:2410] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:04.106376  1560 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
W20260709 13:03:03.623857  9376 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:03:03.495299  9247 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:04.138309  2350 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.139597  9247 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:04.140172  9247 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:04.140390  9372 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.605s	user 0.000s	sys 0.000s
I20260709 13:03:04.140537  9247 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.140899  9247 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:03.511349  9198 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:04.141973  9198 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.100993  9282 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.144362  9282 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:04.145341  9197 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:03.578419  9388 consensus_queue.cc:1059] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.615s
I20260709 13:03:03.622260  9383 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:04.146967  9336 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:04.146840  9383 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:03.500947  9314 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:04.151099  9315 consensus_meta.cc:287] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.714s	user 0.009s	sys 0.000s
W20260709 13:03:03.775168  9375 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 13:03:04.152802  1559 raft_consensus.cc:1240] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:03:04.152053  9233 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:04.152057  9314 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:04.130445  9386 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:04.156147  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1394) took 2173 ms. Trace:
I20260709 13:03:03.625528  9379 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.572s
I20260709 13:03:04.211218  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2673) took 1069 ms. Trace:
I20260709 13:03:04.176977  2350 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:03:04.158905  1634 raft_consensus.cc:1217] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Deduplicated request from leader. Original: 12.11->[13.12-18.15]   Dedup: 13.12->[15.13-18.15]
I20260709 13:03:04.239631  1634 pending_rounds.cc:85] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 12
I20260709 13:03:04.240075  1634 pending_rounds.cc:107] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:03:04.240391  1634 raft_consensus.cc:2889] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:04.156535  1628 rpcz_store.cc:276] 0709 13:03:01.983294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.843544 (+860250us) service_pool.cc:224] Handling call
0709 13:03:02.843877 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.844200 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.844226 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.844673 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:03:03.053034 (+208361us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:03.053077 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.053143 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.155924 (+1102781us) raft_consensus.cc:1685] finished
0709 13:03:04.155957 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.156088 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2718,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":4929,"thread_start_us":207990,"threads_started":1,"wal-append.queue_time_us":470623}
I20260709 13:03:04.198252  1627 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:03.500448  9380 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:04.180992  9198 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:03.774319  9377 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.340s	user 0.000s	sys 0.000s
W20260709 13:03:04.246928  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.014s	sys 0.080s
W20260709 13:03:04.250412  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.014s	sys 0.080s
W20260709 13:03:04.251410  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 810171us
I20260709 13:03:04.238560  1709 rpcz_store.cc:276] 0709 13:03:03.141775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.142585 (+   810us) service_pool.cc:224] Handling call
0709 13:03:04.210950 (+1068365us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54001865d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:04.211106 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662759808}
W20260709 13:03:04.130205  9243 consensus_meta.cc:287] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.688s	user 0.008s	sys 0.004s
I20260709 13:03:03.706246  9168 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:04.149312  9383 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:04.258630  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:04.260488  9288 raft_consensus.cc:3055] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:03:04.252104  9249 consensus_meta.cc:287] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.276s	user 0.004s	sys 0.008s
I20260709 13:03:04.158226  9387 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:04.157419  9230 consensus_meta.cc:287] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.708s	user 0.007s	sys 0.000s
I20260709 13:03:04.186664  8809 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:04.190550  9266 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:03.731498  9385 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:03:04.133208  9183 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.268s	user 0.006s	sys 0.009s
W20260709 13:03:04.181365  9280 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.729s	user 0.007s	sys 0.000s
I20260709 13:03:03.624106  9296 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:03.511093  9252 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:04.261983  1554 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 18. (term mismatch)
I20260709 13:03:04.503644  1554 pending_rounds.cc:85] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Aborting all ops after (but not including) 17
I20260709 13:03:04.504076  1554 pending_rounds.cc:107] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Aborting uncommitted NO_OP operation due to leader change: 20.18
I20260709 13:03:04.504447  1554 raft_consensus.cc:2889] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:04.268399  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3517) took 1992 ms. Trace:
I20260709 13:03:04.505990  1555 rpcz_store.cc:276] 0709 13:03:02.275844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.445503 (+169659us) service_pool.cc:224] Handling call
0709 13:03:02.445808 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.690318 (+244510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.690358 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.690424 (+    66us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:02.690471 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.268113 (+1577642us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5400208354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:04.268176 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.268331 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":561412864}
I20260709 13:03:04.270462  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:03:04.508334  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 375359us
I20260709 13:03:04.508737  1558 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:04.298650  1632 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:04.310738  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:04.261448  9351 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:04.445648  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 342998us
I20260709 13:03:04.314563  9385 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:04.180107  9282 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:04.336979  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.518069  1673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.987s	user 0.006s	sys 0.003s
W20260709 13:03:04.518479  1673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.988s	user 0.007s	sys 0.003s
I20260709 13:03:04.269903  9324 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:04.269398  9265 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:04.287523  9386 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:04.164384  9247 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:04.306213  9387 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.305941  9249 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:04.294765  9288 raft_consensus.cc:740] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:04.264742  9219 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:04.300515  9321 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:04.491300  1701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.000s	sys 0.031s
W20260709 13:03:04.593086  1701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.058s	user 0.000s	sys 0.032s
I20260709 13:03:04.513082  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:04.595208  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 2663) took 1573 ms. Trace:
I20260709 13:03:04.595912  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3324) took 2674 ms. Trace:
I20260709 13:03:04.594695  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.595702  1701 rpcz_store.cc:276] 0709 13:03:03.021935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.047289 (+ 25354us) service_pool.cc:224] Handling call
0709 13:03:03.073991 (+ 26702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.074211 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:03.312622 (+238411us) log.cc:844] Serialized 14 byte log entry
0709 13:03:04.594785 (+1282163us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.594852 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:04.594918 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.594995 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.595143 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":379255,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":623013,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":1500767,"threads_started":3,"wal-append.queue_time_us":1204211}
I20260709 13:03:04.516729  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:04.598925  1554 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:03.723984  1453 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:04.531440  9324 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:04.522480  9351 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:04.255275  9380 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:04.564218  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.554674  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:04.613507  1702 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:04.519270  9385 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:04.577858  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:04.620389  1556 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:04.620709  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.317034  9230 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:04.625502  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.363780  9280 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:04.629541  1701 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:04.629796  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:04.588438  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.630982  1628 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 25.
I20260709 13:03:04.631760  1709 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:03:04.533336  9260 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.092s	user 0.013s	sys 0.000s
W20260709 13:03:04.518958  2318 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.005s	sys 0.004s
W20260709 13:03:04.635847  2318 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.006s	sys 0.004s
W20260709 13:03:04.636286  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 882626us
W20260709 13:03:04.593945  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 80926.9us
I20260709 13:03:04.596500  1629 rpcz_store.cc:276] 0709 13:03:01.921787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.784750 (+862963us) service_pool.cc:224] Handling call
0709 13:03:02.798398 (+ 13648us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:02.798765 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.798804 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:02.810407 (+ 11603us) log.cc:844] Serialized 52 byte log entry
0709 13:03:03.152312 (+341905us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:03.152350 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.152415 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.595675 (+1443260us) raft_consensus.cc:1685] finished
0709 13:03:04.595701 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.595850 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3293,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":19758,"spinlock_wait_cycles":48512,"thread_start_us":275490,"threads_started":1,"wal-append.queue_time_us":416021}
I20260709 13:03:04.602015  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:04.602631  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.277421  9243 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:04.609655  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.577551  9387 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.651711  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:04.652456  1628 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22.
I20260709 13:03:04.567687  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:04.653746  1555 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:04.615443  2351 raft_consensus.cc:1240] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:04.624348  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:03:04.579304  1674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.089s	user 0.004s	sys 0.013s
W20260709 13:03:04.654603  1674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.004s	sys 0.013s
W20260709 13:03:04.654995  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 1.16375e+06us
I20260709 13:03:04.638413  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.639940  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:04.641048  1704 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
W20260709 13:03:04.644693  9391 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:03:04.645232  9386 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:04.645555  9183 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:04.605381  9158 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.646090  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:04.647040  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.647234  9380 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:04.646122  9282 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:04.684128  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44295.9us
I20260709 13:03:04.611300  9219 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:04.601763  9392 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:04.688915  1554 raft_consensus.cc:2468] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:03:04.690074  1703 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:03:04.747704  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.897014  2317 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:04.897136  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2676) took 2407 ms (client timeout 1743 ms). Trace:
W20260709 13:03:04.897667  1560 rpcz_store.cc:269] 0709 13:03:02.491106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.541371 (+ 50265us) service_pool.cc:224] Handling call
0709 13:03:04.896922 (+2355551us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400207454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:04.897037 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3399240064}
W20260709 13:03:04.898195  2350 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1739) took 1816 ms (client timeout 2102 ms). Trace:
I20260709 13:03:04.834101  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1729) took 3403 ms. Trace:
I20260709 13:03:04.716984  2347 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:03:04.898670  2350 rpcz_store.cc:269] 0709 13:03:03.081944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.449389 (+367445us) service_pool.cc:224] Handling call
0709 13:03:04.898009 (+1448620us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54007701d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:04.898134 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1581055360}
I20260709 13:03:04.717752  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.718853  1630 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:04.721732  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:04.900014  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:04.727582  1702 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 26.
I20260709 13:03:04.732981  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:04.836922  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.828897  1634 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.001s	sys 0.009s
W20260709 13:03:04.903656  2344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1369) took 2077 ms (client timeout 2383 ms). Trace:
I20260709 13:03:04.905722  1706 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:03:04.734822  1527 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:04.736013  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 80307.6us
I20260709 13:03:04.907055  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:04.624006  9351 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:04.904264  2344 rpcz_store.cc:269] 0709 13:03:02.826772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.827594 (+   822us) service_pool.cc:224] Handling call
0709 13:03:04.903482 (+2075888us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b540049b454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:04.903597 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11011840}
I20260709 13:03:04.739194  1556 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 26.
I20260709 13:03:04.721432  9260 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:04.720624  9266 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:04.722414  9405 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:03:04.922636  2353 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1745) took 1775 ms (client timeout 1625 ms). Trace:
I20260709 13:03:04.694990  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:04.896661  9298 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:04.898041  2317 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:04.715631  1558 raft_consensus.cc:2468] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:04.926491  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:04.927196  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:04.927791  1558 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:03:04.900657  2350 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.649305  9243 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:04.902685  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:04.904121  1629 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
W20260709 13:03:04.801954  9393 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.419s	user 0.000s	sys 0.001s
I20260709 13:03:04.735939  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:04.926079  9266 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:04.934662  9386 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:04.935143  9386 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:04.793694  9397 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:04.935603  9386 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:04.935837  9397 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:04.936352  9266 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:04.937922  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2686) took 2086 ms. Trace:
I20260709 13:03:04.938333  1559 rpcz_store.cc:276] 0709 13:03:02.851362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.889427 (+ 38065us) service_pool.cc:224] Handling call
0709 13:03:02.909101 (+ 19674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:04.153193 (+1244092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.937619 (+784426us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003205d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:04.937839 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2857894272}
I20260709 13:03:04.938825  9351 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:04.932394  1554 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:04.940141  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:04.941630  1558 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:04.939778  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:04.902684  1634 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.001s	sys 0.015s
I20260709 13:03:04.912737  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:04.843071  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3644) took 2640 ms. Trace:
I20260709 13:03:04.898886  2349 rpcz_store.cc:276] 0709 13:03:01.430678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.784592 (+353914us) service_pool.cc:224] Handling call
0709 13:03:02.441838 (+657246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.493779 (+ 51941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.190669 (+696890us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b54007bff54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:04.833784 (+1643115us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b54007bff50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:04.833981 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1526988800}
I20260709 13:03:04.781718  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:04.913993  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:04.918845  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:04.958925  1553 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:03:04.920680  1557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47122 (request call id 2682) took 2422 ms (client timeout 1727 ms). Trace:
I20260709 13:03:04.921257  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:04.959824  1632 rpcz_store.cc:276] 0709 13:03:02.202396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.103097 (+1900701us) service_pool.cc:224] Handling call
0709 13:03:04.141850 (+ 38753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.509762 (+367912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.842677 (+332915us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400010ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:04.842983 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359831808}
I20260709 13:03:04.729158  8809 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:03:04.923209  2353 rpcz_store.cc:269] 0709 13:03:03.146987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.314541 (+167554us) service_pool.cc:224] Handling call
0709 13:03:04.922421 (+1607880us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54006446d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:04.922551 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244110080}
I20260709 13:03:04.717147  9288 consensus_queue.cc:260] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:04.918993  1674 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:04.921464  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1367) took 2180 ms. Trace:
I20260709 13:03:04.981457  2348 rpcz_store.cc:276] 0709 13:03:02.740949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.741147 (+   198us) service_pool.cc:224] Handling call
0709 13:03:02.741568 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.796321 (+ 54753us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:03:02.796604 (+   283us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.796812 (+   208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.796864 (+    52us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:02.796912 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.921225 (+2124313us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b5400478954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:04.921277 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.921402 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":99283,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":7673,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":9371,"spinlock_wait_cycles":125249792}
I20260709 13:03:04.969250  2317 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:04.733021  9198 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:04.920611  9380 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:05.005678  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:04.735757  9410 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.933670  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:05.011665  1629 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:04.931438  9197 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.013394  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:05.009505  2350 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:03:05.015154  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:04.908865  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3511) took 2674 ms. Trace:
I20260709 13:03:04.744308  9303 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.194s
I20260709 13:03:04.744657  9383 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.190s
I20260709 13:03:04.926267  9282 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.908578  9234 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.801132  9411 raft_consensus.cc:3055] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:05.023192  1553 raft_consensus.cc:1240] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:04.781606  9249 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:04.959618  1557 rpcz_store.cc:269] 0709 13:03:02.498143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.536994 (+ 38851us) service_pool.cc:224] Handling call
0709 13:03:04.920473 (+2383479us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54002ff654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:04.920612 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403578112}
I20260709 13:03:04.978469  1554 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:03:04.985908  1552 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:04.842149  9247 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:04.793509  9412 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:04.789825  9413 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.763475  9396 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:04.798379  9280 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:04.999430  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:04.917258  9260 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.911989  9219 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:04.985317  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:04.759474  9329 consensus_meta.cc:287] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 1.397s	user 0.013s	sys 0.001s
I20260709 13:03:04.929358  9319 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.004854  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:05.280557  2353 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:05.005211  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:05.007179  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:05.004902  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:05.282682  1556 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:03:05.016309  1561 rpcz_store.cc:276] 0709 13:03:02.234031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.235731 (+  1700us) service_pool.cc:224] Handling call
0709 13:03:04.908606 (+2672875us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b5400389ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:04.908791 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2120434432}
I20260709 13:03:05.018810  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:04.936395  9294 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:05.023137  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:04.916864  9183 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:04.980255  9360 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:05.029301  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:05.002089  9298 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:05.034858  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 120853us
I20260709 13:03:05.030575  9234 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.022441  9197 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:05.005957  9380 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.026479  9282 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:05.101038  9266 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.932260  9405 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.034906  9198 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.117690  9358 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:05.024616  9411 raft_consensus.cc:740] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:04.916740  9392 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:05.086750  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.036264  1552 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
W20260709 13:03:05.081964  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 159762us
I20260709 13:03:05.011072  9243 consensus_queue.cc:1059] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:03:05.000303  8809 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.176663  1602 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:05.252876  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:05.260140  1554 raft_consensus.cc:1240] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:03:05.028927  1558 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
W20260709 13:03:05.270784  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.024150  9410 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:03:05.009974  9290 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:05.387598  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 468354us
W20260709 13:03:05.410091  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 674923us
I20260709 13:03:05.683595  2353 raft_consensus.cc:2468] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:05.683926  1527 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:03:05.379647  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 394122us
W20260709 13:03:05.685704  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:05.565176  1525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.006s	sys 0.027s
W20260709 13:03:05.688338  1525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.011s	sys 0.032s
I20260709 13:03:05.688824  1554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47122 (request call id 2692) took 1381 ms. Trace:
I20260709 13:03:05.689424  1554 rpcz_store.cc:276] 0709 13:03:04.307634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.051265 (+743631us) service_pool.cc:224] Handling call
0709 13:03:05.100777 (+ 49512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:05.688422 (+587645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.688748 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.691234  1558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3526) took 1414 ms. Trace:
I20260709 13:03:05.691537  1560 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:03:05.691828  1558 rpcz_store.cc:276] 0709 13:03:04.276948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.939875 (+662927us) service_pool.cc:224] Handling call
0709 13:03:05.691164 (+751289us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:05.693449  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 516599us
I20260709 13:03:05.696864  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3319) took 3847 ms. Trace:
W20260709 13:03:05.680169  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 680388us
I20260709 13:03:05.697903  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:05.697453  1633 rpcz_store.cc:276] 0709 13:03:01.849725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.675921 (+826196us) service_pool.cc:224] Handling call
0709 13:03:03.320402 (+644481us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:03.321504 (+  1102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.696638 (+2375134us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54003cd7d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:05.696794 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1480384384}
I20260709 13:03:05.682634  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:05.684211  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 1544) took 1022 ms. Trace:
I20260709 13:03:05.700888  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:05.701295  2353 rpcz_store.cc:276] 0709 13:03:04.661375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.968177 (+306802us) service_pool.cc:224] Handling call
0709 13:03:05.684147 (+715970us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.701647  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:05.702559  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:05.703366  1702 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:05.702430  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:05.716663  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:05.716626  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:05.704330  1602 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:05.717892  1701 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:05.718371  1703 raft_consensus.cc:2393] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:05.249588  9247 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:05.758314  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.642617  9230 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:05.759508  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 76829.2us
I20260709 13:03:05.725340  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:05.629700  9158 consensus_meta.cc:287] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.718s	user 0.009s	sys 0.003s
I20260709 13:03:05.771674  1704 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:05.790467  2344 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:05.792304  1628 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:03:05.772236  9358 raft_consensus.cc:697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:03:05.720144  9385 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.085s	user 0.012s	sys 0.006s
I20260709 13:03:05.810176  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:05.838727  2353 raft_consensus.cc:2410] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:05.700139  9380 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:05.316247  9329 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.388873  9319 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:05.279661  9260 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:05.634253  9428 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.279866  9219 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:05.305034  9294 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:05.272291  9351 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:05.622026  9426 raft_consensus.cc:2764] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 13:03:05.717288  9290 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.716466  9429 raft_consensus.cc:2804] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:05.544570  9303 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:05.899111  9282 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:05.899595  9303 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:05.900352  9303 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:05.896771  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 212786us
I20260709 13:03:05.903728  9421 raft_consensus.cc:2804] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:05.904119  1527 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:05.904779  9421 raft_consensus.cc:697] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:05.905679  9429 raft_consensus.cc:697] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:05.905879  9421 consensus_queue.cc:237] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.906708  9429 consensus_queue.cc:237] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.909804  9290 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.912595  9290 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:05.913167  9290 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:05.816556  9243 consensus_queue.cc:1059] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.078s
I20260709 13:03:05.920714  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:03:05.820370  2318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.036s	sys 0.100s
I20260709 13:03:05.821075  1629 raft_consensus.cc:2393] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 26: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:05.720299  9424 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:05.608750  9425 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:03:05.589833  9387 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.875s	user 0.015s	sys 0.000s
W20260709 13:03:05.848110  1676 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.021s	sys 0.016s
I20260709 13:03:05.849175  2344 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:03:05.801651  9423 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:05.857745  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 1542) took 1342 ms. Trace:
W20260709 13:03:05.935912  1676 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.021s	sys 0.016s
I20260709 13:03:05.936224  2353 rpcz_store.cc:276] 0709 13:03:04.515033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.740817 (+1225784us) service_pool.cc:224] Handling call
0709 13:03:05.857666 (+116849us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:05.936429  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 725322us
I20260709 13:03:05.816534  9418 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.388360  9233 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:05.719312  9198 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:05.914907  9428 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:05.816969  9225 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:05.857784  9230 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:05.947485  9230 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.947934  9230 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:05.948264  9198 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:05.839841  9247 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.923954  9290 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.953995  9290 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:05.953650  9423 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:05.869604  8809 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:05.870489  9197 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:05.837127  9158 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:05.320626  9396 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:05.871054  9431 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:03:05.758584  9266 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.870664  9183 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:05.878602  9329 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:05.277875  9412 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.881734  9392 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:05.883280  9422 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:03:05.883726  9411 consensus_queue.cc:260] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:05.955238  9398 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:05.886415  9410 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:05.888257  9405 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:05.302258  9413 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:05.934479  9383 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.004s	sys 0.012s
W20260709 13:03:05.510311  9420 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.594s	user 0.002s	sys 0.000s
I20260709 13:03:05.900853  9303 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.977418  9398 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.983322  9303 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:05.983459  9398 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:05.894356  9351 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.984229  9351 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:05.984918  9396 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:05.985347  9413 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:05.977407  9383 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.004s	sys 0.013s
I20260709 13:03:05.986488  9411 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:05.986264  9290 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.961297  9418 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:05.988641  9290 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:05.870661  9358 consensus_queue.cc:237] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:05.989249  9418 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:05.935395  9425 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:05.991271  9412 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:05.959901  9247 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:05.960538  9319 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:05.936581  9426 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:05.859294  9385 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.956802  9387 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.970556  9294 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.995386  9387 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:05.980384  9441 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:05.981253  9266 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:05.935678  9424 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:05.998471  9294 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:05.999903  9424 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:05.998546  9385 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:06.003073  9423 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:06.004282  9324 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.380s	user 0.015s	sys 0.000s
I20260709 13:03:06.004387  9351 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.004828  9324 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.006784  9324 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:06.006784  9351 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:05.991683  9425 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:06.012046  9219 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.014612  9219 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:06.017354  9380 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:06.017995  9386 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.082s	user 0.015s	sys 0.000s
I20260709 13:03:06.018365  9230 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.018536  9386 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.019538  9380 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:06.020552  9386 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:06.020552  9230 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:05.973696  9438 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:06.020623  9260 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.022046  9438 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:06.022979  9318 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.023536  9260 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:06.023536  9318 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:06.024777  9438 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:06.063606  1525 connection.cc:256] RPC call timeout handler was delayed by -1.383552247999944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
W20260709 13:03:06.064791  1525 connection.cc:256] RPC call timeout handler was delayed by -1.326524989999939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1652s for a response
W20260709 13:03:06.065338  1525 connection.cc:256] RPC call timeout handler was delayed by -1.2513317319999828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
W20260709 13:03:06.065901  1525 connection.cc:256] RPC call timeout handler was delayed by -1.1532317319999947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2227s for a response
W20260709 13:03:06.066381  1525 connection.cc:256] RPC call timeout handler was delayed by -1.1180792859999542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 13:03:06.067730  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 146848us
W20260709 13:03:05.927789  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 227149us
I20260709 13:03:06.069005  1633 raft_consensus.cc:2376] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22. Re-sending same reply.
I20260709 13:03:05.928171  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3337) took 1020 ms. Trace:
W20260709 13:03:05.933084  1634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.001s	sys 0.005s
W20260709 13:03:05.929930  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 247232us
I20260709 13:03:06.079434  2350 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:05.973771  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1539) took 2813 ms. Trace:
I20260709 13:03:05.963207  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:05.924595  2318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.036s	sys 0.100s
W20260709 13:03:06.081799  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 377078us
W20260709 13:03:06.082209  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 118964us
I20260709 13:03:05.975764  2348 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:06.075520  1629 rpcz_store.cc:276] 0709 13:03:04.908240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.724939 (+816699us) service_pool.cc:224] Handling call
0709 13:03:05.928104 (+203165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:06.076076  1634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.001s	sys 0.005s
I20260709 13:03:06.090095  2349 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:06.089933  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:05.876963  1631 raft_consensus.cc:1240] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:06.095921  1707 raft_consensus.cc:2393] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:06.073633  1632 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
W20260709 13:03:06.067616  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 247236us
I20260709 13:03:06.080049  1674 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:06.080914  2353 rpcz_store.cc:276] 0709 13:03:03.160632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.967039 (+2806407us) service_pool.cc:224] Handling call
0709 13:03:05.973080 (+  6041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.973342 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.973376 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.973424 (+    48us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:05.973500 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.973567 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.973701 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:06.075078  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 139768us
W20260709 13:03:06.102757  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.101729  9386 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:06.104118  9386 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:06.104789  9386 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:06.105990  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:06.110168  1525 connection.cc:256] RPC call timeout handler was delayed by -1.1108984740000096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
I20260709 13:03:06.110870  9429 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:06.111290  1525 connection.cc:256] RPC call timeout handler was delayed by -1.0991522479999389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
I20260709 13:03:06.113104  1525 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:06.101979  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:06.117838  1629 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:03:06.118924  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3341) took 1209 ms. Trace:
I20260709 13:03:06.119390  1629 rpcz_store.cc:276] 0709 13:03:04.909464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.088088 (+1178624us) service_pool.cc:224] Handling call
0709 13:03:06.088962 (+   874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.118473 (+ 29511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.118769 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:06.126523  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.117177  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:06.128229  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.133975  9446 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:06.136204  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 1549) took 1221 ms. Trace:
W20260709 13:03:06.139274  2352 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1740) took 3050 ms (client timeout 1652 ms). Trace:
W20260709 13:03:06.139936  2352 rpcz_store.cc:269] 0709 13:03:03.088527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.257557 (+169030us) service_pool.cc:224] Handling call
0709 13:03:06.138486 (+2880929us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54009902d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:06.138943 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603972480}
I20260709 13:03:06.141234  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:06.144176  2344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 1545) took 1410 ms (client timeout 1689 ms). Trace:
I20260709 13:03:06.156023  9386 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.145974  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:06.153666  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:06.161931  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 58743.3us
I20260709 13:03:06.163079  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1737) took 3099 ms. Trace:
I20260709 13:03:06.163609  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:06.147342  2344 rpcz_store.cc:269] 0709 13:03:04.734046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.911904 (+177858us) service_pool.cc:224] Handling call
0709 13:03:06.143716 (+1231812us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54006202d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:06.143962 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1906792832}
I20260709 13:03:06.164022  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:06.165138  2345 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
I20260709 13:03:06.166659  9260 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:06.168042  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:06.167925  9260 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.105239  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:06.136902  2350 rpcz_store.cc:276] 0709 13:03:04.914320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.020970 (+106650us) service_pool.cc:224] Handling call
0709 13:03:06.135751 (+1114781us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b5400541ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:06.136071 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897119104}
I20260709 13:03:06.173877  9446 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:06.182598  9446 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:06.172519  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 66825.2us
I20260709 13:03:06.183156  9446 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:06.183169  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:06.184268  2350 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:06.179554  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:06.186674  1629 raft_consensus.cc:2393] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:06.188107  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:06.163961  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:06.193145  9298 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.194288  9437 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:06.200205  9358 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:03:06.194345  9298 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:06.197541  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 118055us
I20260709 13:03:06.200363  9385 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:06.163662  2351 rpcz_store.cc:276] 0709 13:03:03.063799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.139520 (+1075721us) service_pool.cc:224] Handling call
0709 13:03:04.139838 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.654234 (+514396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.162312 (+1508078us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54006185d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:06.162802 (+   490us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1092450688}
I20260709 13:03:06.211186  9386 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:06.218782  9429 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:06.219605  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 114643us
W20260709 13:03:06.219852  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 99318.5us
I20260709 13:03:06.220521  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:06.221807  2351 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:06.225816  9294 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:06.226861  9294 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:06.230391  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:06.231904  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:06.232738  2353 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:06.236917  9298 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:06.237728  9451 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.005s
I20260709 13:03:06.239224  2351 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:06.240643  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:06.241566  2351 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:06.241271  9380 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:03:06.246469  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 83174.9us
I20260709 13:03:06.217221  2352 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:06.250850  1527 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:06.248615  1675 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:06.261273  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:06.265724  9260 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:06.271884  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3336) took 3881 ms. Trace:
I20260709 13:03:06.272302  1635 raft_consensus.cc:1217] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Deduplicated request from leader. Original: 12.11->[13.12-17.14]   Dedup: 17.14->[]
W20260709 13:03:06.273026  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 51741.4us
I20260709 13:03:06.275167  9441 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:06.275790  9441 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:06.279400  9280 raft_consensus.cc:3055] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:06.279937  9280 raft_consensus.cc:740] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:06.277459  9441 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.221225  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:06.273164  1635 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: [13.12-17.14]
I20260709 13:03:06.284210  2349 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:06.272763  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:06.283835  1675 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:06.278978  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:06.282902  1674 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:06.277267  9234 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:06.284394  9158 raft_consensus.cc:2804] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:06.307392  9423 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:03:06.307543  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 161855us
I20260709 13:03:06.308480  9423 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:06.310683  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.294535  9280 consensus_queue.cc:260] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:06.313683  9280 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:06.277944  9425 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:03:06.317804  9168 raft_consensus.cc:2804] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:06.318555  9168 raft_consensus.cc:697] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:06.321342  9158 raft_consensus.cc:697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:06.323096  9158 consensus_queue.cc:237] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.325632  2348 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:03:06.325899  9115 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:06.319902  9168 consensus_queue.cc:237] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:06.327780  9226 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:06.329860  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 109121us
I20260709 13:03:06.309168  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:06.229063  9294 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:06.342043  2351 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:03:06.348111  2352 raft_consensus.cc:1240] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:06.343887  1602 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:06.349037  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:06.350697  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 68139.6us
I20260709 13:03:06.354342  2353 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
W20260709 13:03:06.357287  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 111903us
I20260709 13:03:06.358916  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:06.360061  9234 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:06.361713  9437 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:03:06.363940  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54993us
I20260709 13:03:06.379738  2350 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:03:06.369867  9226 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:06.383687  2351 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:03:06.272743  1634 rpcz_store.cc:276] 0709 13:03:02.390447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.105657 (+1715210us) service_pool.cc:224] Handling call
0709 13:03:04.158683 (+ 53026us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:03:04.276600 (+117917us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:04.956526 (+679926us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.090835 (+1134309us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:06.108595 (+ 17760us) log.cc:844] Serialized 78 byte log entry
0709 13:03:06.108991 (+   396us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:06.202294 (+ 93303us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.218164 (+ 15870us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.254890 (+ 36726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.271430 (+ 16540us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:06.271498 (+    68us) raft_consensus.cc:1685] finished
0709 13:03:06.271509 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.271782 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54737,"prepare.queue_time_us":522552,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":38,"raft.queue_time_us":376081,"raft.run_cpu_time_us":930,"raft.run_wall_time_us":26384,"thread_start_us":1819596,"threads_started":3,"wal-append.queue_time_us":955987}
W20260709 13:03:06.385586  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 133456us
I20260709 13:03:06.380667  9220 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:06.387318  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1406) took 1716 ms. Trace:
I20260709 13:03:06.395658  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:06.391237  1602 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:06.382323  9425 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:06.388904  1631 rpcz_store.cc:276] 0709 13:03:04.628306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.789178 (+1160872us) service_pool.cc:224] Handling call
0709 13:03:05.789672 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.093609 (+303937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.337274 (+243665us) spinlock_profiling.cc:243] Waited 87 ms on lock 0x7b540017af54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:06.344577 (+  7303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200017792}
W20260709 13:03:06.409126  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 130623us
I20260709 13:03:06.406672  9425 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:06.410342  9425 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:06.411175  9252 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.594s	user 0.013s	sys 0.024s
W20260709 13:03:06.411715  9252 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.632s	user 0.014s	sys 0.024s
I20260709 13:03:06.396385  9385 consensus_queue.cc:1059] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 13:03:06.414255  9443 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:06.415165  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:06.415037  9443 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:06.415570  9385 raft_consensus.cc:3055] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:06.416000  9443 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:06.416260  9385 raft_consensus.cc:740] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:06.418108  9426 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:06.407042  9446 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:06.418972  9426 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:06.419445  9426 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:06.419252  9385 consensus_queue.cc:260] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.420928  9385 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:06.421947  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 79978.3us
I20260709 13:03:06.396137  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:06.395394  9115 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:06.425077  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:06.388571  9220 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.429541  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:06.430461  9260 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:06.429586  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:06.433938  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1372) took 3142 ms. Trace:
I20260709 13:03:06.421197  9446 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:06.459452  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 73075.5us
I20260709 13:03:06.442822  2344 rpcz_store.cc:276] 0709 13:03:03.291447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.164931 (+2873484us) service_pool.cc:224] Handling call
0709 13:03:06.166684 (+  1753us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.170362 (+  3678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.170432 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:06.171563 (+  1131us) log.cc:844] Serialized 26 byte log entry
0709 13:03:06.340406 (+168843us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:06.340480 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.340583 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:06.433592 (+ 93009us) raft_consensus.cc:1685] finished
0709 13:03:06.433630 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.433835 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2274,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":308,"thread_start_us":168071,"threads_started":1,"wal-append.queue_time_us":214640}
I20260709 13:03:06.442682  9220 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:06.467717  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 103866us
I20260709 13:03:06.434939  9115 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:06.472662  9233 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:06.472965  1602 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:06.473371  9233 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.473948  9233 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:06.477814  1525 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:06.479262  9329 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:06.479911  9329 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:06.478048  2344 raft_consensus.cc:1275] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:06.480521  9329 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:06.467353  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:06.499310  9252 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:06.484504  9260 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:06.519419  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:06.501200  9252 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.495139  9446 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:06.535923  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45908.9us
I20260709 13:03:06.545326  9226 consensus_queue.cc:1059] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:03:06.546729  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 46539.6us
I20260709 13:03:06.534984  9252 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:06.572330  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:06.573242  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:06.573581  9289 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:06.573915  9294 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:06.574333  1629 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:06.574435  9289 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:06.575325  9226 raft_consensus.cc:3055] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:06.575843  9226 raft_consensus.cc:740] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:06.576516  9289 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:06.574756  9294 consensus_queue.cc:1243] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:06.577409  9226 consensus_queue.cc:260] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.591085  9226 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:06.590725  9115 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.592890  9441 consensus_queue.cc:1059] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:06.594842  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:06.583600  9289 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:06.596625  9289 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:06.597143  9460 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:06.603003  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:06.607621  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.615692  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 129174us
W20260709 13:03:06.619834  9466 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 13:03:06.624585  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 107786us
I20260709 13:03:06.639168  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:06.641991  1708 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:06.642364  1704 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 24.
I20260709 13:03:06.642693  9289 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:06.644979  1707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 1276) took 1929 ms (client timeout 1631 ms). Trace:
I20260709 13:03:06.646555  1701 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:06.647444  9427 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:06.648092  9427 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:06.650197  9427 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.651852  9427 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:06.652606  2353 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 13:03:06.653291  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:06.653398  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:06.654078  1630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3646) took 3485 ms (client timeout 3882 ms). Trace:
I20260709 13:03:06.655032  2350 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:03:06.658573  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:06.661278  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.661545  1707 rpcz_store.cc:269] 0709 13:03:04.715828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.716544 (+   716us) service_pool.cc:224] Handling call
0709 13:03:06.644595 (+1928051us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002e86d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:06.644856 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2742895872}
I20260709 13:03:06.671876  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:06.673077  1631 raft_consensus.cc:2410] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:03:06.615473  9220 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:06.663898  1630 rpcz_store.cc:269] 0709 13:03:03.168203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.483058 (+314855us) service_pool.cc:224] Handling call
0709 13:03:06.653620 (+3170562us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54002dbed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:06.653851 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2836815232}
W20260709 13:03:06.724475  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.725383  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.729622  9289 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:06.731076  9329 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:06.736340  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.739086  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:06.740139  1707 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:06.740527  9318 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:06.743444  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:06.744724  1630 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:03:06.745735  2348 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 1547) took 1832 ms (client timeout 1640 ms). Trace:
I20260709 13:03:06.745895  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:06.746699  1630 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:03:06.746227  2348 rpcz_store.cc:269] 0709 13:03:04.913720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.983162 (+ 69442us) service_pool.cc:224] Handling call
0709 13:03:06.745438 (+1762276us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b5400550354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:06.745647 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1589301120}
I20260709 13:03:06.748871  9441 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:06.743620  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1402) took 3388 ms. Trace:
I20260709 13:03:06.765748  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:06.750252  1635 rpcz_store.cc:276] 0709 13:03:03.355068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.188227 (+833159us) service_pool.cc:224] Handling call
0709 13:03:06.271834 (+2083607us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:06.294128 (+ 22294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.743329 (+449201us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b540017be50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:06.743527 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4787725184}
I20260709 13:03:06.770094  9441 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:06.778831  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:06.779754  1635 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:06.781589  9289 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.782349  9289 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:06.785044  9289 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:06.806772  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1407) took 2178 ms. Trace:
I20260709 13:03:06.828025  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:06.827996  1628 rpcz_store.cc:276] 0709 13:03:04.628699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.669200 (+ 40501us) service_pool.cc:224] Handling call
0709 13:03:06.806321 (+2137121us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b5400349ed4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:06.806546 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1996787072}
W20260709 13:03:06.786126  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 213849us
I20260709 13:03:06.685451  9443 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:06.833416  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.835966  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.807124  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42562 (request call id 1219) took 2158 ms. Trace:
I20260709 13:03:06.786010  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:06.799074  9400 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.998s	user 0.010s	sys 0.009s
W20260709 13:03:06.816031  9462 log.cc:927] Time spent T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.294s	user 0.002s	sys 0.000s
I20260709 13:03:06.842348  1634 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:06.715577  9247 consensus_queue.cc:1048] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.275s
I20260709 13:03:06.845336  9329 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:06.831696  9446 consensus_queue.cc:1059] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.007s
I20260709 13:03:06.849224  9443 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:06.851486  9099 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:06.854609  9336 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.864959  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:06.865949  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:06.867774  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 151461us
I20260709 13:03:06.868669  9410 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:06.836690  1560 rpcz_store.cc:276] 0709 13:03:04.648400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.029747 (+381347us) service_pool.cc:224] Handling call
0709 13:03:05.691183 (+661436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:05.692153 (+   970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.806675 (+1114522us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b5400304dd0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:06.806893 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370479488}
I20260709 13:03:06.844482  9428 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:06.800943  9419 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.705s	user 0.004s	sys 0.002s
W20260709 13:03:06.796306  9389 log.cc:927] Time spent T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 2.194s	user 0.004s	sys 0.027s
W20260709 13:03:06.802717  9402 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 2.082s	user 0.008s	sys 0.038s
I20260709 13:03:06.786458  9441 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:06.898123  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.909603  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.785369  9288 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:06.896008  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:06.916337  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:06.767719  9260 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.940856  1703 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:03:06.949033  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.949301  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 203435us
I20260709 13:03:06.947681  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:06.969595  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:06.972643  1703 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:03:06.978026  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 267778us
I20260709 13:03:06.982767  1628 raft_consensus.cc:2368] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20.
W20260709 13:03:06.984696  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 169210us
I20260709 13:03:06.996707  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:06.972170  9441 consensus_queue.cc:1059] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:03:07.003088  1629 raft_consensus.cc:2468] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 23.
I20260709 13:03:06.981180  9370 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:07.006489  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:07.007599  1709 raft_consensus.cc:2368] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 18. Current term is 19.
I20260709 13:03:07.009848  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:06.994798  9220 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:07.019006  1704 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 17: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:07.019853  9441 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.021677  9099 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.022465  9099 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:06.973191  9336 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:03:07.021188  1702 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:07.022226  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1732) took 4022 ms. Trace:
I20260709 13:03:07.025656  9289 raft_consensus.cc:3055] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:07.026387  9336 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:07.026769  9289 raft_consensus.cc:740] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:07.028117  9289 consensus_queue.cc:260] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.028750  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:07.022328  9446 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.030438  9446 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:07.030819  9470 consensus_queue.cc:1059] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:07.031898  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3649) took 3862 ms. Trace:
I20260709 13:03:07.032406  1676 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:07.033581  9471 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:07.033788  9397 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:07.034346  9471 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:07.034792  9397 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:07.035619  9397 raft_consensus.cc:697] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:07.033020  9446 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:07.037945  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:07.039062  1629 raft_consensus.cc:2393] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:07.026245  2347 rpcz_store.cc:276] 0709 13:03:02.999869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.000034 (+   165us) service_pool.cc:224] Handling call
0709 13:03:03.000400 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.899200 (+1898800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.021623 (+2122423us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b5400686d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:07.021969 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3819602688}
I20260709 13:03:07.040455  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:07.035969  9289 raft_consensus.cc:3060] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:07.032346  1636 rpcz_store.cc:276] 0709 13:03:03.169054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.953616 (+784562us) service_pool.cc:224] Handling call
0709 13:03:04.101443 (+147827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.141800 (+ 40357us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:04.601159 (+459359us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.814793 (+2213634us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b18004231c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a
0709 13:03:06.815224 (+   431us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.815341 (+   117us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:06.815499 (+   158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.031411 (+215912us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b54002db9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:07.031466 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.031692 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44507,"prepare.queue_time_us":603579,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":14928,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":259,"spinlock_wait_cycles":113463808,"thread_start_us":587374,"threads_started":2,"wal-append.queue_time_us":292015}
I20260709 13:03:07.022071  9441 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:07.044548  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:07.045756  9099 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:07.031898  9470 raft_consensus.cc:3055] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:07.047104  9470 raft_consensus.cc:740] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:07.048405  9470 consensus_queue.cc:260] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.050112  9470 raft_consensus.cc:3060] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:07.057883  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
W20260709 13:03:07.052357  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 43571.7us
I20260709 13:03:07.057770  9446 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:07.059736  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.060781  9329 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:07.061390  9329 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:07.062520  9329 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:07.037281  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:07.063258  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:07.066394  9099 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:07.067209  9446 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:07.067653  9446 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:07.068074  1636 raft_consensus.cc:3060] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:07.068615  9446 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.072111  9446 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:07.080354  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:07.081285  1629 raft_consensus.cc:2368] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19.
I20260709 13:03:07.081463  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:07.034909  9471 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:07.044903  9441 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:07.105767  1702 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:07.106662  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:07.106160  1630 raft_consensus.cc:2468] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 26.
I20260709 13:03:07.111613  1635 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:03:07.115985  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:07.081122  9426 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.661s	user 0.008s	sys 0.009s
I20260709 13:03:07.110901  1631 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 17.
I20260709 13:03:07.121598  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:07.120401  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:07.123275  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:07.127444  1634 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:03:07.128165  9441 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:07.037106  9397 consensus_queue.cc:237] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.129845  1702 raft_consensus.cc:1240] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:07.130797  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:07.131618  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:07.119210  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:07.085119  9099 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:07.141366  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:07.142417  1634 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:07.145841  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:07.150331  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:07.152001  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:07.155931  1636 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 16.
I20260709 13:03:07.157279  9442 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:07.157713  1636 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:07.158831  9322 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:07.159669  1636 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:03:07.162937  9374 raft_consensus.cc:3055] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:07.163496  9374 raft_consensus.cc:740] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:07.167266  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:07.169884  1631 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:07.170511  9473 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:07.174279  1636 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:03:07.174651  1632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3339) took 2265 ms (client timeout 1899 ms). Trace:
I20260709 13:03:07.175406  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:07.175599  9379 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.068s
I20260709 13:03:07.176416  1636 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:07.177567  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:07.178143  1629 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:07.178303  1636 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:03:07.178478  9099 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.179644  1636 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:03:07.182770  2347 raft_consensus.cc:1275] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:03:07.175094  1632 rpcz_store.cc:269] 0709 13:03:04.908823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.001593 (+ 92770us) service_pool.cc:224] Handling call
0709 13:03:07.174363 (+2172770us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b5400197654. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:07.174557 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811873024}
W20260709 13:03:07.186661  9425 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.776s	user 0.014s	sys 0.004s
I20260709 13:03:07.187175  9425 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:07.187985  9385 consensus_meta.cc:287] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.767s	user 0.014s	sys 0.002s
I20260709 13:03:07.188300  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1548) took 2274 ms. Trace:
W20260709 13:03:07.189736  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 51089.7us
W20260709 13:03:07.189991  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 178230us
I20260709 13:03:07.188700  2345 rpcz_store.cc:276] 0709 13:03:04.914022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.166933 (+1252911us) service_pool.cc:224] Handling call
0709 13:03:06.167515 (+   582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.257578 (+ 90063us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:06.302200 (+ 44622us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.382795 (+ 80595us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.382865 (+    70us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:06.385647 (+  2782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.188035 (+802388us) spinlock_profiling.cc:243] Waited 85.1 ms on lock 0x7b5400756554. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:07.188088 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.188213 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":62043,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":1626,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":14054,"spinlock_wait_cycles":195818240,"thread_start_us":93204,"threads_started":2,"wal-append.queue_time_us":102496}
I20260709 13:03:07.196585  9441 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:07.197642  9443 raft_consensus.cc:3055] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:07.198139  9443 raft_consensus.cc:740] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:07.201337  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:07.202914  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:07.203433  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:07.203994  1706 raft_consensus.cc:2410] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 13:03:07.204633  1627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1397) took 3906 ms (client timeout 1687 ms). Trace:
I20260709 13:03:07.200930  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:07.206133  1708 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:03:07.213135  1632 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
W20260709 13:03:07.217098  2349 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 1543) took 2557 ms (client timeout 1831 ms). Trace:
I20260709 13:03:07.149853  1630 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
W20260709 13:03:07.221272  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 258122us
I20260709 13:03:07.177381  1633 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:03:07.214111  1707 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:03:07.220533  2349 rpcz_store.cc:269] 0709 13:03:04.659825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.958365 (+298540us) service_pool.cc:224] Handling call
0709 13:03:07.216785 (+2258420us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b54006bd354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:07.217017 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1856684800}
W20260709 13:03:07.230702  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.205052  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:07.205103  1627 rpcz_store.cc:269] 0709 13:03:03.293467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.300333 (+  6866us) service_pool.cc:224] Handling call
0709 13:03:07.199585 (+3899252us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b540010c7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:07.199759 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1907102976}
W20260709 13:03:07.201071  9233 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.726s	user 0.015s	sys 0.002s
I20260709 13:03:07.120537  9115 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 13:03:07.241698  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:07.238881  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:07.205699  9428 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:07.182825  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.246474  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:07.126619  9426 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:07.252024  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 69323.7us
W20260709 13:03:07.254271  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.220194  9425 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:07.257754  9234 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:07.257786  1707 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:07.258814  9423 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:07.164911  9374 consensus_queue.cc:260] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:07.260746  9411 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:07.266147  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:07.278067  9436 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:07.278473  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:07.290499  2347 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:07.278118  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:07.291694  2349 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:03:07.293005  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.260963  1701 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:07.283866  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:07.296245  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:07.297613  1630 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:07.291153  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.302500  1627 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:03:07.306295  2318 connection.cc:256] RPC call timeout handler was delayed by -1.5507209330000364s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260709 13:03:07.312207  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 61241.6us
I20260709 13:03:07.313162  1674 leader_election.cc:400] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19.
I20260709 13:03:07.319859  1674 leader_election.cc:403] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:03:07.241041  9233 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.299605  9441 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:07.325536  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:07.329869  2348 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 25. Current term is 26. Ops: []
I20260709 13:03:07.330159  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:07.336769  2353 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:07.332897  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:07.327903  2346 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEARNER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:07.332145  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:07.338936  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54303.8us
I20260709 13:03:07.356623  1633 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:07.314066  9234 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:07.365202  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:07.372659  1702 raft_consensus.cc:1240] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:03:07.342588  9115 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:03:07.348467  9427 consensus_meta.cc:287] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.673s	user 0.013s	sys 0.003s
I20260709 13:03:07.375352  1557 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:07.375633  9234 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.377770  1632 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:07.377527  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.378283  9294 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.379230  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.380044  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.382268  1634 raft_consensus.cc:3055] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:07.393276  1634 raft_consensus.cc:740] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:07.352275  9441 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:07.398571  9294 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:07.402742  1634 consensus_queue.cc:260] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:07.404928  2347 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:03:07.406788  1553 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 22: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:03:07.412977  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47904.3us
I20260709 13:03:07.410477  1674 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.419163  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:07.398387  9436 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:07.405522  9115 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 26 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:07.427834  1634 raft_consensus.cc:3060] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:07.368160  9298 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 13:03:07.427734  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 231177us
I20260709 13:03:07.348692  9443 consensus_queue.cc:260] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.437072  1555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.444082  1674 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:07.448829  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.440037  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:03:07.455905  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.457718  1558 raft_consensus.cc:2410] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:07.437107  9379 raft_consensus.cc:2804] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:07.465755  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.471798  1554 raft_consensus.cc:2410] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 13:03:07.446906  9427 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.475026  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:07.477689  1555 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:07.485659  9404 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19.
I20260709 13:03:07.470283  9407 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:03:07.499476  9115 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:07.500332  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:07.502031  1708 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:07.490886  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:07.445230  9358 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:03:07.495443  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:07.453223  9288 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:03:07.511042  1632 raft_consensus.cc:2393] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 18: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:07.497531  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:03:07.500849  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 115544us
W20260709 13:03:07.496007  9410 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.576s	user 0.007s	sys 0.006s
I20260709 13:03:07.521752  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.479709  9379 raft_consensus.cc:697] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:07.523945  1552 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:07.483342  9426 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:07.533274  1634 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:03:07.500627  9324 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:07.540771  2317 leader_election.cc:400] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20.
I20260709 13:03:07.543084  2317 leader_election.cc:403] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 13:03:07.537184  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 93133.7us
I20260709 13:03:07.545321  1708 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:07.511766  9443 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:07.534580  1675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.556s (SENT) [suppressed 17 similar messages]
I20260709 13:03:07.548254  1557 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:07.523232  9410 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:07.557806  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:07.549860  9411 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.565702  9411 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:07.570098  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.571259  9423 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.575695  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:07.578089  1556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.578501  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.579131  1556 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:03:07.579665  1554 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 22: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:07.571919  9423 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:07.581831  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 69281.8us
I20260709 13:03:07.582536  9099 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:07.586107  1553 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:03:07.563270  1674 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.593686  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:03:07.582471  9282 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 13:03:07.505650  9427 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.581228  1554 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 21. Current term is 22. Ops: [21.18-21.18]
I20260709 13:03:07.592585  2317 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:07.598937  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 165794us
I20260709 13:03:07.601843  1557 raft_consensus.cc:1240] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:07.601941  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.603273  9404 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:07.603579  1557 raft_consensus.cc:1275] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:07.603855  9404 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:07.604400  9404 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:07.604735  1559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:07.597406  9370 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20.
I20260709 13:03:07.524344  9329 consensus_queue.cc:1048] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:03:07.603231  1674 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.602183  9324 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:03:07.646224  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 106466us
I20260709 13:03:07.679397  1557 raft_consensus.cc:1240] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:07.614341  9324 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.615975  9233 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:07.687286  1560 raft_consensus.cc:1275] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:03:07.690428  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:07.690924  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.707398  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:07.685542  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.710807  1710 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:07.710976  1703 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:03:07.712098  1676 leader_election.cc:336] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257): Timed out: RequestConsensusVote RPC to 127.0.107.196:36257 timed out after 1.579s (SENT)
I20260709 13:03:07.685153  9324 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:07.684253  9298 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:07.684581  9370 raft_consensus.cc:2804] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:07.594615  9441 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:07.674952  9436 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.042s
I20260709 13:03:07.715986  9423 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:07.711736  9407 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.736581  1560 raft_consensus.cc:1275] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:03:07.743373  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 64336.5us
I20260709 13:03:07.720737  9298 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.747690  1557 raft_consensus.cc:1275] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:03:07.717593  9321 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:03:07.766557  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:03:07.776070  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 193098us
W20260709 13:03:07.769814  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 252011us
I20260709 13:03:07.750001  9298 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:07.743453  9370 raft_consensus.cc:697] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:07.694200  9379 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:07.789263  9322 consensus_meta.cc:287] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.630s	user 0.016s	sys 0.001s
I20260709 13:03:07.801911  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.803261  1554 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:03:07.807317  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.807523  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.810451  1553 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:03:07.810731  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.807072  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 263040us
I20260709 13:03:07.763679  9407 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:07.828976  9492 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.829416  1705 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:07.830084  9395 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.830565  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.830067  9370 consensus_queue.cc:237] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.831367  1553 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 17.
W20260709 13:03:07.826566  9252 consensus_meta.cc:287] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.205s	user 0.009s	sys 0.008s
I20260709 13:03:07.833104  9252 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.833653  9427 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:07.835784  9252 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:07.830940  9395 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.829891  9492 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.837980  2351 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:07.840276  9492 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:07.840914  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:07.841713  1701 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
W20260709 13:03:07.844028  1525 connection.cc:256] RPC call timeout handler was delayed by -1.059638479000057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
I20260709 13:03:07.844137  9476 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:07.844980  9476 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:07.847777  9476 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:07.855921  9407 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:07.613003  1706 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:03:07.893055  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:07.893518  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:07.928182  1552 raft_consensus.cc:3060] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:07.892565  1557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:07.892110  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
W20260709 13:03:07.925179  2353 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.561s	user 0.006s	sys 0.005s
I20260709 13:03:07.932446  2353 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:07.884666  1602 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.929733  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:07.807262  9438 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:07.861382  9471 consensus_meta.cc:287] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.734s	user 0.007s	sys 0.002s
W20260709 13:03:07.930546  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 208419us
I20260709 13:03:07.938714  1555 raft_consensus.cc:1240] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 21. Current term is 22. Ops: []
W20260709 13:03:07.934693  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 119909us
W20260709 13:03:07.930465  9446 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.855s	user 0.002s	sys 0.012s
W20260709 13:03:07.950433  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 160706us
I20260709 13:03:07.952302  2317 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.953298  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.954411  2317 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:07.955526  9499 raft_consensus.cc:2804] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:03:07.956882  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:03:07.958098  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 158896us
I20260709 13:03:07.956980  1527 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:07.977312  1557 raft_consensus.cc:3060] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:07.975453  1560 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:07.937506  9329 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:07.936358  9438 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:07.989260  9438 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:07.987022  1554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1"
I20260709 13:03:07.853353  9404 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:08.005618  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:08.012418  1558 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 22: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:07.994269  9427 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:08.031492  2351 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1748) took 1603 ms. Trace:
I20260709 13:03:08.032120  2351 rpcz_store.cc:276] 0709 13:03:06.427756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.155386 (+727630us) service_pool.cc:224] Handling call
0709 13:03:07.155807 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.838525 (+682718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.031086 (+192561us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540048ab54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:08.031391 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1567283072}
I20260709 13:03:07.996731  9358 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 13:03:08.037887  1709 raft_consensus.cc:1240] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:07.945394  9471 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.040704  9471 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:07.997093  9426 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 13:03:08.042341  9294 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:07.999637  9329 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:03:08.043906  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3387) took 4820 ms (client timeout 1567 ms). Trace:
I20260709 13:03:08.044340  9329 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 6
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:03:08.019719  1701 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:08.025804  9498 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:08.026398  9350 raft_consensus.cc:2764] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 13:03:08.044376  1705 rpcz_store.cc:269] 0709 13:03:03.223570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.230856 (+  7286us) service_pool.cc:224] Handling call
0709 13:03:08.043595 (+4812739us) spinlock_profiling.cc:243] Waited 4.53 s on lock 0x7b54002e95d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:08.043802 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10412898176}
I20260709 13:03:08.089303  1561 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:08.089380  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:08.090593  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:08.018339  9404 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:07.997684  9298 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:07.999157  9497 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:08.045862  9321 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:08.046205  1674 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:08.046496  9395 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:08.051935  9483 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.262s	user 0.000s	sys 0.006s
I20260709 13:03:08.054697  9374 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:08.055707  1552 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
W20260709 13:03:08.056705  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.058840  9358 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:03:08.059777  9493 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.109s	user 0.001s	sys 0.001s
W20260709 13:03:08.061239  9485 log.cc:927] Time spent T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.265s	user 0.006s	sys 0.010s
I20260709 13:03:08.066422   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:03:08.117121  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:08.118016  1710 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:08.065202  1453 catalog_manager.cc:5697] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 12 to 13, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:08.071368  9385 raft_consensus.cc:3055] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:03:08.074753  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64122.2us
I20260709 13:03:08.077991  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:08.005126  9099 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:03:08.082793  1554 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 16: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:03:08.083029  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 54857.6us
I20260709 13:03:08.125028  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:08.080238  1558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:08.075296  1559 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:08.086241  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
W20260709 13:03:08.026727  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.080204  9324 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:08.036845  9411 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.103976  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:08.110170  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:08.093359  9260 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.013s	sys 0.033s
I20260709 13:03:08.104214  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:08.113183  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:08.103729  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 57598.9us
I20260709 13:03:08.092535  9498 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:08.126169  1558 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:03:08.133656  1552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:08.136420  1552 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
I20260709 13:03:08.134445  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:08.137425  2347 raft_consensus.cc:2468] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:03:08.138774  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.140825  1553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d62ce354d09a4be59c29094425d09bc1" is_pre_election: true
I20260709 13:03:08.142138  1553 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:03:08.087987  9427 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.041s
I20260709 13:03:08.139029  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:08.139518  1560 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 19.
I20260709 13:03:08.116055  9500 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:08.148875  1634 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
W20260709 13:03:08.142812  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 147809us
I20260709 13:03:08.150240  1704 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
W20260709 13:03:08.131394  9260 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.014s	sys 0.034s
I20260709 13:03:08.156250  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:08.128082  9099 consensus_queue.cc:1243] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer ba0548b7d2a94e8598475a46c279fe0e log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:03:08.152518  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 62026.1us
I20260709 13:03:08.152294  2318 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:08.128424  9471 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:08.154052  2317 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:08.122764  9497 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:08.113821  9358 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:08.171758  9497 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.172581  9358 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.128582  9385 raft_consensus.cc:740] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:08.173916  9498 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.133698  9426 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:08.166765  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:08.127705  9370 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:08.154781  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.163461  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:08.165325  9321 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:08.165920  1627 raft_consensus.cc:1240] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 22. Current term is 23. Ops: [19.13-22.15]
I20260709 13:03:08.173765  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.133514  9115 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.162530  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:08.191325  1710 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:08.181427  1631 raft_consensus.cc:2393] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 16: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:08.186892  2353 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:03:08.166262  9294 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.152997  9500 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:08.204560  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:08.205814  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:08.204715  1632 raft_consensus.cc:2468] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:08.210222  1634 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:08.174081  9358 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:08.147683  9436 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
W20260709 13:03:08.218549  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 353035us
I20260709 13:03:08.178889  9494 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:08.227826  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:08.218194  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:08.216717  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.203039  9427 consensus_queue.cc:1243] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer 95fc6a4cf5874b7eb91280dffcd3b3fb log is divergent from this leader: its last log entry 24.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 13:03:08.236140  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.239006  1631 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:08.240278  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 90261.6us
I20260709 13:03:08.164124  9499 raft_consensus.cc:697] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:08.245919  1632 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:08.203387  9426 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:08.216528  9500 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:08.224243  9506 raft_consensus.cc:2764] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:03:08.265044  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:08.261906  9260 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:08.266836  9495 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 18.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:03:08.258422  9511 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:08.268167  9511 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:08.268610  9511 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:08.268879  9487 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.295s	user 0.000s	sys 0.002s
I20260709 13:03:08.269902  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.270650  1627 raft_consensus.cc:2468] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:03:08.271088  9395 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:08.271549  9395 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:08.271698  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:08.273026  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.272609  9395 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.265894  2344 raft_consensus.cc:2376] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18. Re-sending same reply.
W20260709 13:03:08.265026  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.273313  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:08.195883  9370 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.237397  9494 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:08.277331  1627 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:08.242161  9289 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.333293  9289 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:08.268610  9215 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.334067  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.334568  9215 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:08.335371  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:08.335807  9289 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:08.335981  1630 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:08.336939  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.340759  1634 raft_consensus.cc:2468] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:03:08.300973  2353 raft_consensus.cc:2376] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20. Re-sending same reply.
I20260709 13:03:08.253875  9220 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.204437  9115 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:08.318971  1557 raft_consensus.cc:2468] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
I20260709 13:03:08.332401  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:08.237661  9294 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:08.360646  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 206538us
I20260709 13:03:08.342324  1674 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:08.363240  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 153699us
I20260709 13:03:08.302124  9324 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:08.383679  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:08.366950  1627 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:08.337841  9494 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:08.371547  9411 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:08.335383  9370 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:08.374408  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 115196us
I20260709 13:03:08.379395  1634 raft_consensus.cc:1240] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 13:03:08.422580  1633 raft_consensus.cc:2410] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
W20260709 13:03:08.432694  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.388918  9436 raft_consensus.cc:2764] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:03:08.437987  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.439759  1634 raft_consensus.cc:2410] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:03:08.370675  9220 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.383092  9321 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:03:08.389390  9215 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:08.427562  1631 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:08.443614  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:08.448921  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:08.447252  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:08.451344  1631 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
W20260709 13:03:08.456434  9502 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:03:08.460979  9395 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:08.471024  9115 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:08.485801  1627 raft_consensus.cc:2468] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:08.494445  1630 raft_consensus.cc:2468] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:08.496062  1630 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:08.479171  9215 raft_consensus.cc:2804] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:08.508580  2318 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:08.441296  9370 consensus_queue.cc:1059] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:03:08.401356  9426 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.445147  9379 consensus_queue.cc:1048] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:03:08.498271  9350 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:08.537822  9350 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:08.538292  9350 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:08.538930  2350 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:08.516950  2353 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:03:08.523000  9321 consensus_queue.cc:1059] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:03:08.523367  9491 log.cc:927] Time spent T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
W20260709 13:03:08.524381  9512 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:03:08.419643  9499 consensus_queue.cc:237] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.526252  2345 raft_consensus.cc:3055] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:08.551224  2345 raft_consensus.cc:740] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:08.529412  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:08.530537  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:08.552695  2349 raft_consensus.cc:2393] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:08.533275  1703 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
W20260709 13:03:08.534444  9504 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:03:08.521961  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:08.449083  9234 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.063s
W20260709 13:03:08.568949  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 226562us
W20260709 13:03:08.566008  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 81828.7us
I20260709 13:03:08.595561  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:08.597050  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:08.597920  2350 raft_consensus.cc:2368] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19.
I20260709 13:03:08.592188  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:08.606074  9294 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:08.558971  9438 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.569s	user 0.014s	sys 0.000s
I20260709 13:03:08.613937  9438 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:08.615141  9289 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:08.617092  9438 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:08.512472  9324 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 13:03:08.550593  9517 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:08.594100  9252 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:08.629560  9503 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 13:03:08.649886  1709 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:03:08.630118  9495 raft_consensus.cc:3055] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:08.630992  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:08.519207  9215 raft_consensus.cc:697] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:08.633944  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:08.588373  2345 consensus_queue.cc:260] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.643376  2353 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 21. Current term is 22. Ops: [21.18-21.18]
W20260709 13:03:08.636106  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 60169.9us
I20260709 13:03:08.671161  1631 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 13:03:08.672425  2345 raft_consensus.cc:3060] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:08.533305  9518 consensus_queue.cc:1048] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.114s
W20260709 13:03:08.681953  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 182451us
I20260709 13:03:08.686060  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:08.688139  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.688643  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.689577  1628 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:03:08.652493  9495 raft_consensus.cc:740] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:08.682749  9329 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.707672  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:08.705152  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:08.685398  9321 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.716794  9321 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:08.719913  9321 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:08.710950  9215 consensus_queue.cc:237] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:08.725210  1601 connection.cc:256] RPC call timeout handler was delayed by -2.8168186339999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
I20260709 13:03:08.702924  9252 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.714165  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:08.714373  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:08.715900  9374 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.600s	user 0.014s	sys 0.000s
I20260709 13:03:08.709340  2345 raft_consensus.cc:1275] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:08.722011  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:08.712271  9329 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:08.725739  1601 connection.cc:256] RPC call timeout handler was delayed by -2.6116186339999672s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
I20260709 13:03:08.783213  1555 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:03:08.742301  2345 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:08.746994  2348 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:03:08.751842  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.751377  1632 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:08.753247  1703 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:08.740906  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:08.799584  1559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3540) took 1197 ms. Trace:
I20260709 13:03:08.810385  1559 rpcz_store.cc:276] 0709 13:03:07.574620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.604326 (+ 29706us) service_pool.cc:224] Handling call
0709 13:03:08.771582 (+1167256us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b540038a9d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:08.771720 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":871089536}
I20260709 13:03:08.811348  9294 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:03:08.808481  1601 connection.cc:256] RPC call timeout handler was delayed by -2.263814324000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3024s for a response
I20260709 13:03:08.812722  9520 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:08.742293  2344 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:08.679838  9511 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:08.813272  1601 connection.cc:256] RPC call timeout handler was delayed by -1.9510186339999791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28550000000000003s for a response
I20260709 13:03:08.813393  9520 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:08.814332  2350 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 13:03:08.813848  1601 connection.cc:256] RPC call timeout handler was delayed by -1.6972186339999666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31370000000000003s for a response
W20260709 13:03:08.815526  1601 connection.cc:256] RPC call timeout handler was delayed by -1.5334945249999805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
W20260709 13:03:08.815877  1601 connection.cc:256] RPC call timeout handler was delayed by -1.5172945250000112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
I20260709 13:03:08.814853  9520 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:08.816587  9511 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:08.816799  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:08.742136  9471 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.000s
I20260709 13:03:08.817663  9471 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:08.820497  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.821065  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:08.821031  1601 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:08.822332  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:08.826432  9115 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:08.826557  9294 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:08.827154  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:08.828116  9115 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:08.818467  9520 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:08.833381  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:08.837194  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.838479  1627 raft_consensus.cc:2393] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:08.705365  9438 consensus_queue.cc:1048] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 13:03:08.767724  9220 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:08.755906  9298 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:08.744788  9329 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:08.852330  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.881707  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48938.1us
I20260709 13:03:08.736878  9358 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.095s
W20260709 13:03:08.886092  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 57272us
I20260709 13:03:08.888741  2347 raft_consensus.cc:1217] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 LEADER]: Deduplicated request from leader. Original: 21.17->[23.18-25.19]   Dedup: 23.18->[25.19-25.19]
I20260709 13:03:08.934414  2347 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 25. Current term is 26. Ops: [23.18-25.19]
I20260709 13:03:08.737215  9476 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.110s
I20260709 13:03:08.737336  9289 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 13:03:08.749058  9252 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:08.842371  1628 raft_consensus.cc:2393] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:08.891345  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.894611  1629 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:03:08.937606  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:08.939657  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:08.805888  9428 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.745975  9495 consensus_queue.cc:260] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:08.897217  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 173595us
I20260709 13:03:08.898723  1701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3425) took 1027 ms. Trace:
I20260709 13:03:08.945528  1627 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:08.946071  1627 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:08.881767  9115 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:08.739995  9500 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.912818  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:08.745020  9519 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.726619  9528 consensus_queue.cc:1048] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.187s
W20260709 13:03:08.950136  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.944818  1701 rpcz_store.cc:276] 0709 13:03:07.871382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.871564 (+   182us) service_pool.cc:224] Handling call
0709 13:03:07.871964 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.084005 (+212041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.898249 (+814244us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54002e90d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:08.898631 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323871616}
I20260709 13:03:08.957106  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:08.943969  9350 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:08.959611  9443 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.960059  2347 raft_consensus.cc:1240] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:03:08.960157  9443 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:08.960940  9350 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:08.947129  1627 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:08.960757  9220 consensus_queue.cc:1059] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:08.961381  2346 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:08.962064  1627 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:08.942741  9329 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:08.949741  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 268751us
I20260709 13:03:08.994061  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:09.013607  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56596.2us
I20260709 13:03:08.954005  9498 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:08.991783  1525 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:09.006799  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:08.967487  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 265394us
W20260709 13:03:09.000152  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 51064.7us
I20260709 13:03:09.019197  2347 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:03:09.024488  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.996363  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.016019  1701 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:09.034591  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:09.000205  9507 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:08.997983  9220 raft_consensus.cc:3055] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:09.041388  2352 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:09.043960  9500 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:09.044760  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3541) took 1469 ms. Trace:
I20260709 13:03:08.945878  9428 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:09.046151  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.965268  9446 consensus_queue.cc:1059] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.081s
I20260709 13:03:09.044926  9329 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:03:09.048986  9547 consensus_queue.cc:1059] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:09.013837  9115 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:09.051147  9289 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:09.052052  1675 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:09.054536  1627 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
W20260709 13:03:09.057246  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 65583.7us
I20260709 13:03:09.058468  1561 rpcz_store.cc:276] 0709 13:03:07.575009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.575243 (+   234us) service_pool.cc:224] Handling call
0709 13:03:09.044560 (+1469317us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b5400312754. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:09.044690 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937110400}
I20260709 13:03:08.964601  9519 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:09.098829  2319 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.107407  1701 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 18. (term mismatch)
I20260709 13:03:09.107985  1701 pending_rounds.cc:85] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 17
I20260709 13:03:09.056715  9220 raft_consensus.cc:740] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:03:09.109624  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 58020.2us
I20260709 13:03:09.110420  1627 pending_rounds.cc:85] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 9
I20260709 13:03:09.065989  9492 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:09.112056  9443 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:09.112593  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 84855.6us
I20260709 13:03:09.113435  2318 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.114158  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:09.114899  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:09.116191  9538 log.cc:927] Time spent T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:03:09.117705  9541 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:09.118199  9541 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.118567  9541 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:09.129031  9288 raft_consensus.cc:3055] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:09.129628  9288 raft_consensus.cc:740] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:09.130864  9548 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:09.131337  9548 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:09.131673  9548 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:03:09.135023  9544 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:03:09.141811  2351 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:03:09.108314  1701 pending_rounds.cc:107] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 21.18
I20260709 13:03:09.143396  1701 raft_consensus.cc:2889] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.111469  1627 pending_rounds.cc:107] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 20.10
W20260709 13:03:09.147576  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 207970us
I20260709 13:03:09.144181  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:09.148593  1627 raft_consensus.cc:2889] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.105047  9411 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:09.151780  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 124982us
I20260709 13:03:09.110973  9446 consensus_queue.cc:1243] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer 95fc6a4cf5874b7eb91280dffcd3b3fb log is divergent from this leader: its last log entry 26.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
W20260709 13:03:09.152845  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44001.6us
I20260709 13:03:09.110354  9289 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:09.154255  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 131333us
I20260709 13:03:09.154497  1602 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:09.153365  2352 raft_consensus.cc:2955] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 5 to 15, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:09.147643  9492 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:09.150646  9494 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.739s	user 0.009s	sys 0.005s
I20260709 13:03:09.151058  9322 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:09.106244  9546 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.090s
I20260709 13:03:09.158607  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.149537  9296 consensus_queue.cc:1059] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 21.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:03:09.160737  9322 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.167204  9215 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:09.167639  9322 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:09.168604  9296 raft_consensus.cc:3055] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:09.168787  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:09.169878  9436 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:09.170359  9436 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:09.171489  9436 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.165956  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.165504  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.155553  9289 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:09.169044  9296 raft_consensus.cc:740] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:09.192394  9296 consensus_queue.cc:260] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.9, Last appended by leader: 9, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:09.193892  9296 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:09.172099  9329 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:09.195710  1704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3426) took 1322 ms. Trace:
I20260709 13:03:09.196151  1704 rpcz_store.cc:276] 0709 13:03:07.872892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.873050 (+   158us) service_pool.cc:224] Handling call
0709 13:03:07.873495 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.153649 (+280154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.195395 (+1041746us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b5400224f54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:09.195616 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629250304}
I20260709 13:03:09.162861  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:09.200203  9379 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:09.200824  9379 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:09.163534  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.155650  9547 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.149833  9443 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.011s
I20260709 13:03:09.178268  9495 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:09.185627  1602 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:09.191001  9551 consensus_queue.cc:1059] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:09.209270  1676 leader_election.cc:400] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 18. Current term is 19.
W20260709 13:03:09.219274  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 53410.9us
I20260709 13:03:09.166778  9115 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:03:09.222476  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 57951.3us
I20260709 13:03:09.208256  9546 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.159477  9298 consensus_queue.cc:1059] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 13:03:09.215288  9528 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:09.214763  9492 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:09.248252  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.161406  9494 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:09.256522  9546 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.257405  9494 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:09.201911  9379 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.259265  9546 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:09.263710  9492 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:09.263813  9541 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.265141  2318 leader_election.cc:400] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 18. Current term is 19.
I20260709 13:03:09.215651  9220 consensus_queue.cc:260] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:09.242913  9552 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:09.257160  1705 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:09.266381  9220 raft_consensus.cc:3060] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:09.266541  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.267555  9492 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:09.268131  9492 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:09.269143  9492 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.244161  9507 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:09.211454  9547 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:09.217360  9446 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.059s
I20260709 13:03:09.279886  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:09.280335  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 71169.7us
I20260709 13:03:09.288046  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.261210  9546 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:03:09.301468  9322 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:09.309156  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:09.309945  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:09.310318  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:09.316931  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 146506us
W20260709 13:03:09.319693  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.278919  9548 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.317319  9426 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:09.339020  9548 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:09.341609  9322 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:09.325353  9541 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:09.348018  9289 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:09.351272  1554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 3562) took 1395 ms (client timeout 1556 ms). Trace:
W20260709 13:03:09.351859  1554 rpcz_store.cc:269] 0709 13:03:07.956001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.961791 (+  5790us) service_pool.cc:224] Handling call
0709 13:03:09.350976 (+1389185us) spinlock_profiling.cc:243] Waited 88.6 ms on lock 0x7b5400389ad4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:09.351183 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203739392}
I20260709 13:03:09.354537  9289 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:09.355799  1636 raft_consensus.cc:1240] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:09.358402  1708 raft_consensus.cc:1240] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 LEADER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:09.361197  9547 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:09.322319  9329 raft_consensus.cc:3055] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:09.364302  9329 raft_consensus.cc:740] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:09.365338  9329 consensus_queue.cc:260] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:09.366441  9329 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:09.386009  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 53422.1us
W20260709 13:03:09.391227  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.394501  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3689) took 1226 ms. Trace:
I20260709 13:03:09.395323  1633 rpcz_store.cc:276] 0709 13:03:08.167511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.202238 (+ 34727us) service_pool.cc:224] Handling call
0709 13:03:09.394033 (+1191795us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54002da5d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:09.394346 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362945280}
I20260709 13:03:09.396574  1601 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:09.401533  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:09.402383  2352 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:09.369186  9288 consensus_queue.cc:260] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.405418  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3655) took 2482 ms. Trace:
I20260709 13:03:09.407125  1635 rpcz_store.cc:276] 0709 13:03:06.923300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.168950 (+245650us) service_pool.cc:224] Handling call
0709 13:03:07.207591 (+ 38641us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:07.215911 (+  8320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.215951 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:07.216726 (+   775us) log.cc:844] Serialized 52 byte log entry
0709 13:03:07.474902 (+258176us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:07.474963 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.475055 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:08.050969 (+575914us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:03:08.230502 (+179533us) raft_consensus.cc:1685] finished
0709 13:03:08.230537 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.405187 (+1174650us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b540010c7d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:09.405331 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":142,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"spinlock_wait_cycles":113382016,"thread_start_us":257655,"threads_started":1,"wal-append.queue_time_us":497426,"wal-append.run_cpu_time_us":11608,"wal-append.run_wall_time_us":1539595}
I20260709 13:03:09.414602  9508 raft_consensus.cc:3055] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:09.415328  9508 raft_consensus.cc:740] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:09.418445  9404 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:09.419240  9404 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:09.438138  9288 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:09.441103  1555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 3561) took 1485 ms. Trace:
I20260709 13:03:09.441754  1555 rpcz_store.cc:276] 0709 13:03:07.955687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.961569 (+  5882us) service_pool.cc:224] Handling call
0709 13:03:08.771979 (+810410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.807622 (+ 35643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.440804 (+633182us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b540038a9d0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:09.441018 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1787469568}
I20260709 13:03:09.348474  9476 raft_consensus.cc:2955] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Committing config change with OpId 19.15: config changed from index 5 to 15, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:09.457363  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:09.459498  1704 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:09.466044  9428 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:09.470757  9425 raft_consensus.cc:3055] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:03:09.471459  9425 raft_consensus.cc:740] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:09.420610  9404 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.476504  9404 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:09.416519  9508 consensus_queue.cc:260] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.472602  9425 consensus_queue.cc:260] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.479360  9425 raft_consensus.cc:3060] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:03:09.481137  9539 log.cc:927] Time spent T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.570s	user 0.002s	sys 0.000s
I20260709 13:03:09.483631  9411 raft_consensus.cc:2804] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:09.491276  9411 raft_consensus.cc:697] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:09.491500  9520 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:09.492484  9520 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:09.492697  9411 consensus_queue.cc:237] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.493598  9220 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:09.494223  9220 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:09.495075  9531 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.495720  9531 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.495424  9220 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:09.486791  9385 consensus_queue.cc:260] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.499641  9385 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:09.501827  9531 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:09.512809  1453 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 17 to 20, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 20 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:09.488679  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:09.515128  9436 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:09.514081  9220 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:09.478560  9508 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:09.515800  1453 catalog_manager.cc:5697] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 10 to 19, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 19 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:09.516295  9519 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:09.526923  9507 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:09.549743  9507 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:09.550284  9507 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:09.527889  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 239797us
I20260709 13:03:09.559252  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.560670  9234 raft_consensus.cc:2764] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:03:09.568276  9552 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:09.569660  9552 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.573066  1453 catalog_manager.cc:5697] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 13 to 14, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:09.597369  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:09.597957  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:09.597368  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 372016us
I20260709 13:03:09.616737  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:03:09.622431  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.622722  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:09.625118  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:09.626971  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 463199us
I20260709 13:03:09.627841  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:09.629449  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:09.595613  9407 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:03:09.631825  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:09.632421  2350 raft_consensus.cc:1275] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:03:09.632936  1707 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:09.636485  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:09.635960  1453 catalog_manager.cc:5697] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 8 to 14, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:09.639830  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:09.641342  2353 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:09.629446  9329 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 13:03:09.643008  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:09.645318  9507 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.646807  2353 raft_consensus.cc:1240] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:09.650192  9500 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:09.650417  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:09.651525  9500 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:09.651783  9296 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:09.652344  9296 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:09.636613  9518 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.657094  1635 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20. Ops: []
W20260709 13:03:09.580243  1454 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 273) took 64329 ms (client timeout 15000 ms). Trace:
I20260709 13:03:09.655555  1634 raft_consensus.cc:1275] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:09.659034  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:09.659421  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.655844  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:09.629899  9099 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:09.652428  9500 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:09.658238  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:09.657094  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:09.661386  2318 leader_election.cc:336] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.661909  1707 raft_consensus.cc:2410] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:09.669782  2345 raft_consensus.cc:1217] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 LEADER]: Deduplicated request from leader. Original: 21.17->[23.18-25.19]   Dedup: 23.18->[25.19-25.19]
I20260709 13:03:09.670538  2345 raft_consensus.cc:1240] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 25. Current term is 26. Ops: [23.18-25.19]
I20260709 13:03:09.670598  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:03:09.672204  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.674566  1634 raft_consensus.cc:1275] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:03:09.595263  9519 consensus_queue.cc:1048] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:03:09.688110  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:09.689435  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.705091  1634 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:09.707257  1627 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:09.710253  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:09.713219  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:09.714586  1633 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:09.713207  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:09.716511  2350 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:03:09.718571  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 48035.5us
I20260709 13:03:09.727617  1525 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:09.720405  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 331105us
I20260709 13:03:09.729633  1705 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:09.715405  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:09.734956  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:09.737149  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.737787  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:03:09.738561  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:09.739846  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.673446  9425 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:09.672793  9099 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.653854  9296 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.662833  9500 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:09.703881  9548 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:09.679765  9115 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:09.743263  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:09.758199  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 96788.2us
W20260709 13:03:09.759163  2318 leader_election.cc:336] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:09.760398  2318 leader_election.cc:336] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.704474  9507 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:09.766405  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.766206  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:09.737243  9427 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.736934  9520 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:09.764855  1710 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:03:09.764963  1631 raft_consensus.cc:1240] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:09.663049  9518 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.662708  9436 raft_consensus.cc:2804] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:09.739799  9407 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.774418  1674 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:09.784417  1634 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:03:09.749593  9099 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:09.753194  9497 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:09.790861  1674 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:09.796402  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.770275  9443 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:09.773393  9499 consensus_queue.cc:1048] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:03:09.800933  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2696) took 1111 ms. Trace:
I20260709 13:03:09.804847  1710 rpcz_store.cc:276] 0709 13:03:08.689385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.689899 (+   514us) service_pool.cc:224] Handling call
0709 13:03:09.800705 (+1110806us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400077e54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:09.800854 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2426183424}
W20260709 13:03:09.815527  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 53519.2us
I20260709 13:03:09.819881  1706 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:09.820760  1633 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:09.786625  9438 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.785202  9425 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:09.827541  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:09.822597  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:09.764091  9517 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:03:09.793790  9436 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.843300  9436 raft_consensus.cc:3060] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:09.793591  9507 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:09.847942  9507 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:09.848460  9507 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:09.832522  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:09.849610  1635 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:03:09.770774  9548 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.816730  9385 raft_consensus.cc:2804] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:09.803439  9324 raft_consensus.cc:2804] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:09.816259  1630 pending_rounds.cc:85] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 7
I20260709 13:03:09.868891  1630 pending_rounds.cc:107] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:03:09.869189  1630 raft_consensus.cc:2889] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.803233  9443 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.836104  9438 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.835897  9500 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.774003  9234 consensus_queue.cc:1048] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 13:03:09.841979  9115 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.773578  9492 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 13:03:09.860352  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:09.833613  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51501.6us
W20260709 13:03:09.835024  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 74609.2us
W20260709 13:03:09.882887  2318 leader_election.cc:336] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.863554  9518 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:09.864750  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.866349  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.824608  9520 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:09.870210  9426 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:09.870970  9517 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:09.803046  9407 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:09.672423  1453 catalog_manager.cc:5697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 10 to 14, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:09.799523  9427 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:09.892639  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.871241  9443 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:09.858392  9519 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 13:03:09.905963  9561 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:09.889674  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:09.914808  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:09.888299  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:09.876139  9385 raft_consensus.cc:697] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:09.886811  9557 raft_consensus.cc:2804] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:09.922878  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.905347  9548 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:09.951427  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 85156.1us
I20260709 13:03:09.910192  9517 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:09.942029  1635 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:09.909364  9324 raft_consensus.cc:697] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:03:09.955802  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 67593.1us
W20260709 13:03:09.942476  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 56522.2us
I20260709 13:03:09.888089  9558 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:09.964424  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:09.947865  9500 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:09.958344  9569 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:09.970422  9569 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:09.970883  9569 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:09.971498  9324 consensus_queue.cc:237] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:09.950582  9557 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.975787  9557 raft_consensus.cc:3060] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:09.940609  9561 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:09.977770  9561 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:09.988935  1635 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:09.991742  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:09.992725  1633 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:09.995510  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:09.958544  9546 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:09.953512  9426 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:09.959118  9558 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:10.009685  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 205164us
I20260709 13:03:09.992425  9508 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.014997  1629 raft_consensus.cc:1275] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:03:10.014268  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:10.022940  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 206296us
W20260709 13:03:10.009289  9476 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.002s
I20260709 13:03:10.009264  9558 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:10.027554  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:10.035336  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.028365  1635 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:03:10.014225  9288 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:10.045286  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 161268us
I20260709 13:03:10.016162  9220 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.052711  9436 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.053131  9518 consensus_queue.cc:1048] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:03:10.059983  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:10.061192  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:10.024292  9546 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:10.065276  1629 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:10.072466  1633 raft_consensus.cc:2410] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
W20260709 13:03:10.077913  1675 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:10.080770  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.085237  2318 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:10.015244  9562 consensus_queue.cc:1059] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:03:10.060964  9548 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:10.077736  9574 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:03:10.025147  9508 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.107352  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:10.100409  9425 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:03:10.011463  9517 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:10.110093  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.108613  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 74602us
W20260709 13:03:10.115192  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.084859  9499 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:10.117865  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66655.5us
I20260709 13:03:10.005450  9385 consensus_queue.cc:237] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.134192  9428 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.134872  9428 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:10.122958  9404 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.646s	user 0.008s	sys 0.006s
I20260709 13:03:10.125953  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:10.167232  1629 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:10.075966  9492 consensus_queue.cc:1059] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:03:10.094305  9370 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:03:10.138949  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.170089  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 213271us
W20260709 13:03:10.170887  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.143121  9436 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.146960  9569 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.151803  9508 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:10.159824  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:10.097034  9546 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:10.058097  9426 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.105650  9548 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.164985  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:10.066000  9220 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:09.661906  1454 rpcz_store.cc:269] 0709 13:02:05.250351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.252287 (+  1936us) service_pool.cc:224] Handling call
0709 13:03:09.579848 (+64327561us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:51.117216 (+     0us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda245d00ac 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda2b10f16c
0709 13:02:51.490375 (+373159us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:51.511337 (+ 20962us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:51.511411 (+    74us) write_op.cc:435] Acquired schema lock
0709 13:02:51.511438 (+    27us) tablet.cc:662] Decoding operations
0709 13:02:51.714939 (+203501us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:51.747789 (+ 32850us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:51.796601 (+ 48812us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:02:52.174589 (+377988us) tablet.cc:701] Row locks acquired
0709 13:02:52.185214 (+ 10625us) write_op.cc:260] PREPARE: finished
0709 13:02:52.236332 (+ 51118us) write_op.cc:270] Start()
0709 13:02:52.267879 (+ 31547us) write_op.cc:276] Timestamp: P: 1783602172206588 usec, L: 0
0709 13:02:52.267930 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:02:52.268821 (+   891us) log.cc:844] Serialized 5629 byte log entry
0709 13:02:53.259793 (+990972us) op_driver.cc:464] REPLICATION: finished
0709 13:02:53.925992 (+666199us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda245d00ac 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2b110ef0
0709 13:02:54.239369 (+313377us) write_op.cc:301] APPLY: starting
0709 13:02:54.255778 (+ 16409us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:54.788208 (+532430us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b18002a6760. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda274190ec 00007fda2741baf7 00007fda2741cfb0 00007fda2b1311bb 00007fda2b131532 00007fda2b178b93 00007fda2b025de3 00007fda2b0256ab 00007fda2b026e77
0709 13:02:55.153391 (+365183us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:55.153435 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:56.121290 (+967855us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:56.261895 (+140605us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=0,mrs_lookups=0
0709 13:02:56.267892 (+  5997us) write_op.cc:312] APPLY: finished
0709 13:02:56.603767 (+335875us) log.cc:844] Serialized 129 byte log entry
0709 13:02:56.740912 (+137145us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:56.898395 (+157483us) write_op.cc:454] Released schema lock
0709 13:02:56.898525 (+   130us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:56.899339 (+   814us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:56.899369 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":188966,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":910500,"apply.run_cpu_time_us":79663,"apply.run_wall_time_us":2685505,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62647,"num_ops":14,"prepare.queue_time_us":1879056,"prepare.run_cpu_time_us":18017,"prepare.run_wall_time_us":996918,"raft.queue_time_us":108709,"raft.run_cpu_time_us":36248,"raft.run_wall_time_us":836622,"replication_time_us":991609,"spinlock_wait_cycles":35572224,"thread_start_us":3109034,"threads_started":5,"wal-append.queue_time_us":482017,"wal-append.run_cpu_time_us":69053,"wal-append.run_wall_time_us":2771707}]]}
I20260709 13:03:10.175000  2346 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:10.187359  1631 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:10.062705  9288 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:10.184721  9566 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.239s	user 0.000s	sys 0.004s
I20260709 13:03:10.215242  2348 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 24. Current term is 25. Ops: []
W20260709 13:03:10.227419  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 62542.9us
I20260709 13:03:10.203504  9517 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:10.203868  9427 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.118371  9578 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:03:10.241820  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:10.254642  2317 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:10.260051  2347 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:03:10.228770  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.248306  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.248785  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 64353.4us
I20260709 13:03:10.140990  9099 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.209041  9436 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:10.251377  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.254693  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 152765us
I20260709 13:03:10.255503  1633 raft_consensus.cc:1275] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:03:10.263363  1674 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:10.265165  2351 raft_consensus.cc:1275] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:10.274574  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:10.275280  2351 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:03:10.282449  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:10.294083  9594 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.111s	user 0.001s	sys 0.001s
I20260709 13:03:10.270423  9587 raft_consensus.cc:3060] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:10.265789  9234 consensus_queue.cc:1059] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:03:10.259130  9561 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.278246  9379 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:03:10.347360  9552 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.351305  9552 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:10.330106  9576 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
W20260709 13:03:10.331585  9583 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:03:10.333703  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.336185  9590 consensus_queue.cc:1048] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:10.282621  9436 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.286738  9427 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.053s
W20260709 13:03:10.346074  1601 connection.cc:256] RPC call timeout handler was delayed by -1.4288565929999777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
W20260709 13:03:10.370972  1601 connection.cc:256] RPC call timeout handler was delayed by -1.3937565929999778s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
I20260709 13:03:10.285770  9492 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 13:03:10.327441  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:10.324980  9329 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:10.353952  9370 consensus_queue.cc:1048] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.023s
I20260709 13:03:10.281931  9508 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:03:10.408553  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 81151.4us
I20260709 13:03:10.409889  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:10.371790  1601 connection.cc:256] RPC call timeout handler was delayed by -1.3847565929999632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
W20260709 13:03:10.413252  1601 connection.cc:256] RPC call timeout handler was delayed by -1.3082565929999532s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
I20260709 13:03:10.297006  9553 consensus_queue.cc:1048] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 13:03:10.293694  9565 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:03:10.355911  9407 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.361497  9557 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.412025  1674 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:10.374967  9579 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:10.380160  9220 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:10.441089  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.424520  1601 connection.cc:256] RPC call timeout handler was delayed by -1.2848565929999722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
W20260709 13:03:10.448195  1601 connection.cc:256] RPC call timeout handler was delayed by -1.256956592999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 13:03:10.450521  1601 connection.cc:256] RPC call timeout handler was delayed by -1.2515565929999752s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
I20260709 13:03:10.422835  9552 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.427516  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:10.296670  9518 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:10.389499  9234 raft_consensus.cc:3055] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:10.417995  9495 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:10.417637  9379 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:10.451238  1601 connection.cc:256] RPC call timeout handler was delayed by -1.1390565929999639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1962s for a response
W20260709 13:03:10.463800  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0553565929999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20550000000000002s for a response
I20260709 13:03:10.429585  9551 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.296967  9562 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:10.491683  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:10.414575  9329 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.448438  9115 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:10.487303  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:10.500092  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:10.463174  9379 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.527702  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:10.561901  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 312722us
I20260709 13:03:10.564391  1525 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:10.463342  9495 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.459057  9234 raft_consensus.cc:740] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:03:10.503504  1675 leader_election.cc:336] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:10.581146  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 81697.6us
W20260709 13:03:10.582072  1675 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:10.459313  9518 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.528074  9288 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.502786  9569 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.507539  9562 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.595920  9562 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:10.537863  9115 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:10.552687  9551 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:10.558527  9598 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
W20260709 13:03:10.506228  9593 log.cc:927] Time spent T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:03:10.498477  9590 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:03:10.640699  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 228635us
I20260709 13:03:10.569921  9546 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:10.573252  9557 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:10.527899  9577 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 13:03:10.573729  9585 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.286s	user 0.001s	sys 0.001s
I20260709 13:03:10.577749  9329 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:10.582603  1675 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:10.583993  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:10.538367  9507 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.689s	user 0.014s	sys 0.002s
I20260709 13:03:10.581230  9495 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:10.593930  9552 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:10.450762  9579 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.597263  9115 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.570497  9379 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:10.583953  9518 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:10.635529  9443 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.691s	user 0.010s	sys 0.004s
I20260709 13:03:10.621259  9234 consensus_queue.cc:260] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.636096  9561 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:10.560792  9548 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:10.583392  9385 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.621047  9436 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.570510  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:10.720363  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 78542.8us
I20260709 13:03:10.652490  9551 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:10.655184  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.732400  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.643467  9590 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:10.696055  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 113942us
W20260709 13:03:10.735005  1675 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:10.735841  1675 leader_election.cc:336] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:10.736545  1675 leader_election.cc:336] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:10.737200  1675 leader_election.cc:336] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:10.662945  9608 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.654161  2351 raft_consensus.cc:2468] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
W20260709 13:03:10.658488  1599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.642305  9558 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.004s
I20260709 13:03:10.739765  9558 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.742690  9558 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.675021  9099 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.689055  9610 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.680907  9497 raft_consensus.cc:3055] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:10.684850  9587 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:10.641708  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 154465us
I20260709 13:03:10.744328  9497 raft_consensus.cc:740] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:10.745873  9234 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:10.747117  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:10.749158  9561 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:10.748617  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:10.750077  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 179544us
I20260709 13:03:10.737954  9590 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:10.750662  9561 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:10.751076  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:10.753147  9546 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:10.754154  9546 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:10.754729  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:10.744381  9610 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.757062  9610 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:10.718919  9499 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.721665  9443 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:10.758548  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:10.758131  9499 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.764065  9608 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.766601  9499 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:10.734619  9385 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:10.771616  1675 leader_election.cc:336] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:10.759420  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:10.760057  9443 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:10.705219  9507 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.762887  9579 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.772387  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:10.787089  9497 consensus_queue.cc:260] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.788878  9289 raft_consensus.cc:3055] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:10.798802  9289 raft_consensus.cc:740] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:10.800153  9289 consensus_queue.cc:260] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.803714  9099 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:10.779330  9541 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.804948  9541 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.781616  9436 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:03:10.770489  9385 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.807355  9541 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:10.766610  9608 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:10.808367  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 49029.6us
W20260709 13:03:10.791080  1675 leader_election.cc:336] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:10.809176  1675 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:10.810367  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:10.753017  9562 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.811844  9385 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:10.813122  9428 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:10.813367  9562 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.779330  9619 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:10.787348  9551 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:10.816169  9507 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:10.816983  9438 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.817904  9379 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.821319  1708 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:10.822273  9438 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:10.822521  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.823290  9619 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.823789  9619 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.824560  1710 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:03:10.826247  1706 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:10.829309  2345 raft_consensus.cc:1240] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:10.801273  9289 raft_consensus.cc:3060] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:10.760111  9610 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:10.837256  9443 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:10.822867  9438 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.837975  9443 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.838218  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:10.838814  9609 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:10.839488  9609 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.840440  9438 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.840701  9443 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.826259  9619 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:10.834637  9546 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.834722  9587 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:10.841672  9426 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.841971  9587 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.842175  9518 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.842293  9426 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.843848  9609 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.844898  9587 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.836615  9436 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:10.845360  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:10.846481  1710 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:10.845949  9436 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.847400  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1570) took 1111 ms. Trace:
I20260709 13:03:10.848685  9436 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:10.850147  9379 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:10.811087  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:10.841704  9546 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:10.851900  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:10.852552  1675 consensus_peers.cc:597] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:10.852773  9546 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:10.844784  9426 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:10.858084  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:10.844956  9518 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:10.847903  2353 rpcz_store.cc:276] 0709 13:03:09.736051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.737037 (+   986us) service_pool.cc:224] Handling call
0709 13:03:09.737611 (+   574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.875826 (+138215us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:09.891418 (+ 15592us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.167963 (+276545us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.168035 (+    72us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:10.168099 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.847161 (+679062us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b540005bc54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:10.847211 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.847339 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10510,"prepare.queue_time_us":15793,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":146,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":13881,"spinlock_wait_cycles":317055872,"thread_start_us":276052,"threads_started":1,"wal-append.queue_time_us":355914}
I20260709 13:03:10.841358  9443 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.856701  9561 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:10.819917  9495 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.860010  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3703) took 1047 ms. Trace:
I20260709 13:03:10.873903  1706 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:03:10.871093  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 140044us
I20260709 13:03:10.877254  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:10.879952  1708 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:10.881210  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:10.881618  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:10.883453  1705 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:10.877998  1628 rpcz_store.cc:276] 0709 13:03:09.812722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.813575 (+   853us) service_pool.cc:224] Handling call
0709 13:03:09.832601 (+ 19026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.837182 (+  4581us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:10.650187 (+813005us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.859356 (+209169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.859438 (+    82us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:10.859522 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.859630 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.859911 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":501505,"prepare.queue_time_us":567054,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":231,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":6012,"spinlock_wait_cycles":70784,"thread_start_us":504424,"threads_started":2}
W20260709 13:03:10.886126  1675 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:10.887054  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:10.888128  1707 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:10.887662  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:10.885659  9438 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:10.882797  9443 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.899052  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:10.900032  1706 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
W20260709 13:03:10.892357  1675 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:10.896493  9438 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.902818  1632 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:10.903023  9590 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.893963  9552 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.904065  9552 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:10.904734  9438 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:10.914284  1627 raft_consensus.cc:1240] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:10.923388  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:10.925333  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:10.893450  9608 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:10.928005  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:10.930936  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:10.936347  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.931978  1636 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:03:10.905267  9590 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:10.938378  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:10.940557  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:10.944453  1706 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
W20260709 13:03:10.944868  1675 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:10.950282  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:10.905319  9495 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:10.951387  1675 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:10.953166  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:10.956990  9507 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.957684  9507 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.921572  9234 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:10.967609  9234 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:10.951377  1631 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:10.974888  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:10.952319  1675 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:11.002059  1706 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:03:11.007184  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 55882.9us
W20260709 13:03:11.008431  1675 leader_election.cc:336] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:11.008985  1675 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:11.021229  1675 leader_election.cc:336] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:11.022013  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:11.027696  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:11.028841  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:11.030222  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.031036  1633 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
W20260709 13:03:11.034515  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.030447  1675 leader_election.cc:336] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:11.036512  1675 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:11.037304  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:11.037942  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:11.006551  9608 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.012773  9426 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.028254  9476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.013s	sys 0.020s
I20260709 13:03:11.005918  9495 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:03:11.035058  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 148922us
I20260709 13:03:11.025348  9590 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:11.030076  9438 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:11.034224  9579 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:11.048255  1706 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:11.033897  9546 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.049391  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:11.054195  1601 leader_election.cc:336] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:11.038903  9507 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:11.042622  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:11.026309  9379 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:03:11.058993  1629 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:11.046731  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:11.060839  1635 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:11.067453  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.069306  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.042699  9099 consensus_queue.cc:1059] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
W20260709 13:03:11.047037  9476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.013s	sys 0.020s
W20260709 13:03:11.102932  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 48734.2us
W20260709 13:03:11.122126  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.048972  9495 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.149811  1706 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 13:03:11.151698  1601 leader_election.cc:336] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:11.152428  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:11.153630  9567 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:11.045745  9608 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.067252  9579 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:03:11.164223  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.175454  9628 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:03:11.183691  9567 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:03:11.190012  9625 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:03:11.047685  9426 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.083937  9234 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:11.093097  9552 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:11.220662  1635 raft_consensus.cc:2468] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
W20260709 13:03:11.222623  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 186583us
I20260709 13:03:11.139676  9587 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.230389  1674 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:11.060184  9546 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.233070  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.241616  1629 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:03:11.240947  9587 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.247069  9608 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:11.248260  9546 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:11.251025  9426 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:11.253541  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:11.254544  1710 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:11.254424  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.254714  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:11.255358  1701 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:03:11.255744  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:11.256527  1705 raft_consensus.cc:2393] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:11.236728  9443 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:11.257398  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.257820  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.257905  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.255383  1707 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:11.259871  9495 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:11.260226  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.260857  1709 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 26.
I20260709 13:03:11.263800  1708 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:03:11.269084  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:11.275406  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:11.277153  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 164832us
W20260709 13:03:11.279193  1601 leader_election.cc:336] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:11.279642  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:11.282300  9629 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:11.283195  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:11.289752  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.289563  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:11.307407  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.309005  1710 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 18: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:11.311378  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:11.314424  1601 leader_election.cc:336] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:11.315267  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:11.317464  1601 leader_election.cc:336] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:11.318141  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:11.245396  9587 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:11.223172  9579 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.226778  9567 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:11.325264  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.286742  9379 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:11.330164  1707 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
W20260709 13:03:11.351575  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:11.314105  9551 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:11.354182  1526 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:11.352684  1601 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:11.291652  9546 consensus_queue.cc:1059] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:03:11.322621  9552 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:11.330104  9443 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:11.323684  9562 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:03:11.359027  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:11.346751  9545 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:11.337711  9379 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.338891  9411 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:11.326603  9520 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.377040  9379 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:11.376739  9520 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.384580  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.384526  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.390303  1710 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:11.376940  9115 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:11.390167  9552 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:11.400652  9552 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.394394  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:11.402773  9589 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:11.402783  9552 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:11.403438  9589 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.403910  9589 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:11.393177  1708 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:11.399160  1709 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:03:11.369587  9587 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:11.370069  9562 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.372888  9426 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:11.372486  9559 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:11.370420  9443 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.411319  9552 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:11.379725  9520 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:11.439870  9554 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.443135  9554 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.443706  9296 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:11.440209  9411 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:11.445405  9411 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.445665  9552 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.448098  9552 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:11.448282  9597 log.cc:927] Time spent T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.016s	user 0.003s	sys 0.019s
I20260709 13:03:11.448401  9411 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:11.448755  9552 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:11.439821  9425 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.450179  9425 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:11.452533  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:11.452941  9425 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:11.453099  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 1.29081e+06us
I20260709 13:03:11.454381  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.455057  1709 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 24.
I20260709 13:03:11.455227  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:11.456074  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:11.456750  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.457043  2351 raft_consensus.cc:2410] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:03:11.457490  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:11.458024  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.458779  2351 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:11.459086  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.426823  9579 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:11.460153  2344 raft_consensus.cc:2410] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:03:11.418592  9565 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:11.460999  9476 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.460879  9565 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.461627  9476 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.463330  9565 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:11.427265  9426 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.464262  9497 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:11.418267  9370 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.465950  9425 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:11.465909  9370 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.466535  9425 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.432127  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:11.467196  1628 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:11.429308  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:11.432605  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.468048  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1414) took 1378 ms. Trace:
I20260709 13:03:11.436039  2350 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:11.410307  9587 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:11.456154  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:11.477854  2352 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:11.468106  1632 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:11.478111  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.408134  9551 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.466360  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:11.424592  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.412748  9289 consensus_meta.cc:287] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.552s	user 0.014s	sys 0.000s
I20260709 13:03:11.433341  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:11.459631  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.468626  2349 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:11.465981  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.469050  2345 rpcz_store.cc:276] 0709 13:03:10.089457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.089641 (+   184us) service_pool.cc:224] Handling call
0709 13:03:10.090065 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.829785 (+739720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.467748 (+637963us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b54004b7154. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:11.467966 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1698531072}
I20260709 13:03:11.439600  9564 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.496304  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.411132  9562 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:03:11.492285  1674 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:11.472276  2318 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.498422  2351 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:11.440255  1601 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:11.479521  1633 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:03:11.487092  2346 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:03:11.500627  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 60446.5us
I20260709 13:03:11.500806  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.434854  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:11.421290  9588 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.923s	user 0.001s	sys 0.005s
I20260709 13:03:11.439425  9443 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:11.499614  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:11.509315  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.510133  1634 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:11.395859  9567 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:11.382144  9545 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.502449  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.514987  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.515981  1632 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:11.459086  9438 raft_consensus.cc:3055] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:11.520992  1674 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:11.485723  9425 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:11.524678  2346 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:11.488276  9428 raft_consensus.cc:3055] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:11.534073  2349 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
I20260709 13:03:11.533877  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.487596  9579 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:03:11.536468  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.492269  9559 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.537963  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:11.538849  2350 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 16.
I20260709 13:03:11.512776  9411 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:11.577309  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:11.572386  1634 raft_consensus.cc:2410] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:03:11.576952  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:11.579597  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.580229  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:11.520704  9438 raft_consensus.cc:740] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:11.539309  9590 raft_consensus.cc:2804] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:11.537827  9428 raft_consensus.cc:740] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:11.594004  1633 raft_consensus.cc:1240] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:03:11.542753  9579 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.498713  9551 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.597790  2352 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:03:11.518956  9564 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.601264  1633 raft_consensus.cc:1275] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:03:11.520196  9567 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 13:03:11.524331  9565 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:03:11.608351  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.614550  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.615477  1633 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:03:11.622421  1633 raft_consensus.cc:1275] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:03:11.631083  1633 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:11.633271  1633 raft_consensus.cc:1240] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:11.597656  9578 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:03:11.638223  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 129641us
I20260709 13:03:11.607714  9579 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:11.643446  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.650065  1633 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:03:11.576026  9559 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.630095  9289 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.639209  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:11.637634  9578 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:11.661242  9618 consensus_queue.cc:1048] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:11.662179  9578 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:11.662456  9500 consensus_queue.cc:1059] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:11.663786  9618 raft_consensus.cc:3055] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:11.664189  9618 raft_consensus.cc:740] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:11.664423  9564 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:11.665678  9614 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:11.665198  9618 consensus_queue.cc:260] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.630335  9590 raft_consensus.cc:697] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:03:11.655369  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 214675us
I20260709 13:03:11.686091  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:11.681696  1633 raft_consensus.cc:1240] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: [19.15-21.16]
W20260709 13:03:11.684688  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45559.2us
I20260709 13:03:11.694096  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:11.693152  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.697360  1633 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:11.650586  9370 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:11.645615  9426 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:11.643168  9385 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:11.700596  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:11.704329  9646 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:11.744356  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.598074  9587 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.699834  9411 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:11.649688  9608 consensus_queue.cc:1048] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:03:11.766569  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.706113  9552 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:11.661159  9289 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.697938  9564 raft_consensus.cc:2804] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:11.688788  9614 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:11.754874  9370 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:03:11.755903  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 55348.9us
I20260709 13:03:11.823613  1527 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:11.773406  9426 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:11.776063  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.661723  9620 consensus_queue.cc:1059] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 13:03:11.773864  1633 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:11.758384  9476 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:11.821165  1628 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
W20260709 13:03:11.822314  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 165082us
I20260709 13:03:11.774418  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:11.828274  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 72395.2us
I20260709 13:03:11.828817  1708 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:11.777863  9644 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.777737  9552 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.812891  9564 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.780555  9411 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:11.757294  9385 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.836745  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:11.658242  9428 consensus_queue.cc:260] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.668923  9438 consensus_queue.cc:260] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:11.852157  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 152921us
I20260709 13:03:11.828349  9426 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.847941  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.856739  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.835807  9545 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:11.843781  9552 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:11.834491  9234 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:03:11.877864  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.879375  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.846117  9476 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:11.829221  9565 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:03:11.842532  9564 raft_consensus.cc:3060] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:11.892073  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.880036  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.875461  1706 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:11.850890  9411 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:11.885907  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.892154  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.872592  9426 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:11.897768  1702 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 24: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:11.826730  9370 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.900388  1701 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:11.918248  1703 raft_consensus.cc:2393] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:03:11.917932  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64041.8us
I20260709 13:03:11.923862  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:11.924199  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 100656us
I20260709 13:03:11.925055  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:11.926151  2346 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 19: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:11.926738  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.923951  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:11.920663  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 65413us
I20260709 13:03:11.853828  9649 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:03:11.930884  2345 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 25: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:11.932956  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.911240  9428 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:11.849519  9644 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.935351  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.939453  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:11.940058  2350 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:11.930038  1674 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:11.842209  9590 consensus_queue.cc:237] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:11.944363  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.940891  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.942492  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:11.946324  2348 raft_consensus.cc:2410] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:03:11.942834  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:11.948766  2346 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:03:11.945204  2353 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:11.919239  9545 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:11.892530  9234 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:11.931855  9438 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:11.966518  9647 log.cc:927] Time spent T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 13:03:11.967633  9648 log.cc:927] Time spent T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:03:11.954856  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.919003  9659 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:11.971094  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:11.971836  1635 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:11.957893  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:12.002076  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 72113.8us
I20260709 13:03:11.915803  9651 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.071s
I20260709 13:03:11.922382  9507 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:03:12.009037  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:12.010938  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:12.013249  2318 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:11.955921  9551 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:12.013959  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:12.028844  2345 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:11.994681  9663 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:11.898746  9614 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.021044  9559 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:12.065457  1635 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:12.038013  9653 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:11.998066  9545 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.038312  9234 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.079754  2350 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:12.048636  9670 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.090152  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:12.098623  1635 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
I20260709 13:03:12.064325  9659 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:03:12.116191  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 102920us
W20260709 13:03:12.117409  2318 leader_election.cc:336] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.118031  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:12.052536  9587 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:03:12.122535  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:12.085454  9289 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.129320  9234 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:12.134053  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:12.055922  9663 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:12.134841  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:12.089751  9653 raft_consensus.cc:697] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:03:12.150766  2318 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.151938  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:12.152436  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:12.153702  2318 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.133116  9385 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:12.158933  9497 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.694s	user 0.008s	sys 0.006s
I20260709 13:03:12.159996  2344 raft_consensus.cc:2410] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:03:12.161114  9644 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:12.169909  9661 log.cc:927] Time spent T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:03:12.172590  9545 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:12.173728  9656 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 13:03:12.176263  2318 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.176913  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:12.178366  9681 log.cc:927] Time spent T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:03:12.178149  9653 consensus_queue.cc:237] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.184571  9673 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.185333  9673 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.186421  9662 consensus_queue.cc:1059] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:12.186939  9497 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:12.187736  9662 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:03:12.198746  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:12.202199  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:12.234704  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:12.196615  9379 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.819s	user 0.006s	sys 0.007s
W20260709 13:03:12.199692  9425 consensus_meta.cc:287] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.651s	user 0.010s	sys 0.002s
I20260709 13:03:12.234519  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:12.110487  9670 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.240479  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:12.197791  9578 consensus_meta.cc:287] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.535s	user 0.009s	sys 0.004s
I20260709 13:03:12.247424  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:12.251426  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 27 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:12.195715  9673 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:12.230490  9589 consensus_meta.cc:287] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.826s	user 0.010s	sys 0.006s
I20260709 13:03:12.201647  9476 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:12.255131  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:12.190155  9649 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:12.257187  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:03:12.233724  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:12.260288  2350 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:12.262574  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:12.265959  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:12.254611  9115 consensus_meta.cc:287] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.848s	user 0.013s	sys 0.000s
I20260709 13:03:12.232225  9565 raft_consensus.cc:3055] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:12.202728  9370 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:12.297946  1525 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:12.302083  1634 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:03:12.304848  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:12.305305  1602 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:12.294868  2345 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:12.308280  1525 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:12.261734  9587 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:12.316224  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:12.245666  9425 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.245198  9379 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.296938  9322 raft_consensus.cc:3055] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:12.338886  9322 raft_consensus.cc:740] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:12.339931  9322 consensus_queue.cc:260] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.341070  9322 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:12.343719  9579 consensus_meta.cc:287] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.684s	user 0.006s	sys 0.006s
I20260709 13:03:12.344336  9579 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.321692  9644 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:12.346369  9663 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.347797  1703 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:12.350656  9670 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:12.310880  9674 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:12.337419  9379 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:12.338282  9587 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:12.317167  9673 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:12.363525  9425 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:12.326720  9562 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.804s	user 0.012s	sys 0.002s
W20260709 13:03:12.331195  2318 leader_election.cc:336] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.301636  9565 raft_consensus.cc:740] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:12.366225  2318 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:12.313035  9559 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:12.261247  9589 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:12.370194  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.364785  9562 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.311535  9476 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:12.292021  9657 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.130s
W20260709 13:03:12.376055  9443 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.842s	user 0.006s	sys 0.008s
I20260709 13:03:12.349323  9552 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.329205  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:12.366540  9672 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.384227  9559 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:12.404690  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 111604us
I20260709 13:03:12.402209  9517 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:12.526599  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 197568us
I20260709 13:03:12.260321  9578 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.407043  9645 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:12.411093  9476 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.532733  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:12.533509  9545 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:12.534636  9618 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:12.536194  9552 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:12.551201  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.428639  9322 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:12.600946  1601 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.015s	sys 0.071s
W20260709 13:03:12.607884  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 64640.5us
W20260709 13:03:12.607982  1601 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.016s	sys 0.071s
I20260709 13:03:12.609195  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:12.609613  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 553503us
W20260709 13:03:12.612174  1601 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 83 similar messages]
W20260709 13:03:12.542022  9494 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.583s	user 0.044s	sys 0.105s
I20260709 13:03:12.532372  9559 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:12.648767  1601 leader_election.cc:336] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:12.622185  1602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.529997  9660 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:12.682508  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:12.680541  1709 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:12.546478  9476 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:12.542582  9673 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:12.429136  9443 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:12.704703  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 178082us
I20260709 13:03:12.537420  9425 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:12.618520  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:12.694203  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47482.8us
W20260709 13:03:12.707839  1601 leader_election.cc:336] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.708287  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:12.710213  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:12.711099  1636 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:12.604149  9674 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.603725  9497 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.624518  9428 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:12.715134  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:12.694736  9296 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.251s	user 0.013s	sys 0.004s
I20260709 13:03:12.717968  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:12.704933  9411 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.769s	user 0.010s	sys 0.002s
I20260709 13:03:12.612586  9670 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:12.716172  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 97746.2us
I20260709 13:03:12.724400  1627 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 13:03:12.707227  9494 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.758s	user 0.044s	sys 0.121s
W20260709 13:03:12.728191  1601 leader_election.cc:336] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.711169  9646 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:12.603673  9545 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.723201  9552 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:12.624589  9322 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.717854  9663 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:12.739171  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:12.734412  9411 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.736899  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:12.543481  9565 consensus_queue.cc:260] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.734519  9518 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.769537  1636 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:12.765224  9579 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:12.738417  9685 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.775370  9411 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:12.779237  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51051.2us
W20260709 13:03:12.788898  9682 log.cc:927] Time spent T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
W20260709 13:03:12.801347  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 64661.2us
I20260709 13:03:12.538013  9672 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.802951  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:12.793442  1601 leader_election.cc:336] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.726228  9497 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:12.770185  9671 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.340s	user 0.000s	sys 0.001s
I20260709 13:03:12.804560  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:12.779068  9564 consensus_meta.cc:287] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.863s	user 0.016s	sys 0.000s
I20260709 13:03:12.812332  1525 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:12.827107  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:12.740831  9670 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.705165  9660 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:12.830428  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:12.843596  1601 leader_election.cc:336] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.844180  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:12.838752  1636 raft_consensus.cc:2468] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:12.836875  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:12.842027  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.790620  9552 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:12.855275  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:12.852645  1601 leader_election.cc:336] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.856454  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:12.822233  9565 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:12.842056  9564 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.866850  9322 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:12.868196  9564 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:12.868477  9672 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.870316  9545 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.871300  9668 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:12.872200  9497 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.827080  9587 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:12.841751  9411 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:12.873790  9443 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.783838  9518 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.733982  9674 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.877686  1704 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:12.878013  9562 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.801379  9685 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.604871  9645 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:12.859400  9673 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:12.882403  9685 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.883351  9578 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:12.883948  9645 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.885259  9670 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:12.841117  9691 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:03:12.901599  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:03:12.908991  2318 leader_election.cc:336] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.910248  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:12.823482  9697 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:12.869010  9564 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:03:12.894845  1601 leader_election.cc:336] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.880256  9674 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:12.915197  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.918502  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.841686  9589 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.855578  9425 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:12.925177  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:12.926395  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.941073  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:12.941810  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:12.956827  2346 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:12.916316  9495 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.914103  9497 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:12.899590  9667 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:12.908314  9545 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:12.958446  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:12.943976  1526 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.018s	sys 0.039s
W20260709 13:03:12.944253  2318 leader_election.cc:336] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.946213  9684 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.942727  9672 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.946744  9517 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:12.958990  2351 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:03:12.963285  1526 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.018s	sys 0.039s
W20260709 13:03:12.963343  2318 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:12.970001  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 566993us
W20260709 13:03:12.971212  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 76457.5us
W20260709 13:03:12.972030  1601 leader_election.cc:336] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:12.972970  1601 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.917042  9670 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:03:12.977349  2318 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.979595  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:12.963481  9552 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:12.962916  9660 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:12.986646  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.984493  2318 leader_election.cc:336] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:12.992327  2318 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.990839  9551 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:12.995381  9551 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:12.997874  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:12.998302  9551 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:12.999028  2350 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:03:12.965389  9684 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.004009  9684 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:13.008060  9689 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
W20260709 13:03:13.013499  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3457) took 1759 ms (client timeout 1559 ms). Trace:
W20260709 13:03:13.013998  1704 rpcz_store.cc:269] 0709 13:03:11.253639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.257571 (+  3932us) service_pool.cc:224] Handling call
0709 13:03:13.013227 (+1755656us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54002e77d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:13.013378 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3721352064}
I20260709 13:03:13.015965  2352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59274 (request call id 1416) took 1094 ms. Trace:
I20260709 13:03:12.962627  9495 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.016449  2352 rpcz_store.cc:276] 0709 13:03:11.921868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.922056 (+   188us) service_pool.cc:224] Handling call
0709 13:03:11.922422 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.930464 (+  8042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.930498 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.930976 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.320365 (+389389us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:03:12.320424 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.320524 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.015714 (+695190us) raft_consensus.cc:1685] finished
0709 13:03:13.015739 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.015898 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81550,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":32863,"thread_start_us":315838,"threads_started":1,"wal-append.queue_time_us":1037805}
I20260709 13:03:12.964214  9497 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.019846  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:13.020536  9670 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:13.028101  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:13.029654  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.029059  9670 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.030412  1631 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:13.030776  9428 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:13.031360  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.019353  2317 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:13.027608  9589 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:13.028131  1527 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:12.967608  9672 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.027486  9663 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:12.973402  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:13.027367  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3386) took 1406 ms. Trace:
I20260709 13:03:13.038954  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.048754  1630 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
W20260709 13:03:13.043023  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 48867.7us
I20260709 13:03:13.023237  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.056927  1635 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:13.026822  1708 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:13.040654  2351 raft_consensus.cc:1240] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:13.046638  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3747) took 1505 ms. Trace:
I20260709 13:03:13.066145  1634 rpcz_store.cc:276] 0709 13:03:11.541336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.579318 (+ 37982us) service_pool.cc:224] Handling call
0709 13:03:13.046431 (+1467113us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b540010bdd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:13.046574 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1659015424}
W20260709 13:03:13.068216  2318 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.047227  1633 rpcz_store.cc:276] 0709 13:03:11.587059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.868981 (+281922us) service_pool.cc:224] Handling call
0709 13:03:11.869383 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.929236 (+ 59853us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.929287 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.930063 (+   776us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.256244 (+326181us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:12.613262 (+357018us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.659219 (+ 45957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.659364 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:12.993384 (+334020us) raft_consensus.cc:1685] finished
0709 13:03:12.993410 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.993589 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75441,"prepare.queue_time_us":481257,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":2803,"raft.queue_time_us":123274,"raft.run_cpu_time_us":429,"raft.run_wall_time_us":29536,"thread_start_us":687367,"threads_started":3,"wal-append.queue_time_us":819855}
I20260709 13:03:13.032152  1631 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:03:13.080614  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:13.080988  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 107972us
I20260709 13:03:13.081521  1634 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 23.
W20260709 13:03:13.069502  2318 leader_election.cc:336] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:13.081807  1601 leader_election.cc:336] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.082083  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:13.082736  1701 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:03:13.020700  9426 consensus_meta.cc:287] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.099s	user 0.014s	sys 0.000s
I20260709 13:03:13.078995  9404 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.043666  9517 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:13.090600  9690 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:03:13.015112  9518 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.038594  9294 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:13.090255  9404 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:13.090708  9099 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.047482  9552 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:13.019718  9495 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.034294  9589 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:13.036527  9675 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.701s	user 0.000s	sys 0.002s
W20260709 13:03:13.259940  1706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3469) took 1493 ms (client timeout 1986 ms). Trace:
I20260709 13:03:13.138350  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:13.255832  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 400370us
I20260709 13:03:13.193011  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.137732  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:13.275977  1710 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:13.096488  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:13.254227  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.097811  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.137087  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:13.278398  1630 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 25.
I20260709 13:03:13.275861  1707 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:13.279346  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:13.150398  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 66818.9us
W20260709 13:03:13.086331  2318 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.281428  2318 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:13.039140  9672 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:13.274426  1706 rpcz_store.cc:269] 0709 13:03:11.766483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.774022 (+  7539us) service_pool.cc:224] Handling call
0709 13:03:13.259733 (+1485711us) spinlock_profiling.cc:243] Waited 39.1 ms on lock 0x7b54002a71d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:13.259872 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89952000}
I20260709 13:03:13.290509  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:13.139950  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:13.277029  1702 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
W20260709 13:03:13.285741  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 199311us
I20260709 13:03:13.073828  9476 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:13.322932  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:13.094261  9294 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.295079  1707 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:13.192991  9578 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.091359  9667 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:13.269284  9640 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.917s	user 0.003s	sys 0.009s
I20260709 13:03:13.292066  9685 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:13.171612  9559 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.323953  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:13.292451  9099 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.289390  9552 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:13.357528  9294 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:13.283157  9589 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:13.349284  9657 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:13.356721  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 216713us
I20260709 13:03:13.358662  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.365101  1707 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
W20260709 13:03:13.353631  9675 log.cc:927] Time spent T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:13.295991  9565 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:13.092772  9426 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:13.366990  2318 leader_election.cc:336] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.353346  9703 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:13.352231  1705 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:03:13.366079  9640 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.007s	sys 0.011s
I20260709 13:03:13.390028  1630 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:13.393877  1701 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 25.
W20260709 13:03:13.412315  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46636.3us
I20260709 13:03:13.359714  9709 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:13.401588  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:13.368332  9497 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:13.377195  9567 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.410634  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.376734  9545 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:13.417229  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.433903  1705 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 17.
I20260709 13:03:13.418627  9099 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.436523  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:13.437131  1707 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:03:13.418830  9552 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:13.419966  9703 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.421213  9426 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.421811  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.429621  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:13.431116  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:03:13.432467  2318 leader_election.cc:336] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.380141  9578 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.446328  1707 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 27.
I20260709 13:03:13.444262  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3750) took 1607 ms. Trace:
I20260709 13:03:13.446768  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:13.447356  1630 rpcz_store.cc:276] 0709 13:03:11.836879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.431498 (+1594619us) service_pool.cc:224] Handling call
0709 13:03:13.443764 (+ 12266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.443958 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.443982 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:13.444020 (+    38us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:13.444051 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.444100 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.444205 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.442480  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:13.445703  9670 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:13.439267  9545 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.425993  9565 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.411542  9657 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:13.435901  9567 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:13.487124  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.495251  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3749) took 1658 ms. Trace:
W20260709 13:03:13.496443  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.519164  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.519495  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3398) took 1521 ms. Trace:
I20260709 13:03:13.519945  1630 rpcz_store.cc:276] 0709 13:03:11.998642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.502999 (+1504357us) service_pool.cc:224] Handling call
0709 13:03:13.519082 (+ 16083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.519237 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.519253 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:13.519288 (+    35us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:13.519310 (+    22us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.519358 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.519443 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.517975  1627 rpcz_store.cc:276] 0709 13:03:11.836579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.747310 (+910731us) service_pool.cc:224] Handling call
0709 13:03:12.747808 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.762204 (+ 14396us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:12.783686 (+ 21482us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.998585 (+214899us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b18004231c0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed3f3 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:03:13.197934 (+199349us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:13.202798 (+  4864us) log.cc:844] Serialized 26 byte log entry
0709 13:03:13.202993 (+   195us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:13.203046 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.203193 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.444809 (+241616us) raft_consensus.cc:1685] finished
0709 13:03:13.448953 (+  4144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.495150 (+ 46197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1462,"prepare.queue_time_us":21772,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":49,"raft.queue_time_us":15212,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":11357,"spinlock_wait_cycles":28092288,"thread_start_us":340821,"threads_started":1,"wal-append.queue_time_us":448193}
W20260709 13:03:13.513664  9706 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.177s	user 0.000s	sys 0.005s
I20260709 13:03:13.430917  9709 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:13.506223  9564 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:13.500424  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.498459  1710 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
I20260709 13:03:13.503480  9704 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:13.515060  9670 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:03:13.499356  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 56921.7us
I20260709 13:03:13.530886  1710 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 26. Current term is 27. Ops: [24.15-26.16]
I20260709 13:03:13.544363  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.546147  1710 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:13.536249  1630 raft_consensus.cc:1240] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:13.526057  1707 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:13.547583  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.549345  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:13.551206  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3751) took 1547 ms. Trace:
I20260709 13:03:13.551263  1707 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:13.551801  1630 rpcz_store.cc:276] 0709 13:03:12.003636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.520825 (+1517189us) service_pool.cc:224] Handling call
0709 13:03:13.536051 (+ 15226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.550918 (+ 14867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.551139 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.555495  1707 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:03:13.542203  9704 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:13.555483  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:13.577816  1600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.547019  9564 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 25 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:13.579937  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3753) took 1501 ms. Trace:
I20260709 13:03:13.580401  1630 rpcz_store.cc:276] 0709 13:03:12.078366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.554798 (+1476432us) service_pool.cc:224] Handling call
0709 13:03:13.579201 (+ 24403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.579456 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:13.579495 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:13.579551 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:13.579588 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.579655 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.579873 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.563526  9476 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:13.580688  9704 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:13.582100  9425 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:03:13.584436  9703 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.586993  9559 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.613241  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.620512  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:13.623531  1707 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:13.610342  9425 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 27 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:13.602922  9294 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:13.631400  9713 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:03:13.678210  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1786) took 2223 ms. Trace:
W20260709 13:03:13.653096  1676 connection.cc:256] RPC call timeout handler was delayed by -1.4911007549999908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2014s for a response
I20260709 13:03:13.678709  2349 rpcz_store.cc:276] 0709 13:03:11.454635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.454965 (+   330us) service_pool.cc:224] Handling call
0709 13:03:13.678010 (+2223045us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b54004be1d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:13.678144 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29300096}
I20260709 13:03:13.680104  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:13.680168  1676 connection.cc:256] RPC call timeout handler was delayed by -1.4368947689999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
I20260709 13:03:13.680842  2349 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:13.682135  2353 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:03:13.636790  9712 log.cc:927] Time spent T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:03:13.647712  9294 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.650100  9545 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.655609  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3739) took 2224 ms. Trace:
I20260709 13:03:13.700364  1629 rpcz_store.cc:276] 0709 13:03:11.431576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.431710 (+   134us) service_pool.cc:224] Handling call
0709 13:03:11.432095 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.449384 (+ 17289us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:11.449749 (+   365us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.449975 (+   226us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.450025 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:11.450074 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.655318 (+2205244us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b54003cd7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:13.655384 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.655537 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":747,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":13979,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":39050112}
I20260709 13:03:13.656148  9657 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:13.656239  1526 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:13.657316  9589 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.708437  9567 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:13.658454  9578 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.662938  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:13.709609  2351 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:13.710387  9589 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.710722  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:13.711041  9589 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:13.711412  2351 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:13.711532  9589 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:13.712378  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:13.712524  9589 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.9, Last appended by leader: 9, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:13.713019  2351 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:03:13.713996  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:13.664032  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:13.716058  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:13.716328  2352 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:13.669694  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3754) took 1518 ms. Trace:
I20260709 13:03:13.718102  1628 rpcz_store.cc:276] 0709 13:03:12.151259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.648467 (+1497208us) service_pool.cc:224] Handling call
0709 13:03:13.648927 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.649188 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.649227 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:13.649289 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:13.649337 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.649410 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.669602 (+ 20192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:13.719906  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.721055  1628 raft_consensus.cc:2410] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:03:13.671777  9665 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:13.721812  9665 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:13.723126  9665 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.724982  9665 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:13.671341  9552 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:13.726140  9552 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:03:13.726624  9552 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 26 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:13.728106  9552 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:13.632448  9554 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.687533  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:13.688416  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:13.730371  9618 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.730805  1710 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:03:13.715535  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.730962  9618 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:13.699059  9099 consensus_queue.cc:1059] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 13:03:13.733788  9099 raft_consensus.cc:3055] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:13.734221  9099 raft_consensus.cc:740] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:13.735100  9099 consensus_queue.cc:260] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.736395  9099 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:13.597584  9670 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:13.628847  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.675606  9715 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:03:13.741182  1710 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:13.741617  9578 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.742265  9578 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.742506  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.745071  9578 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.745524  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.750044  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:13.751324  1705 raft_consensus.cc:1240] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:03:13.698954  9426 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.754186  9426 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:13.755270  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.756088  1701 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:03:13.756426  9426 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:13.625989  9564 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:13.741067  9425 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:13.761189  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:13.761751  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.761911  1629 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:03:13.752646  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:03:13.770150  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 78235.2us
I20260709 13:03:13.776381  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.777411  1707 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:13.778486  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.779166  1707 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
I20260709 13:03:13.780082  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.780967  1707 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:03:13.780098  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.781970  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.780731  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:13.786155  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:13.749325  1675 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.790408  1602 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:13.791465  1707 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:13.792644  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.707010  9565 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:13.794667  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:13.795383  1628 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:03:13.786914  1705 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:03:13.799389  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.800438  1705 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 24.
I20260709 13:03:13.801853  1707 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:03:13.802839  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.803592  1707 raft_consensus.cc:2393] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 23: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:03:13.804194  9642 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 1.973s	user 0.001s	sys 0.009s
I20260709 13:03:13.804704  1707 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:03:13.806682  9657 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:13.807499  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3755) took 1423 ms. Trace:
I20260709 13:03:13.733400  9618 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:13.807915  1636 rpcz_store.cc:276] 0709 13:03:12.384424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.717665 (+333241us) service_pool.cc:224] Handling call
0709 13:03:13.807337 (+1089672us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b540010e0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:13.807441 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90150656}
I20260709 13:03:13.808929  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.809021  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.808995  1602 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:03:13.814956  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.825888  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:13.826352  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 45697.4us
I20260709 13:03:13.827262  1636 raft_consensus.cc:2393] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:13.697650  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:13.836347  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 140036us
I20260709 13:03:13.837234  1676 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:13.842897  1629 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 25.
I20260709 13:03:13.834707  1525 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:13.786821  9552 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:03:13.851140  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.868191  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:13.851369  9620 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:13.823686  9667 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:13.853853  9685 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:13.874114  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 2717) took 2017 ms (client timeout 2362 ms). Trace:
W20260709 13:03:13.874565  1708 rpcz_store.cc:269] 0709 13:03:11.856278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.856455 (+   177us) service_pool.cc:224] Handling call
0709 13:03:13.873944 (+2017489us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540016bf54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:13.874054 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2660700544}
I20260709 13:03:13.877827  1628 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:13.878876  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.879664  1628 raft_consensus.cc:2410] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:03:13.880422  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:13.882483  9497 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.882613  2317 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:13.883530  1628 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:03:13.858704  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.860656  9655 log.cc:927] Time spent T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.943s	user 0.003s	sys 0.000s
I20260709 13:03:13.885093  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:13.885767  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3383) took 2392 ms. Trace:
I20260709 13:03:13.886231  9404 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:13.887383  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:13.886153  1632 rpcz_store.cc:276] 0709 13:03:11.493791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.630355 (+136564us) service_pool.cc:224] Handling call
0709 13:03:11.654224 (+ 23869us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:11.661019 (+  6795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.661058 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:11.661879 (+   821us) log.cc:844] Serialized 52 byte log entry
0709 13:03:11.764675 (+102796us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:11.817950 (+ 53275us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.886890 (+1068940us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b18004dd460. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ea455 00007fda276d2060 00007fda276e4716 00007fda276f5287
0709 13:03:12.887333 (+   443us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.887545 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.887636 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.792719 (+905083us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b18004dd460. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:03:13.859982 (+ 67263us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:03:13.885552 (+ 25570us) raft_consensus.cc:1685] finished
0709 13:03:13.885573 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.885707 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":963094,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":94,"raft.queue_time_us":69230,"raft.run_cpu_time_us":440,"raft.run_wall_time_us":23072,"spinlock_wait_cycles":31809664,"thread_start_us":179636,"threads_started":2,"wal-append.queue_time_us":177327}
I20260709 13:03:13.888103  1628 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:03:13.888867  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.889596  9234 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:13.890112  1632 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 22: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:13.862463  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.890792  1635 raft_consensus.cc:2468] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:13.891270  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:13.892148  9684 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:13.892565  9684 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.892616  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:13.893110  9684 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:13.894524  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:13.870186  1703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3471) took 1677 ms. Trace:
I20260709 13:03:13.897855  1703 rpcz_store.cc:276] 0709 13:03:12.192960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.193143 (+   183us) service_pool.cc:224] Handling call
0709 13:03:13.861399 (+1668256us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400078354. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:13.870113 (+  8714us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255235456}
I20260709 13:03:13.883106  9497 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.890079  9234 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:13.900907  9234 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:13.869238  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.788516  1675 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:13.907104  9497 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:13.907354  9404 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:13.907841  9497 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:13.907168  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 157811us
I20260709 13:03:13.871243  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:13.844298  9657 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:13.919451  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.918035  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.920361  1634 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:13.920727  1633 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:13.843870  9618 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:13.924861  2318 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:13.867297  9703 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.908289  9497 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:13.921692  9404 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:13.910033  9554 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.925359  1674 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:13.940709  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:13.919807  9657 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.934037  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:13.911455  9289 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:13.939273  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.923599  9426 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.921895  9545 raft_consensus.cc:2764] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:03:13.918694  9559 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:13.915076  9476 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:13.949150  9546 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:13.950942  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:13.955858  1634 raft_consensus.cc:2410] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for term 24 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:03:13.858204  1676 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:13.959499  2318 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:13.990119  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:13.977170  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 320848us
I20260709 13:03:13.991968  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:13.997164  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:14.005678  1628 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
I20260709 13:03:13.938710  9497 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:14.009435  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 58536.4us
I20260709 13:03:13.940423  9404 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:13.953449  9657 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:14.043493  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:13.979387  9289 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:14.013922  9722 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:13.952250  9554 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.053939  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:13.986604  9426 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:13.938117  9703 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:13.956344  9685 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:14.107963  1635 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:03:14.108280  9685 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:14.108978  9685 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.109422  9703 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:14.113912  9723 log.cc:927] Time spent T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:03:14.111706  9685 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:14.113150  9726 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:03:14.115854  9725 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:03:14.117018  9685 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:14.118227  9685 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:14.118680  9685 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.119117  9685 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:14.132205  9564 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:14.142513  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:14.143692  1701 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:14.041811  9407 raft_consensus.cc:2804] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:03:14.044943  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 47891.3us
I20260709 13:03:14.068224  9404 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:14.035022  9294 raft_consensus.cc:2804] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:14.034296  9559 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.198217  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1590) took 1767 ms. Trace:
I20260709 13:03:14.051614  9476 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:14.211210  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1456) took 1319 ms. Trace:
I20260709 13:03:14.213840  2347 rpcz_store.cc:276] 0709 13:03:12.430578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.430770 (+   192us) service_pool.cc:224] Handling call
0709 13:03:12.431280 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.625710 (+194430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.625760 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:12.625822 (+    62us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:12.625863 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.197884 (+1572021us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b540005bc54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:14.197946 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.198143 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446192896}
I20260709 13:03:14.214141  1631 rpcz_store.cc:276] 0709 13:03:12.892021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.901270 (+1009249us) service_pool.cc:224] Handling call
0709 13:03:13.901657 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.901896 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:03:13.902179 (+   283us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.210242 (+308063us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.210388 (+   146us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:14.210528 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.210903 (+   375us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.211120 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2464,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":256,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":135,"thread_start_us":306913,"threads_started":1}
I20260709 13:03:14.058593  1628 raft_consensus.cc:1240] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:14.171085  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1588) took 1904 ms. Trace:
I20260709 13:03:14.225872  2348 rpcz_store.cc:276] 0709 13:03:12.266891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.293983 (+ 27092us) service_pool.cc:224] Handling call
0709 13:03:12.294330 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.294555 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:03:12.315593 (+ 21038us) log.cc:844] Serialized 14 byte log entry
0709 13:03:13.879146 (+1563553us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b18004dfda0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a23a 00007fda276e4a2a
0709 13:03:14.170441 (+291295us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.170552 (+   111us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:14.170631 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.170758 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.170984 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20630,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"spinlock_wait_cycles":13241216,"thread_start_us":290918,"threads_started":1}
I20260709 13:03:14.222100  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1457) took 1329 ms. Trace:
I20260709 13:03:14.230612  1628 rpcz_store.cc:276] 0709 13:03:12.892343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.057951 (+1165608us) service_pool.cc:224] Handling call
0709 13:03:14.058359 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.221705 (+163346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.222016 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.238545  9589 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:14.085466  9722 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:14.249265  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 356513us
I20260709 13:03:14.272130  2317 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:14.078151  9684 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.197741  9672 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:14.206250  9294 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:14.281031  9294 raft_consensus.cc:3060] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:14.282596  9554 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:14.278812  9722 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:14.284047  9704 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.675s	user 0.012s	sys 0.000s
W20260709 13:03:14.255192  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 329691us
I20260709 13:03:14.245424  9476 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:14.245653  9407 raft_consensus.cc:697] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:14.284513  9704 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.285084  9476 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:14.285923  1632 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:14.285778  9407 consensus_queue.cc:237] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:14.252105  9638 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:14.291005  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:14.295585  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:14.296943  2348 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:14.297173  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:14.297966  2349 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:03:14.300491  2347 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:14.323227  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:14.326606  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 403767us
I20260709 13:03:14.349243  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:03:14.350630  2344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59258 (request call id 1794) took 2893 ms (client timeout 2578 ms). Trace:
W20260709 13:03:14.351351  2344 rpcz_store.cc:269] 0709 13:03:11.457198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.465626 (+  8428us) service_pool.cc:224] Handling call
0709 13:03:14.323234 (+2857608us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b540055ffd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:14.350498 (+ 27264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1525579136}
I20260709 13:03:14.328536  9704 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:14.334086  9719 log.cc:927] Time spent T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:03:14.335012  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:14.340660  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1450) took 2215 ms. Trace:
I20260709 13:03:14.342109  1629 raft_consensus.cc:2410] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 13:03:14.389104  2346 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:14.368149  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:14.389092  1630 rpcz_store.cc:276] 0709 13:03:12.125282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.584100 (+1458818us) service_pool.cc:224] Handling call
0709 13:03:13.585827 (+  1727us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:13.602799 (+ 16972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.602833 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:13.611906 (+  9073us) log.cc:844] Serialized 52 byte log entry
0709 13:03:14.063311 (+451405us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:03:14.063410 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.063642 (+   232us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:14.340130 (+276488us) raft_consensus.cc:1685] finished
0709 13:03:14.340186 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.340504 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":4353,"thread_start_us":439187,"threads_started":1,"wal-append.queue_time_us":709734}
W20260709 13:03:14.398150  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:14.383651  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 525161us
I20260709 13:03:14.384766  2349 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
I20260709 13:03:14.414844  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:14.416476  1676 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:14.293843  9672 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.380240  9565 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.396986  9665 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.007s
I20260709 13:03:14.420985  9638 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:03:14.424212  9099 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.687s	user 0.009s	sys 0.004s
W20260709 13:03:14.473706  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 57300.3us
W20260709 13:03:14.419051  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 428752us
W20260709 13:03:14.524063  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 485551us
W20260709 13:03:14.419983  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 211953us
I20260709 13:03:14.338673  9685 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.410758  9731 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.498907  2317 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:14.494355  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:14.488968  2348 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
W20260709 13:03:14.551757  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 52937.2us
I20260709 13:03:14.529762  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1452) took 2371 ms. Trace:
I20260709 13:03:14.495714  9665 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:14.496059  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.495097  9653 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:14.591933  2346 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:14.545644  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:14.542280  9638 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:14.548844  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:14.552641  1632 rpcz_store.cc:276] 0709 13:03:12.158127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.892938 (+1734811us) service_pool.cc:224] Handling call
0709 13:03:13.893267 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.286361 (+393094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.529214 (+242853us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b5400343854. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:14.529665 (+   451us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901826816}
I20260709 13:03:14.570443  9294 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:14.610008  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 64416us
W20260709 13:03:14.611392  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 62607.6us
I20260709 13:03:14.612263  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:14.610351  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:14.597432  1710 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:03:14.539345  9730 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:14.613685  2348 raft_consensus.cc:2410] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:03:14.445583  9565 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.616549  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:14.612157  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:14.547111  1526 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:14.539921  9234 consensus_meta.cc:287] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.638s	user 0.015s	sys 0.010s
I20260709 13:03:14.554003  9289 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:14.621254  1602 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:03:14.627919  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 75356.1us
W20260709 13:03:14.632879  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 85812.7us
I20260709 13:03:14.617331  1630 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
I20260709 13:03:14.633831  1526 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:14.636819  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:14.604763  9665 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:14.610110  9638 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:14.557886  9731 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.640828  1527 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:14.643147  1676 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:14.641710  1602 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:14.644804  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:14.790783  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 162041us
W20260709 13:03:14.808660  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 166931us
I20260709 13:03:14.615952  9730 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.615995  9578 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:14.622978  9494 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:03:14.636416  9546 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.623s	user 0.012s	sys 0.000s
W20260709 13:03:14.810585  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 167629us
I20260709 13:03:14.826869  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:14.658200  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:14.622174  9733 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:03:14.839128  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.842021  9739 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.929157  9739 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:14.932173  9739 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:14.819471  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 207261us
I20260709 13:03:14.835095  9426 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:14.820535  9653 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:14.816725  9662 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:14.623078  9740 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:14.933934  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:14.934572  9740 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.937640  9740 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:14.938395  9740 consensus_queue.cc:1059] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:14.834088  9742 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:14.833406  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:14.940618  9742 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:14.940902  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 107527us
I20260709 13:03:14.941856  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:14.941988  9653 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:14.942804  9497 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.893s	user 0.013s	sys 0.002s
I20260709 13:03:14.941233  9742 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:14.943356  9497 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.836114  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1459) took 1943 ms. Trace:
I20260709 13:03:14.945283  1628 rpcz_store.cc:276] 0709 13:03:12.892893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.232114 (+1339221us) service_pool.cc:224] Handling call
0709 13:03:14.232600 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.256407 (+ 23807us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:03:14.286868 (+ 30461us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.547825 (+260957us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.547887 (+    62us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:14.547929 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.835202 (+287273us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b180044f140. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245bb2a6 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419
0709 13:03:14.835874 (+   672us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b54001b6054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:14.835936 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.836057 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16545,"prepare.queue_time_us":30676,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":15600,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5809,"spinlock_wait_cycles":57006208,"thread_start_us":245048,"threads_started":1,"wal-append.queue_time_us":447520}
I20260709 13:03:14.947844  1628 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:03:14.948467  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3412) took 1257 ms. Trace:
I20260709 13:03:14.948768  1628 rpcz_store.cc:276] 0709 13:03:13.691331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.947204 (+1255873us) service_pool.cc:224] Handling call
0709 13:03:14.947594 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.948191 (+   597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.948409 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.839779  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1451) took 2682 ms. Trace:
I20260709 13:03:14.949653  1635 rpcz_store.cc:276] 0709 13:03:12.157746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.891450 (+1733704us) service_pool.cc:224] Handling call
0709 13:03:13.891817 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.108459 (+216642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.837446 (+728987us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54003cdcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:14.839714 (+  2268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496036736}
W20260709 13:03:14.837308  9667 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.921s	user 0.008s	sys 0.002s
W20260709 13:03:14.950731   431 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 1532)
I20260709 13:03:14.951203  9667 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:14.952095  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3467) took 3348 ms. Trace:
I20260709 13:03:14.877492  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3758) took 1969 ms. Trace:
I20260709 13:03:14.953154  1632 rpcz_store.cc:276] 0709 13:03:12.907729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.821440 (+1913711us) service_pool.cc:224] Handling call
0709 13:03:14.876477 (+ 55037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.876957 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.877019 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.877094 (+    75us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:14.877144 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.877216 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.877380 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.636029  9684 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:14.955060  9684 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:03:14.955989  9684 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:14.956683  9684 raft_consensus.cc:3055] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:14.957027  9684 raft_consensus.cc:740] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:14.829770  1630 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 17.
I20260709 13:03:14.686926  9294 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:14.688171  9545 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.710080  9734 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.882433  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 248362us
I20260709 13:03:14.972539  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:14.710698  9743 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:14.890417  9672 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:14.820652  9561 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:14.825775  9730 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:14.710971  9520 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:14.637104  9554 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.675374  9728 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
W20260709 13:03:14.731458  9620 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.820s	user 0.006s	sys 0.007s
W20260709 13:03:14.694820  9729 log.cc:927] Time spent T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:03:14.838927  9741 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:14.631532  9704 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:14.822366  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 181533us
I20260709 13:03:14.899502  9754 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:14.731664  9722 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:14.905951  9738 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:14.906980  9685 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:14.667289  9665 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.640568  9289 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.246s
I20260709 13:03:14.630777  9234 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.010839  9731 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:14.806989  9425 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.097s
I20260709 13:03:14.912415  9494 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.736928  9404 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.918355  9752 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:14.917212  9638 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.919678  9565 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:14.921522  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:15.012481  9567 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.942706  9653 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:14.952457  1709 rpcz_store.cc:276] 0709 13:03:11.603587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.603789 (+   202us) service_pool.cc:224] Handling call
0709 13:03:11.632455 (+ 28666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.687464 (+1055009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.951807 (+2264343us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54002d87d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:14.952006 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2281694592}
I20260709 13:03:14.957239  9559 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:14.957667  1631 raft_consensus.cc:1240] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:14.829133  9546 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:14.645740  9670 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:03:14.959038  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.645591  9737 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.257s
I20260709 13:03:14.967908  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:14.958412  9476 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.673s	user 0.007s	sys 0.005s
I20260709 13:03:14.983661  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:14.982071  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:14.975703  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3415) took 1140 ms. Trace:
I20260709 13:03:15.107334  1628 rpcz_store.cc:276] 0709 13:03:13.835338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.949438 (+1114100us) service_pool.cc:224] Handling call
0709 13:03:14.973752 (+ 24314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.975259 (+  1507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.975298 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.975356 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:14.975403 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.975474 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.975627 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.984306  9734 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:14.988709  1701 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:03:15.122298  1701 pending_rounds.cc:85] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 12
I20260709 13:03:15.122717  1701 pending_rounds.cc:107] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:03:14.989791  1708 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 13:03:14.991328  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:15.118170  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3408) took 1431 ms. Trace:
I20260709 13:03:15.123546  1629 rpcz_store.cc:276] 0709 13:03:13.690132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.867861 (+177729us) service_pool.cc:224] Handling call
0709 13:03:15.117872 (+1250011us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b5400196c54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:15.118079 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1086291328}
I20260709 13:03:15.124112  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:15.124519  1708 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:14.997249  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:14.964859  9742 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:14.974922  9736 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:15.006104  9758 raft_consensus.cc:3060] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:15.141345  1708 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:14.965018  9709 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.965072  9426 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.139634  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:14.970094  9541 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:14.965096  9739 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:15.011451  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:15.053810  1702 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 9. Preceding OpId from leader: term: 23 index: 10. (index mismatch)
I20260709 13:03:14.972644  9497 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:15.003187  9672 raft_consensus.cc:3055] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:15.007632  9554 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.057413  9704 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:15.058313  1632 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:15.053544  1527 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:15.106866  9760 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.002882  9662 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.019726  9657 consensus_meta.cc:287] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.934s	user 0.013s	sys 0.000s
I20260709 13:03:14.986255  9294 consensus_queue.cc:1059] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.088s
W20260709 13:03:15.067690  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 247269us
I20260709 13:03:15.129700  9685 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:15.072667  9746 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.069620  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 170917us
W20260709 13:03:15.376622  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.019208  9620 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:15.078485  2348 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
W20260709 13:03:14.928180  9564 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.795s	user 0.008s	sys 0.004s
W20260709 13:03:15.079241  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 145339us
I20260709 13:03:15.378877  9564 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:15.080679  9768 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:15.379750  9564 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.379992  9768 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.380376  9564 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.383455  9768 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:15.384192  9768 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.006471  9684 consensus_queue.cc:260] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.385769  9684 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:15.377403  1675 leader_election.cc:336] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:15.003707  9561 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:15.388401  1675 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:15.389451  1630 raft_consensus.cc:2410] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 24 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 13:03:15.390647  9561 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:15.108825  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:15.394447  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:15.395398  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3763) took 1480 ms. Trace:
I20260709 13:03:14.987659  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:15.088676  9520 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:15.119455  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:15.003296  2316 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.395908  1628 rpcz_store.cc:276] 0709 13:03:13.914638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.394255 (+1479617us) service_pool.cc:224] Handling call
0709 13:03:15.394708 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.394972 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.395010 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.395070 (+    60us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:15.395132 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.395204 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.395332 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.122954  1701 raft_consensus.cc:2889] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:15.126020  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3762) took 1211 ms. Trace:
I20260709 13:03:15.397008  9520 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.138458  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:15.087661  9565 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.006367  1704 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:03:14.961788  9667 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:15.398480  9565 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.398694  9667 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.397890  1629 rpcz_store.cc:276] 0709 13:03:13.914386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.124861 (+1210475us) service_pool.cc:224] Handling call
0709 13:03:15.125132 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.125360 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.125399 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.125463 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:15.125587 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.125714 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.125943 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.400750  9667 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:15.070076  9738 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:15.143770  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 146579us
I20260709 13:03:15.403228  9738 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:15.400693  9565 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:15.148794  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 95322.3us
I20260709 13:03:15.401356  9667 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:15.054252  9741 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.083184  9494 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.406407  9667 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.407167  9741 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:15.408986  9494 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:15.108933  9752 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.409758  9752 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.410764  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3765) took 1495 ms. Trace:
I20260709 13:03:15.411377  1629 rpcz_store.cc:276] 0709 13:03:13.915117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.400881 (+1485764us) service_pool.cc:224] Handling call
0709 13:03:15.401256 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.410245 (+  8989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.410291 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.410360 (+    69us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:15.410425 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.410540 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.410690 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.412173  9752 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:15.152771  9766 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:15.147927  9426 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.413300  9766 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:15.414371  9766 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.144025  9541 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.417982  9541 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:15.416903  9620 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:15.148180  9438 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:15.421690  9438 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.422202  9438 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:15.138381  9736 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.425818  9736 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:15.162936  1707 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:15.114270  9665 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:15.128844  9546 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.129098  9731 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.071846  9756 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.447777  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3760) took 1533 ms. Trace:
I20260709 13:03:14.973980  9545 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:15.454638  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 75043.5us
I20260709 13:03:15.455487  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:15.397746  1628 raft_consensus.cc:1275] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:03:15.415500  9426 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:15.088434  9567 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.412797  9768 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.409337  9667 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:15.396108  1675 leader_election.cc:336] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:15.473667  2348 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:03:15.479271  2348 pending_rounds.cc:85] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting all ops after (but not including) 13
I20260709 13:03:15.479545  2348 pending_rounds.cc:107] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Aborting uncommitted NO_OP operation due to leader change: 19.14
W20260709 13:03:15.479058  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.480041  2348 raft_consensus.cc:2889] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:15.453924  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 82716.8us
I20260709 13:03:15.147760  9670 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:15.127105  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:15.448331  1632 rpcz_store.cc:276] 0709 13:03:13.913880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.954181 (+1040301us) service_pool.cc:224] Handling call
0709 13:03:14.954532 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.148131 (+193599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.447508 (+299377us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54003cdcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:15.447711 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237515008}
I20260709 13:03:15.456667  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3764) took 1541 ms. Trace:
I20260709 13:03:15.115089  9476 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.513660  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:15.513923  9734 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:15.514374  2347 raft_consensus.cc:2468] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 16.
I20260709 13:03:15.511507  1706 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:15.511235  1628 rpcz_store.cc:276] 0709 13:03:13.914886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.397117 (+1482231us) service_pool.cc:224] Handling call
0709 13:03:15.397425 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.456300 (+ 58875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.456590 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.516358  2349 raft_consensus.cc:1240] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:15.430047  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:15.470952  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:15.471735  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:15.173761  9742 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:15.146338  9672 raft_consensus.cc:740] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:15.482741  1527 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:03:15.479568  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 83454.9us
I20260709 13:03:15.487825  1633 raft_consensus.cc:2393] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:15.477020  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3769) took 1559 ms. Trace:
W20260709 13:03:15.644044  1675 leader_election.cc:336] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:15.485611  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:15.644858  1675 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:15.645555  2348 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:15.439810  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59270 (request call id 1592) took 2435 ms. Trace:
I20260709 13:03:15.485616  9754 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.513295  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:15.521057  2346 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
W20260709 13:03:15.635501  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 104267us
I20260709 13:03:15.638445  1629 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:15.150211  9739 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.646065  2350 rpcz_store.cc:276] 0709 13:03:13.004614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.028013 (+ 23399us) service_pool.cc:224] Handling call
0709 13:03:13.028377 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.616387 (+588010us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:13.616712 (+   325us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.254794 (+638082us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.254856 (+    62us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:14.254906 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.439523 (+1184617us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b5400643a54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:15.439586 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.439732 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":128,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":36105,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":4032,"spinlock_wait_cycles":1349078912,"thread_start_us":637688,"threads_started":2,"wal-append.queue_time_us":423855,"wal-append.run_cpu_time_us":2589,"wal-append.run_wall_time_us":1368425}
I20260709 13:03:15.145689  9709 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.644127  1634 rpcz_store.cc:276] 0709 13:03:13.917512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.437918 (+1520406us) service_pool.cc:224] Handling call
0709 13:03:15.475934 (+ 38016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.476288 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.476348 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.476482 (+   134us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:15.476573 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.476663 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.476867 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.460865  9564 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:15.655617  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 172897us
W20260709 13:03:15.657467  1675 leader_election.cc:336] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:15.668926  1675 leader_election.cc:336] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:15.670177  1675 leader_election.cc:336] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:15.658732  1602 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:15.179960  9704 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.665926  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:15.682593  1601 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.077s	sys 0.162s
W20260709 13:03:15.682971  1601 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.077s	sys 0.162s
W20260709 13:03:15.683270  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 542664us
I20260709 13:03:15.371366  9657 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.430423  9546 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.478276  9753 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:15.515301  9670 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:15.455065  9685 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:15.723927  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 65209.3us
I20260709 13:03:15.755914  1636 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:15.731877  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3772) took 1343 ms. Trace:
W20260709 13:03:15.721338  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.758694  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 73067us
I20260709 13:03:15.758586  1634 rpcz_store.cc:276] 0709 13:03:14.388302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.653951 (+1265649us) service_pool.cc:224] Handling call
0709 13:03:15.730954 (+ 77003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.731374 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.731424 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.731488 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:15.731544 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.731613 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.731778 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.759500  1601 leader_election.cc:336] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:15.759986  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:15.735399  1600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.049s	sys 0.079s
W20260709 13:03:15.760751  1600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.049s	sys 0.079s
I20260709 13:03:15.682353  9099 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:15.463658  9497 consensus_queue.cc:1059] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.207s
I20260709 13:03:15.655318  9667 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.742449  9750 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.032s	sys 0.056s
I20260709 13:03:15.750466  2349 raft_consensus.cc:1240] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:03:15.371062  9662 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.373857  9760 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.761720  2346 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 23: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:15.430727  9731 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.737537  9520 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:15.457979  9746 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.693534  9740 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
W20260709 13:03:15.527509  1526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.007s	sys 0.039s
W20260709 13:03:15.830770  1526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.007s	sys 0.052s
W20260709 13:03:15.776680  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 344116us
I20260709 13:03:15.797628  9234 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:15.751582  9670 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:15.684298  9592 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.132s
I20260709 13:03:15.802680  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:15.847168  1707 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:15.803872  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:15.849870  1701 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:03:15.850744  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:15.799939  1601 leader_election.cc:336] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:15.806309  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:15.461534  9756 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:15.852381  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52450.6us
I20260709 13:03:15.853121  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:15.853765  1634 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:15.853912  1601 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:15.854378  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:15.854545  9756 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:15.856242  9546 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:15.857558  9704 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:15.859090  9727 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 1.278s	user 0.019s	sys 0.007s
I20260709 13:03:15.860599  9739 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:15.861114  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3411) took 2170 ms. Trace:
I20260709 13:03:15.861538  1636 rpcz_store.cc:276] 0709 13:03:13.691054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.868994 (+177940us) service_pool.cc:224] Handling call
0709 13:03:15.860886 (+1991892us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b540010d6d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:15.861043 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4250062080}
I20260709 13:03:15.862063  9546 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.808609  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:15.808627  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:15.761091  9770 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:15.475078  9426 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:15.766376  9742 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.820188  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:15.822017  9732 log.cc:927] Time spent T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 1.475s	user 0.000s	sys 0.014s
I20260709 13:03:15.768124  9722 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:15.722610  9633 consensus_queue.cc:1059] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.140s
W20260709 13:03:15.799431  9750 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.032s	sys 0.057s
I20260709 13:03:15.798444  9494 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:15.681569  9754 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:15.838150  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 1312) took 2093 ms (client timeout 1504 ms). Trace:
W20260709 13:03:15.839057  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 170453us
W20260709 13:03:15.897377  1710 rpcz_store.cc:269] 0709 13:03:13.744661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.749780 (+  5119us) service_pool.cc:224] Handling call
0709 13:03:15.837940 (+2088160us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54000cbcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:15.838064 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1832054016}
I20260709 13:03:15.809540  9738 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:15.859017  9704 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:15.861407  1707 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:03:15.863631  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:15.864225  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1458) took 2971 ms. Trace:
I20260709 13:03:15.903863  1631 rpcz_store.cc:276] 0709 13:03:12.892615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.223495 (+1330880us) service_pool.cc:224] Handling call
0709 13:03:14.223880 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.087014 (+863134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.863952 (+776938us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b54001b7454. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:15.864160 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1686142208}
W20260709 13:03:15.814999  1675 leader_election.cc:336] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:15.861788  9756 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.875244  9770 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.798167  9667 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.837150  9404 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:15.890928  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:15.922830  2345 raft_consensus.cc:2393] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:15.896323  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:15.925034  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:15.834408  9520 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:15.822036  9545 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:15.905789  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 90795.2us
I20260709 13:03:15.907461  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3752) took 3902 ms. Trace:
I20260709 13:03:15.930963  1627 rpcz_store.cc:276] 0709 13:03:12.005576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.523176 (+1517600us) service_pool.cc:224] Handling call
0709 13:03:13.525517 (+  2341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:13.552965 (+ 27448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.553001 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:13.557118 (+  4117us) log.cc:844] Serialized 26 byte log entry
0709 13:03:13.903859 (+346741us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b18000498a0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245cfe07 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1 00007fda27695a0f
0709 13:03:14.208568 (+304709us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:14.208765 (+   197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.906986 (+1698221us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b5400198054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:15.907067 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.907136 (+    69us) raft_consensus.cc:1685] finished
0709 13:03:15.907164 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.907385 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5776,"raft.queue_time_us":17047,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":4322,"spinlock_wait_cycles":16518912,"thread_start_us":304107,"threads_started":1,"wal-append.queue_time_us":722574}
I20260709 13:03:15.874517  9425 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:15.862541  9546 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.909633  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:15.837839  9740 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.939114  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:15.940044  1631 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:15.880280  9638 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:15.915549  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51993.9us
I20260709 13:03:15.875072  9766 consensus_queue.cc:1059] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:03:15.903421  9554 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:15.882920  9742 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.818549  9672 consensus_queue.cc:260] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.933148  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:15.941318  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:15.946142  1634 raft_consensus.cc:2468] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 24.
I20260709 13:03:15.862852  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:15.948788  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:15.932221  9762 log.cc:927] Time spent T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.734s	user 0.001s	sys 0.001s
I20260709 13:03:15.883428  9426 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:15.921034  9770 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:15.953437  1675 leader_election.cc:336] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:15.998212  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44803.4us
I20260709 13:03:15.987027  1636 raft_consensus.cc:3060] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:03:15.987506  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.000620  1634 raft_consensus.cc:2410] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:03:15.928381  9722 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.991007  1631 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:15.993014  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:15.984658  9736 consensus_meta.cc:287] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.001s
W20260709 13:03:15.980777  9758 consensus_meta.cc:287] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.829s	user 0.009s	sys 0.000s
I20260709 13:03:16.009960  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.011368  1634 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:03:15.928334  9564 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:15.947635  9476 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:15.903970  9494 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:15.934777  9768 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.002493  1629 raft_consensus.cc:2410] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:03:16.019800  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.020460  1631 raft_consensus.cc:2468] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:16.020650  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.021452  1629 raft_consensus.cc:2410] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:03:16.004194  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.021929  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3416) took 1616 ms. Trace:
I20260709 13:03:16.022282  1631 rpcz_store.cc:276] 0709 13:03:14.405713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.021097 (+1615384us) service_pool.cc:224] Handling call
0709 13:03:16.021377 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.021590 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.021617 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.021662 (+    45us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:16.021696 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.021751 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.021876 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.988405  2317 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.001s	sys 0.017s
I20260709 13:03:15.948920  9567 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:15.954511  9404 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.921563  9756 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:16.015444  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 483439us
I20260709 13:03:16.032836  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3759) took 2119 ms. Trace:
I20260709 13:03:16.033191  1635 rpcz_store.cc:276] 0709 13:03:13.913591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.950546 (+1036955us) service_pool.cc:224] Handling call
0709 13:03:14.967174 (+ 16628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.967368 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.967394 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.967440 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:14.967471 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.967527 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.032656 (+1065129us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b5400197650. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:16.032773 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37418368}
W20260709 13:03:16.034168  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.036010  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3421) took 1072 ms. Trace:
I20260709 13:03:16.037001  1635 rpcz_store.cc:276] 0709 13:03:14.963458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.034492 (+1071034us) service_pool.cc:224] Handling call
0709 13:03:16.034944 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.035206 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.035244 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.035316 (+    72us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:16.035376 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.035445 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.035933 (+   488us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:16.026959  2317 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.001s	sys 0.017s
I20260709 13:03:15.991519  9709 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:15.984952  9667 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:15.999318  9294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.005s	sys 0.011s
W20260709 13:03:16.004354  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:16.041349  1634 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 16. (term mismatch)
I20260709 13:03:16.047541  1634 pending_rounds.cc:85] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Aborting all ops after (but not including) 15
I20260709 13:03:16.048110  1634 pending_rounds.cc:107] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Aborting uncommitted NO_OP operation due to leader change: 26.16
I20260709 13:03:16.048476  1634 raft_consensus.cc:2889] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:16.049527  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3420) took 1086 ms. Trace:
I20260709 13:03:16.015916  9758 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:16.051460  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 197445us
I20260709 13:03:16.052237  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.049926  1634 rpcz_store.cc:276] 0709 13:03:14.963076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.033844 (+1070768us) service_pool.cc:224] Handling call
0709 13:03:16.040847 (+  7003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.049126 (+  8279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.049430 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:16.073707  1601 leader_election.cc:336] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.073771  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.041781  1635 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:03:16.014048  9546 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:15.997759  9425 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:03:16.074303  1601 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:16.076273  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:16.079003  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 74670.7us
I20260709 13:03:16.078114  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3423) took 1113 ms. Trace:
I20260709 13:03:16.081001  1635 rpcz_store.cc:276] 0709 13:03:14.964272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.041220 (+1076948us) service_pool.cc:224] Handling call
0709 13:03:16.041439 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.075737 (+ 34298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.078050 (+  2313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.084978  1636 raft_consensus.cc:2468] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
W20260709 13:03:16.081574  1601 leader_election.cc:336] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.083753  2344 raft_consensus.cc:1240] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:16.084092  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.086627  2348 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:16.086165  1601 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:16.090093  2319 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.000s	sys 0.019s
I20260709 13:03:16.012810  9736 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.092053  1636 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 9. Preceding OpId from leader: term: 23 index: 10. (index mismatch)
I20260709 13:03:16.093845  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.094735  2348 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:16.093240  1634 raft_consensus.cc:1275] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:03:16.044627  9657 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.097256  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.028439  9722 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.050884  9234 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:16.045866  9739 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:16.099531  1601 leader_election.cc:336] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:16.047091  9294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.006s	sys 0.011s
I20260709 13:03:16.100451  9619 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.044190  9685 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:16.110870  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:16.084316  9775 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 13:03:16.076855  9746 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.108646  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:16.105319  1448 maintenance_manager.cc:419] P 8f622dabf84e4c0595473d2c01f108d0: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033346
I20260709 13:03:16.101579  9567 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:16.091637  9667 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:16.114020  2344 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:16.032074  9520 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.119465  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:16.123329  1601 leader_election.cc:336] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.129099  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:16.041966  9404 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:16.105734  9731 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.145246  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:16.115571  9619 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.150462  9619 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:16.150906  9438 consensus_meta.cc:287] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.728s	user 0.012s	sys 0.001s
I20260709 13:03:16.151445  9438 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:16.154609  9736 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.155768  9426 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:16.135151  9684 consensus_meta.cc:287] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.749s	user 0.007s	sys 0.008s
I20260709 13:03:16.119361  9740 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.146266  2347 raft_consensus.cc:2376] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22. Re-sending same reply.
I20260709 13:03:16.127565  9760 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.128098  9567 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:16.114773  9739 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.157825  9657 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:16.180514  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 147488us
W20260709 13:03:16.176206  9541 consensus_meta.cc:287] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.758s	user 0.014s	sys 0.000s
I20260709 13:03:16.188289  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:16.189818  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.109985  9234 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:16.203438  2318 leader_election.cc:336] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.185671  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:16.206317  2347 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:03:16.220263  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:16.201164  2317 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:16.222934  9740 raft_consensus.cc:2804] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:16.223359  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:16.223682  9740 raft_consensus.cc:697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:16.224265  9746 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:16.226956  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 278861us
W20260709 13:03:16.234057  9730 consensus_meta.cc:287] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 1.248s	user 0.012s	sys 0.003s
I20260709 13:03:16.244856  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.224922  9426 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:16.246990  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.204762  9731 consensus_queue.cc:1059] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 13:03:16.251626  1707 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:16.202348  9670 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:16.232048  9783 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:16.255702  9438 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:16.329862  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 69601.9us
I20260709 13:03:16.220984  9541 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.347474  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3775) took 1181 ms. Trace:
I20260709 13:03:16.257978  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:16.344686  1632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3771) took 2055 ms. Trace:
I20260709 13:03:16.244663  9742 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:16.236672  9766 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
I20260709 13:03:16.352284  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:16.233664  9294 consensus_queue.cc:1059] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
W20260709 13:03:16.351692  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 93757us
I20260709 13:03:16.249030  9494 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.237110  9746 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
W20260709 13:03:16.262677  9741 consensus_meta.cc:287] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.855s	user 0.015s	sys 0.000s
I20260709 13:03:16.356644  2318 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:16.259866  9653 consensus_meta.cc:287] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 1.164s	user 0.014s	sys 0.000s
I20260709 13:03:16.365612  9653 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.368832  9653 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.369442  9670 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.369827  9741 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.370198  9670 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:16.370338  9234 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.358841  9546 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.371482  9546 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.359066  9789 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:16.372098  9741 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.351184  9740 consensus_queue.cc:237] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.326773  9760 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.104s
I20260709 13:03:16.374058  9760 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:16.374429  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:16.376055  2318 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:16.364377  9787 log.cc:927] Time spent T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:03:16.373214  2353 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:16.359103  9296 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:16.355044  9770 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.377810  9785 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:16.377884  9296 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.378801  9785 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:16.349711  1632 rpcz_store.cc:276] 0709 13:03:14.289279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.643129 (+1353850us) service_pool.cc:224] Handling call
0709 13:03:15.704917 (+ 61788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.705244 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:16.033951 (+328707us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.344076 (+310125us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.344174 (+    98us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:16.344266 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.344380 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.344584 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11612,"prepare.queue_time_us":469121,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":52,"raft.queue_time_us":11806,"thread_start_us":623142,"threads_started":2}
I20260709 13:03:16.379631  9770 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.379562  9785 raft_consensus.cc:2804] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:16.350673  9756 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.380724  9296 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.348078  9784 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:03:16.256774  9730 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.383306  2349 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 26: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:16.355309  9685 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.384249  9730 leader_election.cc:290] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.380753  9785 raft_consensus.cc:697] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:16.381002  1627 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:16.380899  9722 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.385951  9785 consensus_queue.cc:237] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.387238  9541 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.379544  9404 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:16.388643  9546 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.378471  9739 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.374401  9760 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:16.390514  9760 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.384632  9685 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.393226  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.394985  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:16.402889  1710 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:16.403110  9685 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:16.406097  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3773) took 1827 ms. Trace:
I20260709 13:03:16.348629  1629 rpcz_store.cc:276] 0709 13:03:15.166357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.055122 (+888765us) service_pool.cc:224] Handling call
0709 13:03:16.055490 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.055795 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:16.228061 (+172266us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.346996 (+118935us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.347060 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:16.347118 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.347196 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.347323 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":123310,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":101,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":115,"thread_start_us":267536,"threads_started":2,"wal-append.queue_time_us":164380}
I20260709 13:03:16.361974  9783 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:16.409962  9783 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.410651  9783 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:16.412967  1710 raft_consensus.cc:2468] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
W20260709 13:03:16.347620  1601 leader_election.cc:336] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.523159  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:16.409116  1631 rpcz_store.cc:276] 0709 13:03:14.579141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.023234 (+1444093us) service_pool.cc:224] Handling call
0709 13:03:16.023526 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.023729 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:16.141714 (+117985us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.405634 (+263920us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.405702 (+    68us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:16.405768 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.405871 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.406019 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":155110,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":691,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":358295,"threads_started":2,"wal-append.queue_time_us":255912}
I20260709 13:03:16.415115  1707 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:03:16.529742  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 182015us
I20260709 13:03:16.417937  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:16.532022  1710 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:16.534687  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.535878  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.538409  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.539762  2352 raft_consensus.cc:2410] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:03:16.420994  9426 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:16.532665  1601 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.433197  9783 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.537271  2347 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:16.547689  9739 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
W20260709 13:03:16.548888  1601 leader_election.cc:336] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.550592  9545 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:16.570875  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:16.571295  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:16.572299  1707 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:16.572577  2344 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:16.576777  9528 raft_consensus.cc:3055] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:16.578018  9296 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.578317  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.579202  9296 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.590165  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.593552  9730 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.594643  9730 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.595059  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.596189  1706 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 23.
I20260709 13:03:16.577422  9528 raft_consensus.cc:740] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:16.598635  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.593904  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.600066  9783 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.600764  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:16.603116  1705 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:16.594060  9685 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:16.622731  9685 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.626904  9552 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.630489  1628 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:16.600478  9528 consensus_queue.cc:260] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.627990  9552 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:16.633054  9667 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:16.639786  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.643661  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:16.645030  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.654238  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:16.654816  9662 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.654242  1629 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:16.646275  1630 raft_consensus.cc:1275] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:16.656179  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.647838  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:16.658080  2351 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:16.659511  9554 raft_consensus.cc:3055] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:16.660049  9554 raft_consensus.cc:740] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:16.658103  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:16.662842  1628 raft_consensus.cc:3060] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:16.648753  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.662883  9730 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.645668  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54748 (request call id 3424) took 1680 ms. Trace:
I20260709 13:03:16.657227  1631 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:03:16.655233  2347 raft_consensus.cc:2368] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21.
I20260709 13:03:16.656185  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.668192  1633 rpcz_store.cc:276] 0709 13:03:14.964629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.119090 (+154461us) service_pool.cc:224] Handling call
0709 13:03:16.645060 (+1525970us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b5400343d54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:16.645361 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208724352}
I20260709 13:03:16.672170  9752 raft_consensus.cc:3055] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:16.672657  9752 raft_consensus.cc:740] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:16.673061  9296 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.673449  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.674629  2348 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
I20260709 13:03:16.678216  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.679427  9672 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:16.681751  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1443) took 1152 ms. Trace:
I20260709 13:03:16.682017  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:16.682471  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.682279  2346 rpcz_store.cc:276] 0709 13:03:15.529362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.665640 (+136278us) service_pool.cc:224] Handling call
0709 13:03:16.681232 (+1015592us) spinlock_profiling.cc:243] Waited 87.2 ms on lock 0x7b54004bf0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:16.681537 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200694784}
I20260709 13:03:16.682999  9528 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:16.681149  9754 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.685343  1676 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:16.688480  9404 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:16.689386  9495 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.689822  9638 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:16.690431  9638 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.690243  9495 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:16.690884  9638 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:16.690728  9541 consensus_queue.cc:1059] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:16.692570  9653 raft_consensus.cc:3055] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:16.693051  9653 raft_consensus.cc:740] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:16.693825  9495 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:16.695221  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:16.694880  9653 consensus_queue.cc:260] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.687139  9752 consensus_queue.cc:260] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.696221  2345 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:16.697497  9752 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:16.683015  2344 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:16.698465  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:16.699323  2348 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:16.700542  9554 consensus_queue.cc:260] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:16.702589  9554 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:16.702924  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3766) took 2787 ms. Trace:
I20260709 13:03:16.703822  1630 rpcz_store.cc:276] 0709 13:03:13.915344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.431836 (+1516492us) service_pool.cc:224] Handling call
0709 13:03:16.645349 (+1213513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.655940 (+ 10591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.702671 (+ 46731us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400343d50. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276d98ad 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:16.702833 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2788160896}
I20260709 13:03:16.706554  9685 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.689437  9404 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.710493  9552 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.709211  9743 raft_consensus.cc:3055] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:16.714004  9743 raft_consensus.cc:740] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:16.718609  9743 consensus_queue.cc:260] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.722777  9743 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:16.723469  9746 raft_consensus.cc:3055] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:16.713151  9404 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:16.732424  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 183609us
I20260709 13:03:16.735816  9778 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:16.736658  9792 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.383s	user 0.004s	sys 0.001s
I20260709 13:03:16.737762  1707 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:03:16.736686  9778 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.747542  1710 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 26.
I20260709 13:03:16.750741  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.751726  1705 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 27.
I20260709 13:03:16.729167  9746 raft_consensus.cc:740] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:16.738997  9495 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
W20260709 13:03:16.755673  1601 leader_election.cc:336] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.755193  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:16.764165  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.764657  2353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59274 (request call id 1436) took 1294 ms. Trace:
I20260709 13:03:16.744201  9778 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:16.767095  9746 consensus_queue.cc:260] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:16.769338  2318 leader_election.cc:336] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.762179  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.774025  9653 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:16.767689  9404 consensus_queue.cc:1048] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:03:16.776021  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:16.779460  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.786532  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.787333  1707 raft_consensus.cc:2393] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 26: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:16.781563  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.783478  2353 rpcz_store.cc:276] 0709 13:03:15.470140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.470667 (+   527us) service_pool.cc:224] Handling call
0709 13:03:16.756780 (+1286113us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b540055ffd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:16.764391 (+  7611us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1958734720}
I20260709 13:03:16.771001  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.788758  1630 raft_consensus.cc:2368] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21.
I20260709 13:03:16.790767  2318 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:16.792088  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:16.781005  1455 catalog_manager.cc:5697] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 15 to 16, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:16.797868  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:16.801177  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.802397  1627 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:03:16.806797  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:16.807839  1634 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:16.810256  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:16.811283  1632 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:16.813661  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 50140.1us
I20260709 13:03:16.813908  9564 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.814780  1525 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:16.814637  9564 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.816246  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:16.817987  9744 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:16.818158  9644 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:16.818540  9744 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.819094  9744 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:16.819504  9786 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.819653  1633 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:16.820292  9786 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:16.822520  1676 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:03:16.823442  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 47786.9us
I20260709 13:03:16.824186  9564 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:16.838872  1705 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
I20260709 13:03:16.840031  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:16.840939  2346 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
I20260709 13:03:16.841437  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:16.841951  9768 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:16.841958  9737 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:16.846351  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.847024  1631 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 20.
W20260709 13:03:16.845175  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 111491us
W20260709 13:03:16.848897  1601 leader_election.cc:336] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.849525  1601 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:16.849843  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:16.851447  1601 leader_election.cc:336] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.850785  1455 catalog_manager.cc:5697] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 14, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 14 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:16.852067  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:16.853304  9561 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:16.853739  1706 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:16.857376  9296 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:16.857890  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:16.853780  9768 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:16.860325  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.870301  1709 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:16.869683  1634 raft_consensus.cc:2468] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:16.852862  1455 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:16.871114  2348 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:16.875660  2344 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:16.848959  9737 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.876965  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:16.877370  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61276.9us
I20260709 13:03:16.888139  1632 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:16.882824  1629 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:03:16.857921  9296 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:16.897579  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.899283  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.902117  1627 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:16.894394  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:16.900485  1601 leader_election.cc:336] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.903796  1630 raft_consensus.cc:2393] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 18: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:16.907383  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:16.908986  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.913621  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43820.1us
W20260709 13:03:16.915377  1453 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 275) took 52577 ms (client timeout 15000 ms). Trace:
I20260709 13:03:16.917927  9760 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:16.918462  1455 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 13, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 13 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:16.942142  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 128948us
I20260709 13:03:16.944068  1631 raft_consensus.cc:2393] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:03:16.934264  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 218769us
I20260709 13:03:16.955842  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:16.887277  9737 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:16.958318  2345 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 24.
I20260709 13:03:16.958814  1633 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:16.973922  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:16.921424  1455 catalog_manager.cc:5697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 15 to 17, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 17 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:16.985698  1628 raft_consensus.cc:2468] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 26.
I20260709 13:03:16.962656  1706 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 23.
W20260709 13:03:16.996268  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 104398us
I20260709 13:03:16.901638  9296 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:17.002538  2350 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:03:17.004626  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:17.006114  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 44067.5us
I20260709 13:03:17.009284  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:17.005672  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 105980us
W20260709 13:03:17.013836  2318 leader_election.cc:336] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:16.942694  9746 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:17.016541  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:17.017349  1627 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 25.
I20260709 13:03:17.020458  9234 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:17.020946  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.021672  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 61977.3us
I20260709 13:03:17.026190  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:17.014601  2317 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:17.030176  1628 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:17.036427  1634 raft_consensus.cc:2393] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 24: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:03:17.037631  1675 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:17.042131  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:03:17.042248  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 44110.4us
I20260709 13:03:17.043532  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:17.046573  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:17.047274  1627 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:17.047937  9740 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:17.048525  9740 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:17.049031  9740 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:17.051141  9633 consensus_queue.cc:1048] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:03:17.060349  1527 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:17.040957  1455 catalog_manager.cc:5697] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:17.059715  1675 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:17.068364  1632 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:17.069802  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:17.074419  2351 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:17.076655  1706 raft_consensus.cc:2368] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21.
I20260709 13:03:17.057614  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:17.058945  9234 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:03:17.089679  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 202513us
I20260709 13:03:17.041337  9786 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:17.108616  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.083161  1455 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 16 to 17, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 17 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:17.112950  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 80025.8us
I20260709 13:03:17.116693  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:17.093532  9234 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:17.123217  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.132982  1627 raft_consensus.cc:1240] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
W20260709 13:03:17.134215  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.137423  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:17.119016  9752 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:17.139325  2345 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 22. Current term is 23. Ops: []
W20260709 13:03:17.142189  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 103668us
W20260709 13:03:17.145061  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 45580.5us
I20260709 13:03:17.147567  1602 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:17.148586  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:17.150276  1447 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.332s	user 0.030s	sys 0.064s
I20260709 13:03:17.154915  1455 catalog_manager.cc:5697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 14 to 16, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:17.156416  9426 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:17.156939  9426 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:17.145184  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 112677us
W20260709 13:03:17.162765  1675 leader_election.cc:336] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:17.165208  2349 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:03:17.125653  9234 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:17.177973  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:17.178992  1701 raft_consensus.cc:2393] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:17.178841  9644 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:17.180117  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:17.181519  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:17.135749  9564 consensus_queue.cc:1059] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 13:03:17.182922  9564 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:17.183336  9564 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:17.184461  9564 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:17.185148  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 178109us
I20260709 13:03:17.182934  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:17.186105  2318 leader_election.cc:336] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:17.187786  2318 leader_election.cc:336] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:17.188256  9528 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.001s
I20260709 13:03:17.176999  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:17.157331  9426 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:17.132933  9404 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:17.196029  9734 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:17.196676  9734 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.197103  9734 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:17.188251  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:17.120117  9744 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:17.218776  2351 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:03:17.228479  1706 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:03:17.231232  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 68378us
W20260709 13:03:17.232211  1675 leader_election.cc:336] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:17.232659  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:17.230406  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:17.237457  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:17.223278  9564 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:17.245846  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:17.230545  9672 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.241380  9778 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:17.258117  1636 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:03:17.258917  1628 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:03:17.242872  9754 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:17.262603  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 74845.9us
I20260709 13:03:17.264277  2317 leader_election.cc:304] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:17.259418  9778 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.260848  9768 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:17.261034  9541 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:17.242674  9633 consensus_queue.cc:1048] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:03:17.259896  9672 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:17.281409  9741 raft_consensus.cc:2804] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:03:17.281914  9754 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:17.271548  9778 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:17.280480  2318 leader_election.cc:336] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:17.285684  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 53489us
W20260709 13:03:17.289418  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.290460  1674 leader_election.cc:400] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21.
I20260709 13:03:17.290761  1674 leader_election.cc:403] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 13:03:17.288980  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:17.291777  2345 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
I20260709 13:03:17.295321  2318 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:17.291896  9638 consensus_meta.cc:287] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.600s	user 0.016s	sys 0.000s
I20260709 13:03:17.321595  1705 raft_consensus.cc:1240] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:17.317658  9754 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:17.322235  9672 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:17.323068  9672 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:17.323902  9672 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.324384  9672 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:17.325152  9754 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:17.326725  9672 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:17.330647  1701 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:03:17.311530  9638 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:17.323313  9810 log.cc:927] Time spent T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:03:17.340798  1705 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:03:17.347689  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:17.353430  1704 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:17.354001  1701 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 13:03:17.354453  1701 pending_rounds.cc:85] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 14
I20260709 13:03:17.309787  9296 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:17.354678  1701 pending_rounds.cc:107] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 13:03:17.360384  1701 raft_consensus.cc:2889] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:17.362139  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 51910.4us
W20260709 13:03:17.378352  9811 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:03:17.386139  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:17.387604  1674 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:17.388739  1674 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:17.366340  9776 raft_consensus.cc:2804] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:17.389705  9776 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.390192  9776 raft_consensus.cc:3060] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:17.374083  9741 raft_consensus.cc:697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:17.382962  9744 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:17.389806  9743 consensus_meta.cc:287] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.007s
I20260709 13:03:17.420775  9633 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:17.413969  9404 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:17.421783  9633 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:17.422511  9404 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.422973  9404 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:17.424804  9633 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:17.426476  9495 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:17.427150  9495 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:17.428809  9426 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:17.401355  9296 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:17.432051  9426 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:17.434053  9741 consensus_queue.cc:237] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:17.437645  9638 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:17.446496  9561 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.446832  9752 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:17.447296  9561 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:17.447824  9734 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:17.448542  9752 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:17.450166  9734 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:17.451606  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:17.452718  1629 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22.
I20260709 13:03:17.453161  9644 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:17.453941  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:17.157226  1455 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:17.456735  1455 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 10 to 18, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 18 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:17.458976  9438 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097) with higher term. Message: Invalid argument: T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21.
I20260709 13:03:17.460214  9653 raft_consensus.cc:2804] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:17.460793  9653 raft_consensus.cc:697] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:17.461689  9743 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:17.461869  9653 consensus_queue.cc:237] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:17.464923  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:17.467096  9495 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:17.466696  9561 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:17.471895  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61022.5us
I20260709 13:03:17.473198  1674 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:17.473896  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:17.474788  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:17.476249  9438 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:17.414896  9737 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:17.458521  1455 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 12 to 17, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:17.440819  9234 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:17.516578  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.967494  2317 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:17.557699  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 104177us
I20260709 13:03:17.549964  1455 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 16 to 18, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 18 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:03:16.916118  1453 rpcz_store.cc:269] 0709 13:02:24.337663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.338121 (+   458us) service_pool.cc:224] Handling call
0709 13:03:16.914960 (+52576839us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:10.676171 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:10.677000 (+   829us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:10.677091 (+    91us) write_op.cc:435] Acquired schema lock
0709 13:03:10.677113 (+    22us) tablet.cc:662] Decoding operations
0709 13:03:10.678492 (+  1379us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:10.678622 (+   130us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:10.678656 (+    34us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:03:10.679929 (+  1273us) tablet.cc:701] Row locks acquired
0709 13:03:10.679988 (+    59us) write_op.cc:260] PREPARE: finished
0709 13:03:10.680212 (+   224us) write_op.cc:270] Start()
0709 13:03:10.680442 (+   230us) write_op.cc:276] Timestamp: P: 1783602190680139 usec, L: 0
0709 13:03:10.680471 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:03:10.681374 (+   903us) log.cc:844] Serialized 4721 byte log entry
0709 13:03:11.244392 (+563018us) op_driver.cc:464] REPLICATION: finished
0709 13:03:11.428203 (+183811us) write_op.cc:301] APPLY: starting
0709 13:03:11.428349 (+   146us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:11.875047 (+446698us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:11.896300 (+ 21253us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:12.344333 (+448033us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:12.344762 (+   429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 13:03:12.344818 (+    56us) write_op.cc:312] APPLY: finished
0709 13:03:12.345294 (+   476us) log.cc:844] Serialized 105 byte log entry
0709 13:03:13.001431 (+656137us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda245d00ac 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b
0709 13:03:13.418391 (+416960us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:13.418557 (+   166us) write_op.cc:454] Released schema lock
0709 13:03:13.418657 (+   100us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:13.419172 (+   515us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:13.419211 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9049367,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":164484,"apply.run_cpu_time_us":43664,"apply.run_wall_time_us":2009931,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50107,"num_ops":11,"prepare.queue_time_us":275645,"prepare.run_cpu_time_us":22015,"prepare.run_wall_time_us":206469,"raft.queue_time_us":271868,"raft.run_cpu_time_us":36947,"raft.run_wall_time_us":129727,"replication_time_us":563645,"spinlock_wait_cycles":49576192,"thread_start_us":1561853,"threads_started":5,"wal-append.queue_time_us":1240142,"wal-append.run_cpu_time_us":16715,"wal-append.run_wall_time_us":2247716}]]}
I20260709 13:03:17.970899  1525 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:17.644753  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 369956us
I20260709 13:03:17.699867  9438 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:17.710997  1447 maintenance_manager.cc:643] P 8f622dabf84e4c0595473d2c01f108d0: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 1.520s	user 0.068s	sys 0.078s Metrics: {"bytes_written":4750,"delete_count":0,"lbm_write_time_us":5334,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":3,"reinsert_count":0,"thread_start_us":138847,"threads_started":1,"update_count":176,"wal-append.queue_time_us":273171}
W20260709 13:03:17.736189  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.746155  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 272615us
I20260709 13:03:17.783470  9528 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:17.785531  9115 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:17.793525  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:17.795591  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:17.798381  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:17.855702  9426 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.857167  9760 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:03:17.894272  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.915715  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:17.932325  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:18.006500  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.943629  9815 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.945406  9813 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:17.973410  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:18.027117  2351 raft_consensus.cc:2410] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:17.980588  1525 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:17.983033  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:17.984050  2318 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 119 similar messages]
I20260709 13:03:17.988807  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:17.989602  9564 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:17.991000  1630 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:17.994448  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:17.993806  1455 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:18.008244  1701 raft_consensus.cc:2393] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:18.033159  2347 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:18.036202  1633 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:03:18.007059  9404 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: Time spent flushing consensus metadata: real 0.583s	user 0.015s	sys 0.004s
I20260709 13:03:17.973963  9234 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:18.038462  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:17.998375  9115 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.042572  1710 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
I20260709 13:03:18.056250  2346 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:18.054015  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:18.056416  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:18.045506  9813 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:18.051232  9115 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:18.079880  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:18.050161  9495 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.025694  9426 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:18.007932  9744 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:03:18.097831  1448 maintenance_manager.cc:419] P 8f622dabf84e4c0595473d2c01f108d0: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.203627
W20260709 13:03:18.079993  9778 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.773s	user 0.021s	sys 0.002s
I20260709 13:03:18.099306  9778 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:18.079030  9234 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:18.101641  9234 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.102101  9234 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:18.104909  1631 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:18.102675  9778 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:18.118024  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:18.053277  9815 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:18.125502  9815 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:18.048848  9404 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:18.123262  9426 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:18.151834  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:18.152765  2344 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
I20260709 13:03:18.162263  9768 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:18.169606  9752 raft_consensus.cc:3060] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:18.102807  9495 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:18.123262  9813 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:18.196815  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:18.185326  1705 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:18.185334  2346 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
W20260709 13:03:18.181209  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 197116us
I20260709 13:03:18.205828  1631 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
W20260709 13:03:18.194329  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 171713us
I20260709 13:03:18.149734  9426 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.244549  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:18.233618  9744 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:18.251153  9744 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.251704  9744 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:18.253829  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:18.255414  1455 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 14 to 15, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 15 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:18.253854  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 273214us
I20260709 13:03:18.259763  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:18.257390  1455 catalog_manager.cc:5697] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:18.321146  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 46578.2us
W20260709 13:03:18.323124  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 263791us
I20260709 13:03:18.336021  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:18.335321  9760 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:03:18.356608  2318 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:18.358285  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:18.334769  1455 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:18.365173  1525 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:18.246598  9426 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:18.329841  9564 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:18.416180  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 73659.4us
I20260709 13:03:18.358103  9760 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:18.445744  9760 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:18.452438  9737 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:18.437248  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:18.454100  1636 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:18.454979  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:18.456082  1632 raft_consensus.cc:2410] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 27 because replica has last-logged OpId of term: 26 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 13:03:18.457031  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:18.444411  9746 raft_consensus.cc:3060] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:18.455118  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:18.459576  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:18.459627  1632 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:18.405710  1455 catalog_manager.cc:5697] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 2 to 3, term changed from 15 to 17, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194), VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 17 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:18.459080  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:18.477078  1636 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:18.482681  1455 catalog_manager.cc:5697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 16 to 18, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 18 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:18.427155  9813 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:18.503592  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 180278us
I20260709 13:03:18.468713  9778 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:18.516248  9404 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:18.516499  9752 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:18.519863  9776 consensus_meta.cc:287] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 1.129s	user 0.005s	sys 0.009s
W20260709 13:03:18.529120  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 205195us
I20260709 13:03:18.531263  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:18.485242  9737 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:18.553117  1582 debug-util.cc:398] Leaking SignalData structure 0x7b08002a6500 after lost signal to thread 434
W20260709 13:03:18.589058  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 310019us
I20260709 13:03:18.590399  1455 catalog_manager.cc:5697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:18.609166  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 96370.6us
W20260709 13:03:18.609697  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 270404us
I20260709 13:03:18.499063  9564 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:18.511221  9813 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:18.622376  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.624423  9426 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:18.634358  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 174772us
I20260709 13:03:18.598874  9815 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:18.632627  1455 catalog_manager.cc:5697] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 15 to 19, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 19 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:18.618261  9752 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.668521  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:18.663113  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:18.598223  9404 consensus_queue.cc:1059] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:03:18.582139  9776 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:18.598664  9737 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:18.683545  2318 leader_election.cc:336] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:18.651535  9815 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.639429  9760 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:18.710198  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 72512.7us
I20260709 13:03:18.710048  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:18.704926  1455 catalog_manager.cc:5697] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:18.701094  9815 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:18.746228  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 77782.6us
W20260709 13:03:18.749496   431 thread.cc:527] Waited for 1000ms trying to join with maintenance_scheduler (tid 1587)
I20260709 13:03:18.723377  9778 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:18.752328  1676 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.002s	sys 0.022s
I20260709 13:03:18.740620  9754 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:18.682896  9752 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:18.761945  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 94017.2us
W20260709 13:03:18.757836  1676 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.003s	sys 0.022s
W20260709 13:03:18.762921  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 622852us
W20260709 13:03:18.741063  9234 consensus_meta.cc:287] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.638s	user 0.019s	sys 0.002s
I20260709 13:03:18.736568  9426 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:03:18.778213  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 104048us
I20260709 13:03:18.741722  9404 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:03:18.771520  9115 consensus_meta.cc:287] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.668s	user 0.007s	sys 0.007s
I20260709 13:03:18.796392  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:18.803848  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 47513.3us
W20260709 13:03:18.804344  2318 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:18.811363  9752 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:18.811691  9737 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:18.812223  9752 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:18.812971  9752 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.766235  9778 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:18.790165  9744 consensus_meta.cc:287] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.538s	user 0.013s	sys 0.000s
I20260709 13:03:18.832664  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:18.875286  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 72378.2us
I20260709 13:03:18.777419  9234 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:18.894138  1601 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:18.881136  1675 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.581s (SENT)
W20260709 13:03:18.889842  2318 leader_election.cc:336] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:18.795226  9404 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:18.898470  9768 consensus_meta.cc:287] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.733s	user 0.018s	sys 0.000s
I20260709 13:03:18.813454  9752 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:18.942996  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 115903us
I20260709 13:03:18.803315  9115 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:18.951210  1706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.541s	user 0.023s	sys 0.017s
W20260709 13:03:18.962988  1706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.553s	user 0.024s	sys 0.018s
I20260709 13:03:18.845144  9744 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:18.994185  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:18.995915  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1839) took 1002 ms. Trace:
W20260709 13:03:19.034207  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 153041us
I20260709 13:03:18.976786  9541 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:19.051638  1706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3518) took 1752 ms. Trace:
I20260709 13:03:18.976673  9561 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:03:19.017043  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 78440us
I20260709 13:03:19.084794  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:19.113641  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3801) took 1115 ms. Trace:
W20260709 13:03:19.138676  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.084590  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 107454us
I20260709 13:03:19.084077  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3792) took 1141 ms. Trace:
I20260709 13:03:18.965040  9768 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:18.980559  9746 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:19.164673  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:18.980525  9813 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.151432  1706 rpcz_store.cc:276] 0709 13:03:17.263761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.264181 (+   420us) service_pool.cc:224] Handling call
0709 13:03:17.279843 (+ 15662us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.313900 (+ 34057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.313928 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:17.320954 (+  7026us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.963736 (+1642782us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:18.963806 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.963951 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.016171 (+ 52220us) raft_consensus.cc:1685] finished
0709 13:03:19.016199 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.016410 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":658,"raft.queue_time_us":6390,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":4878,"thread_start_us":1553671,"threads_started":1,"wal-append.queue_time_us":1623465}
W20260709 13:03:19.174043  1675 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.584s (SENT)
I20260709 13:03:19.016083  2349 rpcz_store.cc:276] 0709 13:03:17.975780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.346533 (+370753us) service_pool.cc:224] Handling call
0709 13:03:18.346994 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.347259 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:18.529342 (+182083us) log.cc:844] Serialized 14 byte log entry
0709 13:03:18.977236 (+447894us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:18.977311 (+    75us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:18.977384 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.977470 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.977659 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23631,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":21963,"thread_start_us":605184,"threads_started":2}
W20260709 13:03:19.176191  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 49674us
W20260709 13:03:19.177688  1675 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.627s (SENT)
I20260709 13:03:19.179744  1633 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 18.
I20260709 13:03:19.116384  9722 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:19.086526  9760 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.079859  9837 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.086118  9778 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.151921  1629 rpcz_store.cc:276] 0709 13:03:17.943776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.944023 (+   247us) service_pool.cc:224] Handling call
0709 13:03:17.944555 (+   532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.945236 (+   681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.945272 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:17.946185 (+   913us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.128532 (+182347us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:18.128603 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.128719 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.083751 (+955032us) raft_consensus.cc:1685] finished
0709 13:03:19.083776 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.084008 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3551,"raft.run_cpu_time_us":396,"raft.run_wall_time_us":26171,"thread_start_us":161750,"threads_started":1,"wal-append.queue_time_us":1135551}
I20260709 13:03:19.080397  9847 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.184415  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:19.083087  9744 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:19.078294  9404 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:19.181272  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 96023.8us
I20260709 13:03:19.114214  9115 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.085121  9853 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:19.150988  1630 rpcz_store.cc:276] 0709 13:03:17.973441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.124060 (+150619us) service_pool.cc:224] Handling call
0709 13:03:18.124562 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.125065 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.125101 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.125791 (+   690us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.566474 (+440683us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:18.566668 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.566955 (+   287us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.088234 (+521279us) raft_consensus.cc:1685] finished
0709 13:03:19.088264 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.088427 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32297,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":439618,"threads_started":1,"wal-append.queue_time_us":951446}
W20260709 13:03:19.178589  1675 leader_election.cc:336] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.677s (SENT)
I20260709 13:03:19.195118  1631 raft_consensus.cc:3060] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:19.226413  1675 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:19.117079  9752 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:19.020138  9426 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:19.087142  9829 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:19.075904  9754 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:19.076639  9234 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.076814  9564 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:19.210525  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:19.211758  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:19.272871  1633 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:19.187716  9737 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:19.222680  9561 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:19.259120  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3519) took 1978 ms. Trace:
I20260709 13:03:19.280328  1707 rpcz_store.cc:276] 0709 13:03:17.280446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.280701 (+   255us) service_pool.cc:224] Handling call
0709 13:03:17.291952 (+ 11251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.316513 (+ 24561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.316568 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:17.321368 (+  4800us) log.cc:844] Serialized 26 byte log entry
0709 13:03:17.508459 (+187091us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:17.508530 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.508638 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.258828 (+1750190us) raft_consensus.cc:1685] finished
0709 13:03:19.258854 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.259054 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11771,"raft.queue_time_us":3968,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":1815,"thread_start_us":182289,"threads_started":1,"wal-append.queue_time_us":1931017}
I20260709 13:03:19.259064  9829 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.266335  9752 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.300601  9564 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:19.307157  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 117833us
I20260709 13:03:19.285910  9778 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:19.297166  9829 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:19.297192  9858 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:03:19.320228  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 99937.3us
I20260709 13:03:19.294171  9737 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:19.332998  9817 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.074s	user 0.001s	sys 0.001s
I20260709 13:03:19.313088  9778 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:19.333523  2318 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:19.224999  9837 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.276233  9426 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.300948  9561 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.361303  1631 raft_consensus.cc:2468] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:03:19.362264  9754 raft_consensus.cc:2804] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:03:19.363078  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 72646.2us
W20260709 13:03:19.363128  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 184468us
I20260709 13:03:19.363971  9835 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:03:19.364580  9836 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:03:19.374143  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 210556us
I20260709 13:03:19.335845  9858 raft_consensus.cc:697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 23 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:19.351891  9778 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:19.406987  9737 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:19.320221  9752 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:19.377169  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.378576  1636 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.008s	sys 0.004s
I20260709 13:03:19.380872  1633 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:03:19.389326  1675 leader_election.cc:336] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.698s (SENT)
W20260709 13:03:19.410923  1675 leader_election.cc:336] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.702s (SENT)
I20260709 13:03:19.394586  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:19.388005  9744 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.403169  9754 raft_consensus.cc:697] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:19.403337  1627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3810) took 1152 ms. Trace:
W20260709 13:03:19.403818  9828 log.cc:927] Time spent T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.064s	user 0.001s	sys 0.001s
W20260709 13:03:19.404018  9815 consensus_meta.cc:287] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.004s
I20260709 13:03:19.404630  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:19.405956  9858 consensus_queue.cc:237] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.411270  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:19.410362  1636 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.009s	sys 0.004s
I20260709 13:03:19.223556  9847 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:19.409251  9839 log.cc:927] Time spent T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.107s	user 0.001s	sys 0.001s
W20260709 13:03:19.420856  1675 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.729s (SENT)
I20260709 13:03:19.415320  2350 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59258 (request call id 1838) took 1442 ms. Trace:
I20260709 13:03:19.414177  9760 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.433153  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 457145us
I20260709 13:03:19.393812  9856 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.434155  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:19.422752  2350 rpcz_store.cc:276] 0709 13:03:17.969129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.972193 (+  3064us) service_pool.cc:224] Handling call
0709 13:03:17.992371 (+ 20178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.992836 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.992867 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:17.993424 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.088369 (+ 94945us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:18.088412 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.088500 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.411488 (+1322988us) raft_consensus.cc:1685] finished
0709 13:03:19.411520 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.411686 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119588,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":398,"thread_start_us":94399,"threads_started":1,"wal-append.queue_time_us":1329538}
I20260709 13:03:19.413678  9825 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.414986  1627 rpcz_store.cc:276] 0709 13:03:18.250828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.524355 (+273527us) service_pool.cc:224] Handling call
0709 13:03:18.524618 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.525001 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:18.779319 (+254318us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.398613 (+619294us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:19.398672 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:19.402999 (+  4327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.403100 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.403265 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":152812,"prepare.queue_time_us":852201,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":67,"raft.queue_time_us":533482,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":6749,"spinlock_wait_cycles":8060672,"thread_start_us":666446,"threads_started":3,"wal-append.queue_time_us":391475}
I20260709 13:03:19.434227  9234 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:19.435425  9840 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.435078  9853 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.462742  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46210 (request call id 3517) took 2204 ms. Trace:
I20260709 13:03:19.466523  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:19.417331  9744 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:19.470032  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 65493.4us
I20260709 13:03:19.445209  9404 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.424041  9815 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.466195  1709 rpcz_store.cc:276] 0709 13:03:17.253361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.253522 (+   161us) service_pool.cc:224] Handling call
0709 13:03:17.260857 (+  7335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.268232 (+  7375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.268270 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:17.280241 (+ 11971us) log.cc:844] Serialized 26 byte log entry
0709 13:03:17.627462 (+347221us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:17.627511 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.627594 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.456047 (+1828453us) raft_consensus.cc:1685] finished
0709 13:03:19.456072 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.456234 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3803,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":2929,"thread_start_us":309924,"threads_started":1,"wal-append.queue_time_us":1959308}
I20260709 13:03:19.447717  9746 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:19.436494  9760 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:19.496559  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 62470.3us
W20260709 13:03:19.468884  9741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.024s	user 0.010s	sys 0.026s
I20260709 13:03:19.498667  1526 leader_election.cc:400] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21.
W20260709 13:03:19.497339  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 52517.3us
I20260709 13:03:19.499797  1526 leader_election.cc:403] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:03:19.499725  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:19.504344  9831 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.505013  9831 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.500535  9853 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.509536  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:19.482535  9561 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:19.469372  9234 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:19.510438  1710 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:19.512779  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3794) took 1558 ms. Trace:
I20260709 13:03:19.483155  9860 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:19.514403  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.515277  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.515888  9768 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:19.516455  9859 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.516659  9862 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:03:19.443933  9856 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:19.500974  9741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.056s	user 0.011s	sys 0.027s
I20260709 13:03:19.468048  9840 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.517218  9866 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.536944  1709 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 22: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:19.536620  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.518778  1635 rpcz_store.cc:276] 0709 13:03:17.954700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.954863 (+   163us) service_pool.cc:224] Handling call
0709 13:03:17.991673 (+ 36810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.291026 (+299353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.291314 (+   288us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.343172 (+ 51858us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.500921 (+157749us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:18.500970 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.501068 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.512501 (+1011433us) raft_consensus.cc:1685] finished
0709 13:03:19.512529 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.512703 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36394,"raft.queue_time_us":132530,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":85,"thread_start_us":344081,"threads_started":2,"wal-append.queue_time_us":803943}
I20260709 13:03:19.540778  1628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48246 (request call id 1491) took 1589 ms. Trace:
I20260709 13:03:19.542937  1706 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 23.
I20260709 13:03:19.448738  9825 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.511628  9752 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.562909  9752 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.563596  9752 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.495270  9404 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.572463  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:19.548777  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 127918us
I20260709 13:03:19.553172  1707 raft_consensus.cc:2410] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:19.575834  9868 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.538597  9746 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:19.576450  9868 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:19.575873  1675 leader_election.cc:336] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.760s (SENT)
I20260709 13:03:19.579456  9837 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:19.579774  1675 leader_election.cc:336] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.843s (SENT)
W20260709 13:03:19.580868  1675 leader_election.cc:336] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.939s (SENT)
I20260709 13:03:19.563639  9866 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:19.582211  1675 leader_election.cc:336] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.940s (SENT)
W20260709 13:03:19.583170  1675 leader_election.cc:336] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.954s (SENT)
I20260709 13:03:19.583462  9866 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.584978  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.585160  9815 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.585872  1706 raft_consensus.cc:2393] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 19: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:03:19.586105  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 114932us
I20260709 13:03:19.552847  1628 rpcz_store.cc:276] 0709 13:03:17.951316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.951487 (+   171us) service_pool.cc:224] Handling call
0709 13:03:17.990177 (+ 38690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.027826 (+ 37649us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:18.027864 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.066503 (+ 38639us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.285819 (+219316us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:18.285879 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.285982 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.540395 (+1254413us) raft_consensus.cc:1685] finished
0709 13:03:19.540436 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.540689 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20362,"raft.queue_time_us":133876,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":155568,"threads_started":1,"wal-append.queue_time_us":1124147}
I20260709 13:03:19.586884  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:19.539573  9754 consensus_queue.cc:237] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.558684  9851 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.559276  9833 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.561517  9834 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.561909  9561 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:19.603283  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.604635  1703 raft_consensus.cc:2410] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:03:19.567742  9234 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.624743  9856 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.568300  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:19.590759  1710 raft_consensus.cc:2468] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 24.
I20260709 13:03:19.593281  1706 raft_consensus.cc:1240] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:19.593165  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:19.593577  9740 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 2.544s	user 0.013s	sys 0.010s
W20260709 13:03:19.555223  1601 connection.cc:256] RPC call timeout handler was delayed by -1.0193237390000149s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1507s for a response
I20260709 13:03:19.595901  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:19.536639  9768 raft_consensus.cc:493] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.601552  9722 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:19.594094  9833 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.609215  9564 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:19.565871  9860 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.617837  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.593974  9851 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.616143  9561 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.551215  9859 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.616086  9834 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.629614  1628 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:03:19.627930  1525 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:19.667673  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.632731  1627 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 24: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:19.635373  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.669035  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.669435  1710 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
I20260709 13:03:19.663305  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:19.672052  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 85201.9us
I20260709 13:03:19.673978  1709 raft_consensus.cc:2410] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:03:19.665427  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:19.633495  9768 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:19.633258  9740 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.677814  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.679903  1627 raft_consensus.cc:1240] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:19.680652  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:19.675678  1601 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.507s (SENT)
I20260709 13:03:19.682251  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:19.683687  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:19.678896  9426 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:19.685362  2346 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
I20260709 13:03:19.686021  1627 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:19.688838  9858 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:19.691552  9561 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:19.692494  9864 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.154s	user 0.002s	sys 0.000s
I20260709 13:03:19.693944  9528 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:19.694492  9528 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:19.694773  1629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3450) took 1241 ms. Trace:
I20260709 13:03:19.695194  1629 rpcz_store.cc:276] 0709 13:03:18.453876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.252119 (+798243us) service_pool.cc:224] Handling call
0709 13:03:19.276061 (+ 23942us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.288901 (+ 12840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.288946 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.302980 (+ 14034us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.415720 (+112740us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:19.415788 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.415897 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.691541 (+275644us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda2727586e 00007fda27274bb6 00007fda272747d8 00007fda2727e221 00007fda2727e1d9 00007fda2727e169
0709 13:03:19.694409 (+  2868us) raft_consensus.cc:1685] finished
0709 13:03:19.694434 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.694707 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5407,"raft.queue_time_us":95019,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":4599,"spinlock_wait_cycles":5835008,"thread_start_us":108609,"threads_started":1,"wal-append.queue_time_us":145216}
W20260709 13:03:19.700966  1582 debug-util.cc:398] Leaking SignalData structure 0x7b0800397a40 after lost signal to thread 1493
I20260709 13:03:19.706964  9859 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:19.708763  9438 raft_consensus.cc:3055] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:19.710067  9834 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.710601  9740 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.683411  9856 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:19.709372  9438 raft_consensus.cc:740] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:19.721081  9438 consensus_queue.cc:260] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.722987  9438 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:19.703056  9426 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.725062  9426 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:19.717136  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 89258.7us
I20260709 13:03:19.662957  9860 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.735647  1630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3449) took 1299 ms. Trace:
I20260709 13:03:19.737344  1630 rpcz_store.cc:276] 0709 13:03:18.436091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.250243 (+814152us) service_pool.cc:224] Handling call
0709 13:03:19.256249 (+  6006us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.256763 (+   514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.256795 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.257523 (+   728us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.713717 (+456194us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b1800495460. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1
0709 13:03:19.713886 (+   169us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:19.713921 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.713987 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.714022 (+    35us) raft_consensus.cc:1685] finished
0709 13:03:19.714029 (+     7us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.735568 (+ 21539us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111033,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":7184,"spinlock_wait_cycles":15420288,"thread_start_us":123505,"threads_started":1,"wal-append.queue_time_us":187222}
I20260709 13:03:19.741199  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3451) took 1287 ms. Trace:
W20260709 13:03:19.743023  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 67332us
I20260709 13:03:19.714954  9856 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.742292  1635 rpcz_store.cc:276] 0709 13:03:18.454229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.626408 (+1172179us) service_pool.cc:224] Handling call
0709 13:03:19.627171 (+   763us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.635468 (+  8297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.635516 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.671727 (+ 36211us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.739012 (+ 67285us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:19.739061 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.739138 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.740721 (+  1583us) raft_consensus.cc:1685] finished
0709 13:03:19.740746 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.741126 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3824,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":19255,"thread_start_us":21916,"threads_started":1,"wal-append.queue_time_us":51292}
I20260709 13:03:19.716536  9564 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:19.754101  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.754900  1708 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
W20260709 13:03:19.755481  1708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3526) took 1515 ms (client timeout 1828 ms). Trace:
W20260709 13:03:19.755918  1708 rpcz_store.cc:269] 0709 13:03:18.240024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.753638 (+1513614us) service_pool.cc:224] Handling call
0709 13:03:19.755413 (+  1775us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:19.758351  9855 log.cc:927] Time spent T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.468s	user 0.000s	sys 0.002s
W20260709 13:03:19.762445  1582 debug-util.cc:398] Leaking SignalData structure 0x7b0800374c40 after lost signal to thread 1585
I20260709 13:03:19.764797  9825 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:19.761360  2345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59270 (request call id 1627) took 2532 ms (client timeout 1928 ms). Trace:
W20260709 13:03:19.766527  2345 rpcz_store.cc:269] 0709 13:03:17.228792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.228939 (+   147us) service_pool.cc:224] Handling call
0709 13:03:19.761059 (+2532120us) spinlock_profiling.cc:243] Waited 45.1 ms on lock 0x7b5400478954. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:19.761229 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103837952}
W20260709 13:03:19.767439  1601 leader_election.cc:336] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.647s (SENT)
I20260709 13:03:19.751322  9746 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431) with higher term. Message: Invalid argument: T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21.
I20260709 13:03:19.772996  9528 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.749644  9856 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:19.783283  1704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3501) took 3002 ms (client timeout 2222 ms). Trace:
W20260709 13:03:19.783833  1704 rpcz_store.cc:269] 0709 13:03:16.780604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.780876 (+   272us) service_pool.cc:224] Handling call
0709 13:03:19.783009 (+3002133us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54000cb2d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:19.783159 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1285586176}
I20260709 13:03:19.788491  9740 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:19.789137  9740 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.791535  9740 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:19.801860  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:19.802983  1632 raft_consensus.cc:2410] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 13:03:19.815335  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 142165us
I20260709 13:03:19.815169  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.816431  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:19.815527  1675 leader_election.cc:336] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.033s (SENT)
I20260709 13:03:19.819538  1675 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:19.830545  9426 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.835678  9426 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:19.797348  9778 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:19.830266  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 82793.1us
W20260709 13:03:19.839104  1601 leader_election.cc:336] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.674s (SENT)
I20260709 13:03:19.839654  1601 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:19.840749  9778 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.841841  9813 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:19.840942  1601 leader_election.cc:336] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.700s (SENT)
I20260709 13:03:19.842892  1601 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:19.794548  9860 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:19.856914  1675 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.110s (SENT)
I20260709 13:03:19.838788  9853 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:19.860766  2318 connection.cc:256] RPC call timeout handler was delayed by -1.3231352160000256s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1663s for a response
I20260709 13:03:19.863649  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.864557  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:19.865199  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.865550  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:19.866570  2344 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 23: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
W20260709 13:03:19.867954  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 51580.7us
I20260709 13:03:19.866047  9842 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:19.869609  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.851830  9837 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:19.873095  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:19.872365  2350 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:19.855104  9746 raft_consensus.cc:3060] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:19.862329  9754 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:19.879655  9754 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:19.880549  2317 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.882407  9528 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:19.883625  9740 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.884389  1601 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.726s (SENT)
I20260709 13:03:19.884246  9740 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.884909  1601 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:19.886160  9875 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:19.887274  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.887667  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:19.888228  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54748 (request call id 3448) took 1458 ms. Trace:
I20260709 13:03:19.888981  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.889408  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:19.889847  1582 debug-util.cc:398] Leaking SignalData structure 0x7b08003242a0 after lost signal to thread 1660
I20260709 13:03:19.890476  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:19.891372  1635 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 13:03:19.891470  1582 debug-util.cc:398] Leaking SignalData structure 0x7b080035c3c0 after lost signal to thread 1734
W20260709 13:03:19.891836  1582 debug-util.cc:398] Leaking SignalData structure 0x7b0800367de0 after lost signal to thread 1735
I20260709 13:03:19.892439  1525 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:19.892774  9829 consensus_meta.cc:287] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.000s
I20260709 13:03:19.893337  1632 raft_consensus.cc:1275] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:03:19.893376  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:19.896136  1674 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:19.887394  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:19.898571  9847 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:19.898746  9740 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.900189  9840 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:19.900543  9853 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:19.900878  9840 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:19.901103  9853 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:19.902369  9868 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:19.888618  1636 rpcz_store.cc:276] 0709 13:03:18.435711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.581983 (+146272us) service_pool.cc:224] Handling call
0709 13:03:18.589972 (+  7989us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.749199 (+159227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.749248 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.754738 (+  5490us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.887590 (+1132852us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b1800437ec0. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda272824bc 00007fda245d048b 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b 00007fda27279d23 00007fda2727a0d1
0709 13:03:19.887772 (+   182us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:19.887812 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.887897 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.887947 (+    50us) raft_consensus.cc:1685] finished
0709 13:03:19.887959 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.888166 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7666,"raft.queue_time_us":307417,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":9205,"spinlock_wait_cycles":3767168,"thread_start_us":777176,"threads_started":2,"wal-append.queue_time_us":656967}
I20260709 13:03:19.903038  9868 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:19.902667  9853 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.903489  9868 raft_consensus.cc:493] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.903954  9868 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:19.880831  9768 raft_consensus.cc:515] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:19.888625  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.909225  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.918325  9528 consensus_queue.cc:1059] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:19.924787  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:19.924975  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 68038.6us
I20260709 13:03:19.937759  2345 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:19.928817  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:19.940110  1630 raft_consensus.cc:3060] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:03:19.913727  9786 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:03:19.906724  9752 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:19.942235  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.943962  1629 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:19.893241  9829 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:19.948859  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.920116  9754 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:19.955765  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.951439  1709 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22.
I20260709 13:03:19.959093  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:19.960245  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.960645  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:19.961248  2346 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:19.963033  1631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48246 (request call id 1494) took 1753 ms. Trace:
I20260709 13:03:19.964826  1708 raft_consensus.cc:1275] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:03:19.952438  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.952879  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.952371  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:19.967022  2349 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 19.
I20260709 13:03:19.967240  2345 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:19.963609  1631 rpcz_store.cc:276] 0709 13:03:18.210111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.210291 (+   180us) service_pool.cc:224] Handling call
0709 13:03:19.962812 (+1752521us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b540010c7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:19.962953 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2182687616}
I20260709 13:03:19.959805  1703 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 23.
I20260709 13:03:19.946398  9860 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:19.977337  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:20.025933  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 141552us
I20260709 13:03:19.916328  9847 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
W20260709 13:03:20.034405  1582 debug-util.cc:398] Leaking SignalData structure 0x7b08001b6ae0 after lost signal to thread 2377
I20260709 13:03:20.030169  1633 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:20.028991  2353 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
W20260709 13:03:20.034938  1582 debug-util.cc:398] Leaking SignalData structure 0x7b0800390e60 after lost signal to thread 2378
I20260709 13:03:19.921777  9815 consensus_queue.cc:1048] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.039s
W20260709 13:03:20.037496  1582 debug-util.cc:398] Leaking SignalData structure 0x7b0800365cc0 after lost signal to thread 9589
W20260709 13:03:20.043322  1582 debug-util.cc:398] Leaking SignalData structure 0x7b08000445a0 after lost signal to thread 9775
W20260709 13:03:20.043834  1582 debug-util.cc:398] Leaking SignalData structure 0x7b08002382c0 after lost signal to thread 9794
I20260709 13:03:20.038607  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:03:20.038975  1601 leader_election.cc:336] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.745s (SENT)
I20260709 13:03:20.044548  1601 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.060753  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:20.062225  2349 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:20.027478  9404 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.065840  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.030786  9859 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:20.072607  2317 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:20.069898  1601 leader_election.cc:336] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.925s (SENT)
I20260709 13:03:20.070590  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.075018  1636 raft_consensus.cc:2393] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:20.074563  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.076704  1527 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:20.077291  2318 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:20.074018  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 167441us
I20260709 13:03:20.083653  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:20.084442  1703 raft_consensus.cc:3060] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:20.085247  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.086134  1628 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:03:20.056191  9561 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:20.097718  9858 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:20.099035  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:20.099680  1675 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.890s (SENT)
I20260709 13:03:20.099792  1709 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
W20260709 13:03:20.100704  1675 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.500s (SENT)
I20260709 13:03:20.100829  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.102113  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.102119  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:20.088801  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.102380  1709 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:03:20.118424  1710 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 13:03:20.120596  1675 leader_election.cc:336] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.537s (SENT)
I20260709 13:03:20.080924  9740 raft_consensus.cc:2804] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:03:20.129130  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 46512.4us
I20260709 13:03:20.086961  9754 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:20.120554  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 199778us
I20260709 13:03:20.135564  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:03:20.127310  2318 leader_election.cc:336] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.139623  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.141319  2346 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 24.
W20260709 13:03:20.143494  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 73645.1us
I20260709 13:03:20.147921  9564 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:20.148676  9564 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.154294  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:20.162153  9829 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:20.079138  9859 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:20.170886  1703 raft_consensus.cc:2468] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 23.
W20260709 13:03:20.172108  1601 leader_election.cc:336] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 1.944s (SENT)
I20260709 13:03:20.149151  9564 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:20.190752  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 59593.9us
W20260709 13:03:20.192941  1705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 1340) took 2262 ms (client timeout 1663 ms). Trace:
W20260709 13:03:20.193638  1705 rpcz_store.cc:269] 0709 13:03:17.930950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.931907 (+   957us) service_pool.cc:224] Handling call
0709 13:03:20.192612 (+2260705us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54000cbcd4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:20.192838 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381559680}
I20260709 13:03:20.195681  1525 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.163261  9740 raft_consensus.cc:697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:03:20.200196  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 71518.4us
W20260709 13:03:20.204278   431 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 1586)
W20260709 13:03:20.219451  9883 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:20.201680  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:20.226601  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:20.230520  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 80289.1us
I20260709 13:03:20.197117  9815 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:20.241365  1601 leader_election.cc:336] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.024s (SENT)
I20260709 13:03:20.242322  1601 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:20.251797  1601 leader_election.cc:336] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.033s (SENT)
W20260709 13:03:20.249886  1675 leader_election.cc:336] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.248636  9541 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:20.262077  9740 consensus_queue.cc:237] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.268550  1633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54746 (request call id 3811) took 1008 ms. Trace:
I20260709 13:03:20.268951  1630 raft_consensus.cc:2468] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:03:20.262847  9541 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:20.274528  9859 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:20.276796  9854 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:20.278554  9851 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.249712  9885 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:03:20.280963  9833 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:20.285508  9722 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:20.285645  1675 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.286350  9722 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 24 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:03:20.286976  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43663.7us
W20260709 13:03:20.288441  1675 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.288888  9822 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:20.289143  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:20.291991  1674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.287395  9722 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:20.306434  1675 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.306952  1675 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.307837  9822 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:20.292659  9868 raft_consensus.cc:515] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.293586  9746 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:20.297871  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:20.297142  9860 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:20.298982  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 47196.5us
I20260709 13:03:20.299741  9744 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.301329  9768 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.253190  1676 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.302543  9495 leader_election.cc:290] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.302683  1633 rpcz_store.cc:276] 0709 13:03:19.260686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.453746 (+193060us) service_pool.cc:224] Handling call
0709 13:03:20.268283 (+814537us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b54001397d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:20.268473 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1292504832}
I20260709 13:03:20.280854  9856 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:20.310304  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 120498us
I20260709 13:03:20.285363  9893 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:20.309795  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.316855  2349 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
W20260709 13:03:20.325690  1601 leader_election.cc:336] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.062s (SENT)
W20260709 13:03:20.350302  1601 leader_election.cc:336] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.184s (SENT)
I20260709 13:03:20.350752  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.350842  9633 raft_consensus.cc:3055] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:20.351436  9633 raft_consensus.cc:740] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:03:20.352144  1601 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.222s (SENT)
I20260709 13:03:20.352478  9633 consensus_queue.cc:260] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:20.348850  9856 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:20.344658  9822 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:20.339676  1629 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:20.354321  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.354681  9776 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:20.355013  9866 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:20.356711  9776 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:20.358664  9868 leader_election.cc:290] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.359516  1527 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:20.361864  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:20.362676  1707 raft_consensus.cc:1275] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:03:20.363102  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.363936  9851 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:20.364444  9851 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:20.365337  9851 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.365828  9754 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:20.362691  1706 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 23: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:03:20.366904  1601 leader_election.cc:336] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.657s (SENT)
I20260709 13:03:20.348297  9786 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:20.367345  1601 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.367777  9786 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:03:20.361900  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.369817  9860 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:20.370800  9760 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.371592  2317 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:20.344585  1704 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:20.378329  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:20.375578  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:20.379457  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 126318us
I20260709 13:03:20.380530  1676 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.374784  1701 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:20.346992  9896 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:03:20.384001  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:20.375525  1634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54746 (request call id 3796) took 2420 ms. Trace:
I20260709 13:03:20.376987  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:20.385020  2347 raft_consensus.cc:1240] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Rejecting Update request from peer ba0548b7d2a94e8598475a46c279fe0e for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:20.385128  1702 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:20.381786  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:20.386065  1703 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:20.381951  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.384713  1706 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:20.309146  9831 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:20.388254  2351 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 23: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:20.384902  1634 rpcz_store.cc:276] 0709 13:03:17.955487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.585081 (+1629594us) service_pool.cc:224] Handling call
0709 13:03:19.585434 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.735004 (+149570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.735056 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.750654 (+ 15598us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.855206 (+104552us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:19.855272 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.375081 (+519809us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b540010e0d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276dae01 00007fda276d983e 00007fda2c69e419 00007fda25c15bd4 00007fda25c15b62 00007fda25c15a91 00007fda25c15a0c 00007fda25c14cc2 00007fda25a1548c 00007fda25a148c6
0709 13:03:20.375137 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.375196 (+    59us) raft_consensus.cc:1685] finished
0709 13:03:20.375218 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.375452 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":117,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":8858,"spinlock_wait_cycles":342756736,"thread_start_us":89070,"threads_started":1,"wal-append.queue_time_us":100268}
W20260709 13:03:20.324397  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 128897us
I20260709 13:03:20.387382  2317 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:20.398872  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.400113  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.404213  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.409155  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.412070  1709 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:20.410560  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.420249  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.421227  1705 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:20.422173  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:20.395161  9893 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.424508  9893 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.405128  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:20.436172  1675 leader_election.cc:336] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.427670  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.389921  9754 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:20.395371  9868 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:20.444515  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.445962  1708 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:20.448150  1703 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 24.
I20260709 13:03:20.454861  1701 raft_consensus.cc:2393] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 24: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
W20260709 13:03:20.453145  1675 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.410934  9653 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.411221  9831 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.411404  9815 raft_consensus.cc:2804] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:20.455413  1675 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.459481  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:20.427048  9633 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:20.464555  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 337108us
W20260709 13:03:20.465610  2318 leader_election.cc:336] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.428387  9737 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.472867  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.473826  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.472514  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:20.450677  9754 raft_consensus.cc:493] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.467373  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.481979  1633 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:20.483472  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.484329  1636 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:03:20.490532  1447 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.313s	user 0.000s	sys 0.000s
I20260709 13:03:20.492318  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:20.494611  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 127698us
W20260709 13:03:20.496829  2318 leader_election.cc:336] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.468014  9653 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:20.498476  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:20.504540  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.501343  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.486265  9754 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:20.507222  1675 leader_election.cc:336] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.512370  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.520246  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.521389  1630 raft_consensus.cc:2393] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 24: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:20.530354  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.531091  2351 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 24.
I20260709 13:03:20.518563  1675 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.531936  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.532677  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:20.532797  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.519526  1636 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:03:20.535286  1675 leader_election.cc:336] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.535811  1675 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.536579  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:20.536496  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.538092  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:20.540468  2345 raft_consensus.cc:2368] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 23. Current term is 24.
I20260709 13:03:20.533321  2346 raft_consensus.cc:2376] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21. Re-sending same reply.
I20260709 13:03:20.494949  9815 raft_consensus.cc:697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:20.473546  9831 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:20.461450  9866 consensus_queue.cc:1059] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
W20260709 13:03:20.543918  1675 leader_election.cc:336] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.538790  2353 raft_consensus.cc:2376] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22. Re-sending same reply.
I20260709 13:03:20.546983  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.548153  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:20.506062  9893 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.549847  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:20.551818  1675 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.549311  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:20.482182  9737 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.551576  2350 raft_consensus.cc:2376] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 24. Re-sending same reply.
I20260709 13:03:20.554438  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:20.554545  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:20.552588  1675 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:20.555255  1636 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:20.556015  1706 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:03:20.542389  9822 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:20.559345  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.559986  1704 raft_consensus.cc:2393] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 21: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:20.560916  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:20.562271  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:20.572053  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.573871  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.574179  2351 raft_consensus.cc:1240] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:20.575153  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.577751  9815 consensus_queue.cc:237] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:20.579488  9889 log.cc:927] Time spent T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.377s	user 0.000s	sys 0.006s
I20260709 13:03:20.584759  9825 raft_consensus.cc:3055] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:20.585352  9825 raft_consensus.cc:740] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:20.586582  9825 consensus_queue.cc:260] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.587255  9541 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:20.588279  9825 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:20.590180  9740 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:20.558766  9851 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:20.591760  9851 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.592108  9851 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:20.573338  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:20.620185  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac931d9355284a168be64d7debd009ae" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:20.620325  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:20.619134  1636 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
W20260709 13:03:20.568116  9897 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:03:20.642724  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:20.648524  2353 raft_consensus.cc:2393] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 23: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:20.677130  2318 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.634797  9842 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:03:20.683063  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 187400us
W20260709 13:03:20.685253  1601 leader_election.cc:336] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.815s (SENT)
I20260709 13:03:20.649838  9741 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:03:20.695935  1601 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Timed out: RequestConsensusVote RPC to 127.0.107.193:33149 timed out after 2.901s (SENT)
I20260709 13:03:20.690282  9893 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.701294  1636 raft_consensus.cc:1240] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:20.633893  9786 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:20.699364  9744 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.706952  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb0b8480ae5646bea5cd44d499b6bcaa" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:20.707863  1629 raft_consensus.cc:2393] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:20.676220  9404 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:03:20.697755  1526 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.704584  1601 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.705571  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.701362  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:20.679502  9743 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:20.681859  9840 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.647754  9778 consensus_queue.cc:1048] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:03:20.719892  1627 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:20.718762  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.695729  9741 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:20.700070  9893 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.727620  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.722522  2318 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.730185  1631 raft_consensus.cc:1240] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 28 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 27. Current term is 28. Ops: []
I20260709 13:03:20.735441  1525 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.708642  9541 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:20.737160  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.741616  1628 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:20.743417  1636 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:03:20.753224  1525 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:20.755113  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 54791us
I20260709 13:03:20.719794  9743 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.762519  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:20.759909  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:20.724256  9859 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:20.759682  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:20.763919  1629 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 21.
I20260709 13:03:20.766613  1701 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 25. Current term is 26. Ops: []
I20260709 13:03:20.763402  1706 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 24.
I20260709 13:03:20.731009  9722 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:20.733494  9741 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:20.729197  9234 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:20.726244  9831 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:20.747666  9847 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:20.732929  9884 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.738451  9875 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:20.783903  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 87955.3us
I20260709 13:03:20.717900  9744 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:20.789149  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 92454.4us
I20260709 13:03:20.767040  9743 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:20.795627  1601 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:20.772092  9776 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:20.770313  9746 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:20.800626  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:20.776180  9892 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.813975  1702 raft_consensus.cc:2468] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 24.
I20260709 13:03:20.807574  1601 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:20.821991  1526 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:20.795113  9564 consensus_meta.cc:287] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.587s	user 0.012s	sys 0.002s
W20260709 13:03:20.819896  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 66780.8us
I20260709 13:03:20.730993  9840 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:20.826040  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:20.829836  9722 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:20.830641  9892 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.831104  9786 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:20.832376  9907 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:03:20.839517  9744 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.789659  9851 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.805792  9653 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:20.876282  1676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.819558  9776 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.859692  1526 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:20.894999  9913 log.cc:927] Time spent T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
W20260709 13:03:20.904356  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44785.4us
W20260709 13:03:20.881943  9909 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:03:20.908372  1526 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.845695  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.845314  9860 raft_consensus.cc:2804] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:03:20.911152  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 224855us
W20260709 13:03:20.913803  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 68221.1us
I20260709 13:03:20.914743  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.870517  9778 raft_consensus.cc:2804] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:20.916422  1634 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:03:20.853464  9866 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:20.917325  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.798328  9884 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.864543  9840 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:20.843894  9892 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.870386  9919 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:03:20.910602  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 151549us
I20260709 13:03:20.904779  9776 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:20.854451  9893 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.869997  9754 raft_consensus.cc:515] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.841795  9564 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.912962  9746 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:20.921691  9847 raft_consensus.cc:3060] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:20.873178  9234 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:03:20.852897  9722 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:20.919934  9833 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:20.920118  9866 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:20.929276  9912 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.301s	user 0.005s	sys 0.000s
I20260709 13:03:20.824499  9859 raft_consensus.cc:2749] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:20.945398  1526 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:20.921929  9860 raft_consensus.cc:697] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 24 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:20.962673  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:20.927735  9778 raft_consensus.cc:697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:20.966257  9875 raft_consensus.cc:3060] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:20.931777  9840 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:21.009862  9851 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.997514  9653 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:03:21.011133  9754 leader_election.cc:290] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:21.014497  9884 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:20.963136  9892 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:03:21.022150  9778 consensus_queue.cc:237] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:20.975142  9866 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:21.027662  1707 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:21.029836  9922 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:03:21.031409  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 89291us
I20260709 13:03:21.037600  9722 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:20.965160  9744 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:03:21.045725  9564 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:21.037046  9860 consensus_queue.cc:237] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:21.028736  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 263527us
I20260709 13:03:21.054934  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.056542  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.016587  9741 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:21.059947  1675 leader_election.cc:336] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.074321  2351 raft_consensus.cc:1240] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 19. Current term is 20. Ops: []
W20260709 13:03:21.061015  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 129885us
I20260709 13:03:21.062040  1676 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.065311  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:21.066866  2344 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:03:21.081791  1676 leader_election.cc:400] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 23. Current term is 24.
I20260709 13:03:21.082181  1676 leader_election.cc:403] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:03:21.082484  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:21.083213  1631 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:21.086294  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:21.087325  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:21.088590  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:21.088611  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:21.091928  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:21.093410  2318 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.091868  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:21.094053  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.098253  2353 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:21.101788  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.110908  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:21.123692  1676 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:21.107336  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.078383  9528 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:21.076932  9633 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:21.086650  9754 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.127226  2350 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:03:21.127722  1708 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:03:21.119697  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:21.125828  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc25e0c8bb834899a98a40efe1e66bba" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:03:21.128880  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 299886us
I20260709 13:03:21.129210  1703 raft_consensus.cc:3060] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:21.130556  1601 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:21.133584  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:21.124676  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.134171  2344 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:03:21.134585  2318 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:21.135990  2318 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:21.137185  2318 leader_election.cc:336] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:21.125774  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 175185us
I20260709 13:03:21.118383  9918 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:21.147058  9743 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.141059  2318 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.160857  1631 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
W20260709 13:03:21.164292  2318 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.164178  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:21.116652  9426 consensus_queue.cc:1059] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.034s
W20260709 13:03:21.213003   431 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 1586)
I20260709 13:03:21.216413  1705 raft_consensus.cc:1275] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:21.163157  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:21.122527  9404 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:03:21.217547  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 53415.4us
W20260709 13:03:21.165424  2318 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.149152  9868 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:21.220281  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:21.221549  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 56107.3us
W20260709 13:03:21.226006  2318 leader_election.cc:336] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.226511  2318 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:21.228890  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 65837.5us
I20260709 13:03:21.229833  1527 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:21.162789  9722 consensus_queue.cc:1059] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 13:03:21.235015  9922 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:21.237181  2318 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.237975  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:21.241354  2318 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.242049  2318 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.216534  9860 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:21.250299  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.149346  9859 consensus_queue.cc:1048] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 13:03:21.252905  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:21.219302  9633 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:21.252894  1703 raft_consensus.cc:2468] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:21.153822  9896 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.084s
I20260709 13:03:21.260365  1676 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:21.263970  2344 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:21.165997  9760 consensus_queue.cc:1059] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:03:21.270038  2352 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:21.152529  9754 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.239142  9853 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:21.259416  9528 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:21.249086  9868 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:21.254711  9834 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:21.269979  9653 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:21.265407  9404 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:21.257993  9744 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:21.271848  9842 raft_consensus.cc:2749] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:03:21.283746  9825 consensus_meta.cc:287] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: Time spent flushing consensus metadata: real 0.687s	user 0.010s	sys 0.007s
W20260709 13:03:21.314248  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.292850  9866 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:21.316381  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 93598us
I20260709 13:03:21.317934  1601 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:21.322984  9923 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:21.310151  9875 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:21.334815  9943 raft_consensus.cc:2804] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:21.274395  9776 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:21.336197  9743 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:21.314733  9528 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:21.322122  9931 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:21.326190  9404 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:21.359356  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 99011.7us
I20260709 13:03:21.342196  2353 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 25.
I20260709 13:03:21.312366  9921 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 23. Current term is 24.
W20260709 13:03:21.360533  9928 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:03:21.323397  9927 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:21.364485  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:21.374843  9564 raft_consensus.cc:3055] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:21.375432  9564 raft_consensus.cc:740] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:21.376438  9564 consensus_queue.cc:260] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.378291  9528 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.379933  9564 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:21.318104  1447 maintenance_manager.cc:643] P 8f622dabf84e4c0595473d2c01f108d0: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 3.219s	user 0.125s	sys 0.069s Metrics: {"cfile_cache_miss":6,"cfile_cache_miss_bytes":25519,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":443797,"dirs.run_cpu_time_us":528,"dirs.run_wall_time_us":1176,"lbm_read_time_us":121590,"lbm_reads_1-10_ms":3,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":17,"lbm_write_time_us":1692,"lbm_writes_lt_1ms":10,"peak_mem_usage":263430,"reinsert_count":0,"thread_start_us":287188,"threads_started":2,"update_count":176}
W20260709 13:03:21.359539  1601 leader_election.cc:336] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.392668  1676 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:21.395745  9868 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.390900  1601 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.398183  9860 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:21.398571  9746 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:21.413040  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:21.413066  9840 raft_consensus.cc:3055] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:03:21.414649  9840 raft_consensus.cc:740] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:21.415498  1705 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:21.415562  9840 consensus_queue.cc:260] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:21.417189  9840 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:03:21.400501  9776 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:21.422820  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:21.420506  2318 leader_election.cc:304] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:03:21.425232  2318 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:21.399682  1601 leader_election.cc:336] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.402966  9737 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:21.407347  9945 consensus_queue.cc:1059] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:21.408190  9822 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:21.361745  9943 raft_consensus.cc:697] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 24 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:21.435009  1526 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:21.399345  9866 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:21.427183  2318 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.428215  1702 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:21.426098  9495 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:21.434027  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:21.436363  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:21.427719  9743 raft_consensus.cc:2804] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:21.441948  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:21.408867  9754 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:21.443951  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44281us
I20260709 13:03:21.444475  1676 leader_election.cc:304] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:21.442960  1709 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 23.
I20260709 13:03:21.445765  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:21.447573  1705 raft_consensus.cc:2393] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 19: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:03:21.450341  1601 leader_election.cc:336] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.451923  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:21.450888  1601 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.453743  9866 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:21.453902  9822 raft_consensus.cc:2749] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:21.451839  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:21.454995  1702 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:21.455227  1676 leader_election.cc:304] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:21.428251  9923 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 13:03:21.443877  9746 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:21.431035  9234 consensus_queue.cc:1048] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:03:21.456713  9822 raft_consensus.cc:2749] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:21.483899  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.495157  1710 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:21.473515  9847 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:21.502470  1702 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
I20260709 13:03:21.518182  9943 consensus_queue.cc:237] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:21.460443  9743 raft_consensus.cc:697] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:21.523033  9743 consensus_queue.cc:237] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.490703  9853 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.456950  9875 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:03:21.527506  9847 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.533901  9495 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.545338  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.545519  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:21.547576  2352 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:21.556448  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:21.558290  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 496117us
I20260709 13:03:21.538249  9853 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:21.559244  1674 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:21.549955  9495 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:21.592123  9495 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:21.593503  9916 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:21.593791  1704 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:03:21.551144  9923 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:21.595408  1704 raft_consensus.cc:1240] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 25. Current term is 26. Ops: []
I20260709 13:03:21.599207  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:21.599946  1632 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:21.602248  9866 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:21.602864  9866 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.604130  1627 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:03:21.549320  9778 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 13:03:21.606477  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:21.607338  1629 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:03:21.610147  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:21.613112  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:21.618418  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 59244.5us
I20260709 13:03:21.605284  9866 leader_election.cc:290] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:21.628885  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:21.629537  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:21.633081  2349 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:21.548156  9847 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:21.651788  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:21.654825  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:21.655478  1634 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:21.613030  9868 raft_consensus.cc:493] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:21.625000  9833 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:21.646979  9404 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:21.680243  1632 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:21.681872  1525 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:21.659121  9857 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.653401  9875 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 13:03:21.701557  2349 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 23.
I20260709 13:03:21.675282  9404 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
I20260709 13:03:21.659502  9868 raft_consensus.cc:515] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.701994  9776 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:21.726563  9776 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:21.727679  9776 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.730939  9760 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:03:21.736195  9898 raft_consensus.cc:3055] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:03:21.736763  9898 raft_consensus.cc:740] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:21.737824  1676 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:21.738780  9541 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:21.739248  9541 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.739615  9541 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:21.741868  1707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46210 (request call id 3532) took 1338 ms. Trace:
I20260709 13:03:21.742312  1707 rpcz_store.cc:276] 0709 13:03:20.403361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.403538 (+   177us) service_pool.cc:224] Handling call
0709 13:03:21.741641 (+1338103us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b540014d7d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:21.741769 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244283648}
I20260709 13:03:21.710268  9404 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:21.743611  1527 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:21.737610  9898 consensus_queue.cc:260] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:21.751161  9564 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:21.751989  9564 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 25 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:21.754225  1634 raft_consensus.cc:1275] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:21.754135  9833 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:21.753278  1708 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:03:21.756713  9741 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:21.763149  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:21.771291  1627 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 23.
I20260709 13:03:21.753189  9564 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.789399  9740 consensus_queue.cc:1048] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:21.791689  2317 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:21.798591  2318 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:21.799772  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:21.706233  9857 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:21.804467  1629 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
W20260709 13:03:21.811089  1675 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.809852  9740 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:21.821318  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
W20260709 13:03:21.824534  1675 leader_election.cc:336] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:21.825137  1675 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.810235  9743 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.828110  9743 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.831034  9743 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:21.819664  9740 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.841466  9740 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:21.794219  9901 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 13:03:21.852006  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:21.856148  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:21.857268  1708 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 24.
I20260709 13:03:21.861635  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:21.844319  9896 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.888921  9896 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.891311  9896 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:21.861285  9815 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:21.893077  9857 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:21.894523  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.895743  9868 leader_election.cc:290] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:21.897223  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:21.898164  2352 raft_consensus.cc:2393] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 22: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:03:21.904986  9961 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:03:21.910779  9737 raft_consensus.cc:3055] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:21.911325  9737 raft_consensus.cc:740] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:21.912288  9737 consensus_queue.cc:260] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:21.865407  9916 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:21.858840  9744 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:21.929972  9744 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:21.979079  1526 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:21.982071  1526 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:21.994719  1526 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:21.933413  9744 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:21.960911  9875 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:22.019438  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 157879us
I20260709 13:03:22.021142  9875 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.023849  9875 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:21.996618  9885 raft_consensus.cc:3060] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:22.026831  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:22.017460  9760 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.011s
I20260709 13:03:22.030165  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:22.012547  9884 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:22.016430  9564 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:22.016080  9743 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:22.060727  9740 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.013001  9896 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.057s
W20260709 13:03:22.085532  2318 leader_election.cc:336] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:22.057739  9564 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
W20260709 13:03:22.097010  1675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.108192  9404 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.110863  9564 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:22.117826  9853 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:22.118613  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:22.124795  9746 raft_consensus.cc:3055] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:22.151970  1703 raft_consensus.cc:1240] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Rejecting Update request from peer 95fc6a4cf5874b7eb91280dffcd3b3fb for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:22.140501  9901 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.146513  9875 raft_consensus.cc:493] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.147744  9746 raft_consensus.cc:740] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:22.141414  1627 raft_consensus.cc:1275] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
W20260709 13:03:22.178614  9965 log.cc:927] Time spent T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:03:22.178870  9404 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:22.168035  9901 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:22.141155  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 90494us
I20260709 13:03:22.138279  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:22.189324  9901 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:22.217216   431 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 1586)
W20260709 13:03:22.218294  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 77079.9us
I20260709 13:03:22.194907  9859 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:22.222411  2350 raft_consensus.cc:1240] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:22.176755  9875 raft_consensus.cc:515] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:22.169076  9541 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.235275  1708 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:03:22.207834  9743 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:03:22.239380  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:22.238476  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:22.241186  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:22.243686  1675 leader_election.cc:336] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:22.244614  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:22.245776  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:22.246454  1708 raft_consensus.cc:2468] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
W20260709 13:03:22.261317  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 728112us
I20260709 13:03:22.262308  1526 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:22.263567  1526 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:22.264658  1526 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:22.243659  9743 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:22.273737  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:22.277796  1702 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:03:22.280953  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.288085  1524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.272866  9840 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:22.270534  9853 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:22.232738  9859 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:22.321353  1601 leader_election.cc:336] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:22.322268  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:22.329759  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:22.332994  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 68368us
I20260709 13:03:22.276703  9949 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:22.269166  9776 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:03:22.289891  9963 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:03:22.350097  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 54053.8us
W20260709 13:03:22.355777  1601 leader_election.cc:336] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:22.329830  9885 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:22.328464  9884 raft_consensus.cc:2804] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:22.359355  1702 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 28.
I20260709 13:03:22.358908  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:22.363900  1601 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:22.272469  9746 consensus_queue.cc:260] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.369956  9884 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.351666  9949 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:22.408527  9987 log.cc:927] Time spent T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:03:22.363617  9963 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.392196  9933 consensus_queue.cc:1059] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:03:22.416082  9963 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:22.416565  9768 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:22.404337 10001 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:22.418905  9859 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:22.421046  1708 raft_consensus.cc:1275] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:03:22.399389  9884 raft_consensus.cc:3060] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:22.422637  1708 raft_consensus.cc:1275] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:03:22.426807  9949 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:22.439019  9541 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:22.441568 10004 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.442229 10004 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.444970  2349 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:03:22.446890 10004 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:22.448964  9776 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:22.451097  9847 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:22.454173  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:22.455292  2349 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 24: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:22.463423  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:22.464284  2351 raft_consensus.cc:2393] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 21: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:22.465075  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:22.469660  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:22.470758  1629 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:22.470897  1628 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:22.480774  1629 raft_consensus.cc:1275] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:03:22.492532  9743 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:22.498189  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 142352us
W20260709 13:03:22.500164  1601 leader_election.cc:336] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:22.464303  9831 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.503206  9898 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:22.505553  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:22.509505  1525 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:22.536841  1601 leader_election.cc:336] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:22.537911  1601 leader_election.cc:336] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:22.532289  2317 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:22.538875  1601 leader_election.cc:336] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:22.539330  1601 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:22.507170  9831 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.475080  9930 consensus_queue.cc:1048] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 13:03:22.526819  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:22.528575  9997 log.cc:927] Time spent T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.152s	user 0.002s	sys 0.000s
I20260709 13:03:22.531710  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:22.567305  1627 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
I20260709 13:03:22.545940  9949 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:22.547087  9945 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.547894  9746 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:22.575114  9945 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:22.577907  9840 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:22.579434  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:22.580698  9884 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:22.551267  9744 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:22.556667  1525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.569602 10008 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:22.571362  9743 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:22.584604  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:22.666625  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 82031.1us
I20260709 13:03:22.549427  9815 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:22.595433  9926 log.cc:927] Time spent T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 1.375s	user 0.004s	sys 0.003s
I20260709 13:03:22.598099  9875 leader_election.cc:290] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:22.606952  9737 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:22.623607  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5838267321348eb83df088988da6449" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:22.625918  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
W20260709 13:03:22.570223  1601 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:22.678833  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 108588us
W20260709 13:03:22.679816  1601 leader_election.cc:336] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:22.680253  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:22.628815  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:22.681106  1630 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:22.630626  9740 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:22.635361  1673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.692152  1709 raft_consensus.cc:3055] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:22.692548  1709 raft_consensus.cc:740] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:22.640270  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:22.641438  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:22.657938  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 87417.6us
I20260709 13:03:22.577402  9945 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:22.696588  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:22.699410  1601 leader_election.cc:336] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:22.702476  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:22.703729  1601 leader_election.cc:336] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:22.704206  1601 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:22.688714  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:22.711424  9859 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:22.668476  9815 raft_consensus.cc:493] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.717808  9815 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:22.725890  9840 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.726254  2353 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:22.726502  9840 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:22.728552  9840 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:22.709896  9927 consensus_queue.cc:1059] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:03:22.729347  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:22.729822  9922 raft_consensus.cc:3055] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:22.730198  9922 raft_consensus.cc:740] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:22.731066  9922 consensus_queue.cc:260] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.663460  9923 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:03:22.732244  9922 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:22.733757  9744 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:22.734045  9884 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:22.736243  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:22.739378  1630 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 24.
I20260709 13:03:22.740624  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10315b15aaee40e0a1110e5ce8cf0abf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:22.741919  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:22.742170  1629 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:22.742486  1705 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
W20260709 13:03:22.744437  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44311.6us
I20260709 13:03:22.693246  1709 consensus_queue.cc:260] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.749315  1709 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:22.751475  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:22.752352  1710 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 24: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:22.753235  2350 raft_consensus.cc:1240] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:22.753408  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:22.759531  2353 raft_consensus.cc:1240] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:22.759634  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:22.760665  2349 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 24: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:22.762495  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:22.766587  2350 raft_consensus.cc:2393] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 20: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:22.771411  1706 raft_consensus.cc:1240] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:22.772747  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:22.775580  2352 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:22.775939  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:22.707016  9949 raft_consensus.cc:2749] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:22.707504 10008 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:22.751344  9933 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.794179  1601 leader_election.cc:304] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:22.796692  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
W20260709 13:03:22.799536  1675 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:22.801618  1675 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:22.802626  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:22.808101  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:22.786550  9840 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.812703  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:22.817646  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:22.818401  1705 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 24: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:22.743875  9991 consensus_queue.cc:1048] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 13:03:22.820823  1629 raft_consensus.cc:1240] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Rejecting Update request from peer a2ba7fc60f9641cdac25f1e016ef91d8 for earlier term 22. Current term is 23. Ops: []
W20260709 13:03:22.821332  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 46386.2us
I20260709 13:03:22.823073  1702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:22.789237  9984 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.814599  9847 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:22.828399  1705 raft_consensus.cc:1275] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:22.822739  1601 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:22.810328  9859 raft_consensus.cc:493] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.781358  9885 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 13:03:22.809499  9840 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.791941  9945 consensus_queue.cc:1059] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:03:22.834091  9856 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:22.839674 10008 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:03:22.878688  1702 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:03:22.820937  9884 consensus_queue.cc:1059] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:03:22.854133  9831 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:22.860297  9923 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.815258  9933 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.893606  1674 leader_election.cc:304] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:22.891052  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 126283us
I20260709 13:03:22.899866  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:22.900769  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:22.837297  9984 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:22.853191  9859 raft_consensus.cc:515] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:22.901980  2318 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:22.890976 10008 raft_consensus.cc:697] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 28 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:22.913973  2318 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:22.883343  9854 consensus_queue.cc:1048] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
W20260709 13:03:22.904610  9980 log.cc:927] Time spent T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.748s	user 0.004s	sys 0.000s
I20260709 13:03:22.916826  9917 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:22.920176  1525 leader_election.cc:304] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:22.922127  9840 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:22.905834  9495 raft_consensus.cc:2749] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:03:22.933027  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 94191.2us
I20260709 13:03:22.898298  9923 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.894799 10003 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:03:22.935913  1601 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:22.949457  1676 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:22.953447  1676 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:22.928881  9743 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:22.949314  9885 raft_consensus.cc:2749] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:22.939002  9859 leader_election.cc:290] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:22.969913 10013 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:03:22.971627 10012 log.cc:927] Time spent T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:03:22.962329  9822 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:22.974537  1709 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:03:22.976696  9815 raft_consensus.cc:515] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.959940  9495 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:22.979830  1710 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:03:22.983572  9933 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:22.949069  9856 consensus_queue.cc:1048] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:03:22.965793  9991 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.985278  9991 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:22.942513 10008 consensus_queue.cc:237] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:22.987468  9991 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:22.963943  9945 consensus_queue.cc:1059] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:03:22.993508  9898 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:03:22.994282  9945 raft_consensus.cc:3055] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:22.994627  9945 raft_consensus.cc:740] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:22.995026  9815 leader_election.cc:290] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:22.995821  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 188222us
I20260709 13:03:22.995496  9945 consensus_queue.cc:260] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:22.996474  1675 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 126 similar messages]
I20260709 13:03:22.996884  9945 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:22.941110  9923 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:23.002177  9760 raft_consensus.cc:3055] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:23.002646  9760 raft_consensus.cc:740] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:23.003477  9760 consensus_queue.cc:260] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.004467  9760 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:23.009382  9847 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:23.013648  9983 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.014247  9983 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.016275  9983 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:23.013813  1526 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.965927  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 126848us
I20260709 13:03:23.020816  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:23.019726  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 83855.2us
I20260709 13:03:23.024240  9859 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:23.027240  1601 leader_election.cc:336] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.027725  2352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:23.045864  1601 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:23.027325  1636 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:03:23.029867  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.028003  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:23.047654  1631 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:23.032332  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70981cbf9c1143309f0f78706fe5931d" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.050992  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.051259  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.051725  1629 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:23.052382  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.032820  2351 raft_consensus.cc:2468] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:23.054288  1706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.055269  9923 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.055917  9923 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.058506  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:23.059602  9916 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:23.060060  9916 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.060492  9916 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:23.060163  9884 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:03:23.039410  1675 leader_election.cc:336] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.062973  1675 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:23.077450 10020 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:23.079425  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52207.6us
W20260709 13:03:23.080284  1601 leader_election.cc:336] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.080780  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:23.130378  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 50112us
W20260709 13:03:23.131105  1601 leader_election.cc:336] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.131578  1601 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:23.061784  9884 consensus_queue.cc:1243] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer a2ba7fc60f9641cdac25f1e016ef91d8 log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:03:23.133960  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:23.136821  1634 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:23.142359  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.143352  1632 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:03:23.149222  9916 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.153668  9868 raft_consensus.cc:3055] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:23.154150  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:23.154330  9868 raft_consensus.cc:740] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:23.154942  1632 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:23.155293  9931 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.156425  9931 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.158595  9931 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:23.162881  9916 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:23.173878  1632 raft_consensus.cc:2468] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 24.
I20260709 13:03:23.176424  1676 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:23.179021  9495 raft_consensus.cc:2804] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:23.179617  9495 raft_consensus.cc:697] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:23.180646  9495 consensus_queue.cc:237] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:23.181828 10017 log.cc:927] Time spent T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.270s	user 0.000s	sys 0.002s
I20260709 13:03:23.135387  9943 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:23.197567  9933 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.198093  9933 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.200446  9933 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:23.201409  9923 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:23.202992  2317 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:23.205821  9833 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.206038  9984 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:23.206615  9919 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:23.206425  9833 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.207077  9919 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.207540  9919 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:23.208317 10001 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:23.210176  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 79061.2us
I20260709 13:03:23.210608  9833 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:23.210851  1601 leader_election.cc:336] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.205835  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:23.211606  1601 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:23.214030  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.214617  1710 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 26.
I20260709 13:03:23.210981  2351 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:23.217541  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:23.217846   431 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 1586)
W20260709 13:03:23.218394  1601 leader_election.cc:336] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.187175  1674 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:23.218878  1601 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:23.219899  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:23.229050  1601 leader_election.cc:336] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:23.230559  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:23.224635  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:23.234660  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.223127  1634 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:03:23.235841  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:23.236562  2351 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 23: Already voted for candidate ba0548b7d2a94e8598475a46c279fe0e in this term.
I20260709 13:03:23.240193  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:23.240882  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.130978  9922 consensus_queue.cc:1048] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.096s
W20260709 13:03:23.234201  1601 leader_election.cc:336] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:23.248050  1601 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.248565  1601 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:23.250226  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:23.251711  1601 leader_election.cc:304] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:23.262370  1703 pending_rounds.cc:85] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 15
I20260709 13:03:23.262735  1703 pending_rounds.cc:107] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 13:03:23.222987  9884 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.227635  9923 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.232823  9934 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:23.270882  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.272747  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.273051  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.263039  1703 raft_consensus.cc:2889] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:23.222306 10019 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:23.233059  9984 raft_consensus.cc:2804] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:23.227833  9949 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:23.232687  9768 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:23.239358  9840 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:03:23.234196  9885 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:23.265833  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:23.263002 10021 log.cc:927] Time spent T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:03:23.281628  1455 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 191) took 37167 ms (client timeout 15000 ms). Trace:
I20260709 13:03:23.286228  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0835fee76231431da297ec1cf367640d" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:23.298390  1708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.248194  9833 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:23.261898  9856 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:23.298738  1707 raft_consensus.cc:3060] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:23.299678  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:23.247829  9898 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:03:23.155550  9868 consensus_queue.cc:260] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.276616  9884 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.290550 10006 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 95fc6a4cf5874b7eb91280dffcd3b3fb)
W20260709 13:03:23.326437  2318 leader_election.cc:336] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.363214 10006 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.293839  9984 raft_consensus.cc:697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:23.279490  9923 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.308617  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:23.370285  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:23.368347  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.369021  2353 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
W20260709 13:03:23.375481  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 49038.3us
I20260709 13:03:23.371706  1701 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
W20260709 13:03:23.380679  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 72131.7us
I20260709 13:03:23.306277  2317 leader_election.cc:304] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:23.383893  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 77706.9us
I20260709 13:03:23.388135  9884 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:23.385387  2318 leader_election.cc:336] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:23.297545  1455 rpcz_store.cc:269] 0709 13:02:46.114333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.114748 (+   415us) service_pool.cc:224] Handling call
0709 13:03:23.281559 (+37166811us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:19.555711 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:19.580895 (+ 25184us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:19.580953 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:03:19.580975 (+    22us) tablet.cc:662] Decoding operations
0709 13:03:19.834660 (+253685us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:19.834770 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:19.834800 (+    30us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:03:19.836448 (+  1648us) tablet.cc:701] Row locks acquired
0709 13:03:19.836499 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:03:19.836690 (+   191us) write_op.cc:270] Start()
0709 13:03:19.836860 (+   170us) write_op.cc:276] Timestamp: P: 1783602199836624 usec, L: 0
0709 13:03:19.836900 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:03:19.887897 (+ 50997us) log.cc:844] Serialized 8260 byte log entry
0709 13:03:20.561456 (+673559us) op_driver.cc:464] REPLICATION: finished
0709 13:03:21.220464 (+659008us) write_op.cc:301] APPLY: starting
0709 13:03:21.236357 (+ 15893us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:21.595619 (+359262us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:21.595745 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:22.061430 (+465685us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:22.063973 (+  2543us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=0,mrs_lookups=0
0709 13:03:22.070994 (+  7021us) write_op.cc:312] APPLY: finished
0709 13:03:22.108550 (+ 37556us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x7b1800060b80. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda2d4b5de1 00007fda2d4c25b1 00007fda245ed522 00007fda245ef7c1 00007fda245ef779 00007fda245ef709 00007fda245ef6d1 00007fda245ee9cd 000000000038f354 000000000038f199 00007fda245d083a
0709 13:03:22.157036 (+ 48486us) log.cc:844] Serialized 188 byte log entry
0709 13:03:22.268945 (+111909us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:22.289913 (+ 20968us) write_op.cc:454] Released schema lock
0709 13:03:22.330411 (+ 40498us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:22.373439 (+ 43028us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:22.377459 (+  4020us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":209334,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":581059,"apply.run_cpu_time_us":52149,"apply.run_wall_time_us":1142702,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91907,"cfile_init":1,"lbm_read_time_us":44502,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":3,"num_ops":21,"prepare.queue_time_us":199995,"prepare.run_cpu_time_us":39101,"prepare.run_wall_time_us":416970,"raft.queue_time_us":304454,"raft.run_cpu_time_us":12106,"raft.run_wall_time_us":582055,"replication_time_us":708766,"spinlock_wait_cycles":12140288,"thread_start_us":889234,"threads_started":5,"wal-append.queue_time_us":415744,"wal-append.run_cpu_time_us":37604,"wal-append.run_wall_time_us":1418133}]]}
I20260709 13:03:23.391974  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:23.294687  9840 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.351555  1706 raft_consensus.cc:2410] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:03:23.307545  9933 raft_consensus.cc:493] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.334033  9822 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:23.403123  1527 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:23.282812 10019 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.322698  1454 catalog_manager.cc:5697] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 15 to 21, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.415563  1454 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 18, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.382377  9930 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:03:23.389369  9896 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:23.401115  9438 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:23.365257  9984 consensus_queue.cc:237] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.391920  9931 raft_consensus.cc:2804] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:23.421077  1527 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:23.414085  9884 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:23.417995  1454 catalog_manager.cc:5697] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 11, term changed from 15 to 21, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to <none>, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 21 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.308631  9885 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.401892  9840 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:03:23.466279  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 45264.4us
I20260709 13:03:23.467506  1632 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 25.
I20260709 13:03:23.411027  9833 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:23.472349  1454 catalog_manager.cc:5697] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 20 to 26, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 26 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.400900  9898 consensus_queue.cc:1059] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.068s
I20260709 13:03:23.433650 10006 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:23.413055  9933 raft_consensus.cc:515] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.471865 10003 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:23.478713  9931 raft_consensus.cc:697] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:03:23.531543  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 279767us
I20260709 13:03:23.517740 10003 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.513203  1707 raft_consensus.cc:2468] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:03:23.503969  1454 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.540987  1601 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb
I20260709 13:03:23.545574  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.546273  1701 raft_consensus.cc:2468] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 28.
I20260709 13:03:23.548552  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:23.549100 10019 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:23.550009  9931 consensus_queue.cc:237] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.553464  9884 raft_consensus.cc:2804] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:03:23.553982  9884 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.554399  9884 raft_consensus.cc:3060] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:03:23.555172  9760 consensus_meta.cc:287] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.001s
I20260709 13:03:23.555671  9933 leader_election.cc:290] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:23.563342  9919 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.570947  9885 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:23.574105  9923 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:23.577764  1710 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:23.538058 10003 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:23.598011  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:23.599165  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 318325us
I20260709 13:03:23.541517  1454 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 5 to 15, term changed from 16 to 21, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194), VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 21 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:03:23.608088  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 67152.9us
I20260709 13:03:23.613755  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:23.617051  1454 catalog_manager.cc:5697] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 16, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.618714  1454 catalog_manager.cc:5697] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 12 to 18, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 18 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.624781  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.625521  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.627823  1705 raft_consensus.cc:2393] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 26: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:23.625823  1454 catalog_manager.cc:5697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 17 to 19, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.643502  2318 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:23.646672  1454 catalog_manager.cc:5697] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 16 to 20, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 20 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.620755 10020 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:23.620772  9847 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.657334  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.654873  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.657812  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3897029c4b5f443296641c39af287e98" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.663069  1454 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:23.664222  2318 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.664595  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:23.666801  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.658656 10020 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:23.668143 10020 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:23.674455  9886 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:23.674940  9886 raft_consensus.cc:493] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.626947 10001 raft_consensus.cc:2749] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:23.675319  9886 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:23.677347  9919 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:23.692165  1454 catalog_manager.cc:5697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 16 to 19, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 19 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.677137  9847 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.698573 10003 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.692510  9840 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.706172 10037 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.706841  9840 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:23.709944  1702 raft_consensus.cc:1275] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 13:03:23.710085  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:23.710789  1701 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:23.710222 10003 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:23.713979  1526 leader_election.cc:304] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:23.715023 10003 raft_consensus.cc:2749] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:23.706797 10037 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.719102  9768 consensus_queue.cc:1048] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:03:23.727437  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 63191.5us
W20260709 13:03:23.754436  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 60759.4us
W20260709 13:03:23.748741 10027 log.cc:927] Time spent T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:03:23.803448  1454 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.749797  9815 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:23.808359  1526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 92851.5us
I20260709 13:03:23.808367  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:23.809645  1454 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.811082  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:23.811903  1705 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:03:23.816083  1454 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:23.804437 10039 log.cc:927] Time spent T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:03:23.829366  1454 catalog_manager.cc:5697] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 12 to 21. New cstate: current_term: 21 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.836125  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.837023  1707 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
I20260709 13:03:23.839944  2353 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:23.847818  9847 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:23.865613  2353 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
W20260709 13:03:23.871325 10028 log.cc:927] Time spent T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.403s	user 0.003s	sys 0.000s
I20260709 13:03:23.873803  9564 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.877686  9564 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.879992  9564 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:23.882164  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 74500.7us
W20260709 13:03:23.883009  2318 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.884191  1454 catalog_manager.cc:5697] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.874024  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:23.886215  1454 catalog_manager.cc:5697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.875151  9840 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.889631  9885 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:23.889892  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:23.891592  1627 raft_consensus.cc:3060] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:23.875842  9833 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:23.924304  2318 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.920454  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:23.936915  2318 leader_election.cc:336] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:23.945601  1701 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:23.918260  9885 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.927537 10003 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.928629  9927 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.927806  9746 raft_consensus.cc:493] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.968753  1454 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 18 to 21, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:23.955312  9885 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:23.985944  1454 catalog_manager.cc:5697] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.987633  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.934684  9840 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.991115  1454 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 16 to 24, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 24 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:24.003811 10020 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:23.988144 10037 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:23.995762  9847 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.010401  9847 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.015215  2349 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:24.016698  9840 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:24.017084 10037 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:23.982390 10003 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.019263  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:24.020246 10003 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:24.022910  1632 raft_consensus.cc:1275] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:03:24.035337  1525 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:24.013155  9847 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:24.054389  1627 raft_consensus.cc:1275] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:24.047953  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:24.014037 10020 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:24.050613  1634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:24.055727  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:24.057723  1627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:24.058362  1627 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:24.042804  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:24.063311  1636 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:03:24.064386  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:24.065204  1635 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:24.073228  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:23.984637  9746 raft_consensus.cc:515] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:23.982851  9927 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.053532 10003 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:24.077651  9815 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:24.062233  9896 raft_consensus.cc:3055] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:24.122201  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:24.123170  2351 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 24.
I20260709 13:03:24.153173  1636 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 27.
I20260709 13:03:24.155448  1635 raft_consensus.cc:2468] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22.
I20260709 13:03:24.121289  9896 raft_consensus.cc:740] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:24.142220  1454 catalog_manager.cc:5697] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 16 to 19, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 19 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:03:24.141873  9884 consensus_meta.cc:287] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: Time spent flushing consensus metadata: real 0.587s	user 0.014s	sys 0.000s
W20260709 13:03:24.218362   431 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 1586)
I20260709 13:03:24.184530  9884 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:24.221177  9933 raft_consensus.cc:493] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.201319  1705 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 27.
I20260709 13:03:24.253345  2317 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:24.251614  9983 raft_consensus.cc:2764] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 13:03:24.256423  9933 raft_consensus.cc:515] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:24.265908 10046 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:03:24.271260  9746 leader_election.cc:290] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:24.272100  9927 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:24.284268  1630 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:24.266891  9922 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:24.283301  1526 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:24.288682 10051 log.cc:927] Time spent T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:03:24.290092  9746 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:24.288473  9885 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.290731  9746 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:24.291730  9746 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.286834  1526 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:24.295682  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:24.304879  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:24.226794  9896 consensus_queue.cc:260] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:24.307087 10049 log.cc:927] Time spent T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
W20260709 13:03:24.307780 10048 log.cc:927] Time spent T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:03:24.313906  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba40a6e1d714431be77e8a1518ec8d6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:24.318738  9886 consensus_meta.cc:287] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Time spent flushing consensus metadata: real 0.638s	user 0.003s	sys 0.009s
I20260709 13:03:24.319399  9982 consensus_queue.cc:1048] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:24.335043  1629 raft_consensus.cc:2393] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 24: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:24.319878  2352 raft_consensus.cc:1240] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Rejecting Update request from peer d62ce354d09a4be59c29094425d09bc1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:24.319731  9983 consensus_queue.cc:1048] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:24.320513  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:24.321328  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:24.321687  9868 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:24.298774  9564 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:03:24.300343  9922 raft_consensus.cc:697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:24.324023  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:24.326452  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:24.341418  2348 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:24.326629  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13debc393e8a4121883dda403cb4badf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:24.296375  9992 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:03:24.329185  9896 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:24.330679  1705 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:03:24.331667  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:24.390100  1704 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:03:24.332590  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5aa381f739e4c71869e157bdba8aa97" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:24.395831  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 45364.1us
I20260709 13:03:24.401369  1704 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 25.
I20260709 13:03:24.346220  9922 consensus_queue.cc:237] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.348126  9884 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 29 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:24.336971  9886 raft_consensus.cc:515] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.349056  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:24.349696  9564 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 27 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:24.304370  9933 leader_election.cc:290] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:24.359015  9847 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:24.360862  1601 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:24.362448  2348 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 25.
I20260709 13:03:24.363847  9934 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.371903  1527 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:24.420948  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 49142.3us
I20260709 13:03:24.374720  9854 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:24.292662  9885 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:24.405637  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 56644us
I20260709 13:03:24.408516  1526 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
W20260709 13:03:24.427232  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 66380.4us
I20260709 13:03:24.426937  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:24.430864  2349 raft_consensus.cc:2368] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 21. Current term is 22.
W20260709 13:03:24.428670  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:24.460536  9884 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:24.460677  9856 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.463979  9856 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.464521  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:24.466038  1525 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:24.468199  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:24.474275 10003 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:24.475077  9886 leader_election.cc:290] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:24.475334 10003 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:24.421168 10052 raft_consensus.cc:2804] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:24.476184 10052 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.476604 10052 raft_consensus.cc:3060] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:24.477293  9884 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 25 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:24.477991  1525 leader_election.cc:304] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:24.466629  9856 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:24.482156  1527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 60150.7us
W20260709 13:03:24.483124  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 88823.8us
I20260709 13:03:24.484467  1525 leader_election.cc:304] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:24.484740  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:24.503971  1632 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:24.512866  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:24.424575 10002 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:24.517284 10050 raft_consensus.cc:2749] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:24.427783  9934 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:24.478490  9884 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.430845  9854 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:24.496688  9760 raft_consensus.cc:3060] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:24.513774  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:24.559691  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 45731.3us
W20260709 13:03:24.561595  1675 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:24.566556  2318 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:24.427914  9847 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:24.566354  1675 leader_election.cc:336] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:24.475173  9564 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:24.571367  1675 consensus_peers.cc:597] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:24.573297  1675 leader_election.cc:336] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:24.575022  1632 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22.
I20260709 13:03:24.592204  1708 raft_consensus.cc:2410] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:03:24.594475  1704 raft_consensus.cc:1275] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:24.613088  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:24.614341  1704 raft_consensus.cc:2393] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 25: Already voted for candidate d62ce354d09a4be59c29094425d09bc1 in this term.
I20260709 13:03:24.617900  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e87d77f44ba19ad756277b557c45" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:24.618777  1705 raft_consensus.cc:3060] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:24.633008  1708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52646 (request call id 1390) took 1335 ms. Trace:
I20260709 13:03:24.633612  1708 rpcz_store.cc:276] 0709 13:03:23.297642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.298151 (+   509us) service_pool.cc:224] Handling call
0709 13:03:24.632719 (+1334568us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54002e77d4. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27643571 00007fda276de2fa 00007fda2c69fb17 00007fda25c1a6d4 00007fda25c1a662 00007fda25c1a591 00007fda25c1a50c 00007fda25c197c2 00007fda25a1548c 00007fda25a148c6 00007fda25a1426f
0709 13:03:24.632928 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2961151232}
I20260709 13:03:24.638222  1602 leader_election.cc:400] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 21. Current term is 22.
I20260709 13:03:24.638619  1602 leader_election.cc:403] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:03:24.643749  2317 leader_election.cc:304] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
W20260709 13:03:24.645802  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 162028us
I20260709 13:03:24.622378  9833 raft_consensus.cc:493] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.676285  1705 raft_consensus.cc:2468] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22.
I20260709 13:03:24.649122  9927 raft_consensus.cc:2804] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:24.649741  9854 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:24.691807  1525 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:24.706712  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:24.706429  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:24.707657  1704 raft_consensus.cc:2393] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 21: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:24.645453 10001 raft_consensus.cc:2749] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257) with higher term. Message: Invalid argument: T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for earlier term 21. Current term is 22.
I20260709 13:03:24.658155  9930 consensus_queue.cc:1048] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:03:24.660393  9833 raft_consensus.cc:515] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:24.740550  1601 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:24.737931 10036 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.767133 10036 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:24.739145 10060 log.cc:927] Time spent T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:03:24.711458  9927 raft_consensus.cc:697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:24.785074  9927 consensus_queue.cc:237] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.740099  9746 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.780769  9854 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:24.745391  9896 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 13:03:24.711122 10063 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:24.825224  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 176546us
I20260709 13:03:24.856606  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:24.866680  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:24.869700 10072 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:24.875135  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 133253us
W20260709 13:03:24.881683  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 156383us
W20260709 13:03:24.884666  1525 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 192881us
I20260709 13:03:24.882165  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:24.889850  1629 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:24.890653  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:24.865825 10063 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.897660 10052 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.898269 10063 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:24.899936 10052 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:24.809649  9746 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:24.908720  9746 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:24.913347 10066 log.cc:927] Time spent T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:03:24.915104  9833 leader_election.cc:290] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:24.873261  9958 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:24.924332  1708 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:03:24.856312  9847 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:24.926167 10036 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:24.928503  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:24.929809  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:24.929219  1701 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 21.
I20260709 13:03:24.931412  9760 raft_consensus.cc:2749] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:24.932312  9934 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:24.933161  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:24.934203  1704 raft_consensus.cc:2410] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:03:24.937863  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:24.937130  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:24.950258  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:24.945242  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:24.954216  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:24.955021  1631 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 20.
I20260709 13:03:24.956529  1629 raft_consensus.cc:2393] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 22: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:24.958244  1525 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:24.958663  1601 leader_election.cc:336] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:24.959559  1601 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:24.958490  9982 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.963371  9982 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:24.964211  9833 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:24.958556  9933 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:24.966673  9982 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:24.967103  9933 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.967806  9933 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:24.949518 10052 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:03:24.981945  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb801d86c874e1686eceacb9de1d0a6" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:24.988197  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 121538us
I20260709 13:03:24.985793  9884 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.986588 10006 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:25.005863  9884 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:24.989454 10052 raft_consensus.cc:493] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:24.995816  1601 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:25.014149  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:25.018741  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:25.021714  9884 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:25.013605  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:25.023233  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:25.023149  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:25.023860  2346 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:25.027045  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:25.017400  1525 leader_election.cc:304] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:25.023867  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:25.029660  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:25.030112  1675 leader_election.cc:336] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:25.030385  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:25.030560  1675 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:25.031442  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
W20260709 13:03:25.031668  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:25.033555  9921 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:25.032124  9840 raft_consensus.cc:2749] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:25.022565 10052 raft_consensus.cc:515] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:25.038669  2346 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:03:25.034065  1601 leader_election.cc:336] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:25.045532  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:25.032326 10006 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:25.043954  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:25.047796  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:25.044257  1675 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:25.061630 10006 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:25.063705  1527 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:25.065649 10074 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.066473 10074 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } }
I20260709 13:03:25.067420  9840 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:25.068492  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e952b7f0f34618a5ce435fa48e0448" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:25.049933  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:25.069640  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:25.023895  9933 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:25.075554  9934 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:25.076162  9933 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:25.051512  9564 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:25.076957  1527 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:25.043413 10052 leader_election.cc:290] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:25.078132 10074 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:25.076164  9934 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.086557  9564 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:25.086922  9934 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:25.081311  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:25.089875  1701 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:25.089820  9564 raft_consensus.cc:493] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.090757  9564 raft_consensus.cc:515] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:25.073268  1703 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:03:25.093356  9564 leader_election.cc:290] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:25.096402 10052 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:25.100488 10052 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:25.104243  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:25.104916  2346 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:25.107146  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:25.107719  2346 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:25.114638  1635 raft_consensus.cc:1275] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:25.117837  1525 leader_election.cc:304] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:25.123157  9934 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:25.126257  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:25.126860  1710 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:25.128844  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 25 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:25.129990  1707 raft_consensus.cc:2410] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e for term 25 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:03:25.133419  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:25.133662  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:25.137418  1701 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:25.138417  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c08bb7c6294cf9b33ad3b85e34831d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:25.139473  9847 raft_consensus.cc:2749] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:25.144481  1710 raft_consensus.cc:2410] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:03:25.146852  1526 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:25.148988  9840 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:25.149681  9840 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:25.154246  9840 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:25.160709  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:25.165854  1675 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:25.171518  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:25.172581  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:25.177062  9934 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:25.183250 10006 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:25.185163  1635 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:25.188724  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:25.211836 10067 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.212440 10067 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:25.219069   431 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 1586)
I20260709 13:03:25.219677  9840 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.222971  9840 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:25.225242  9840 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:25.226946  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:25.227854  1635 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 22.
I20260709 13:03:25.229198  1629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8516bd3d9b8948f0a0a32c2e93d4673e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:25.229362 10067 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:25.231228  1525 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:25.233109  9840 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:25.234105  1527 leader_election.cc:304] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:25.235545  9840 raft_consensus.cc:2749] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:25.235423  1631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:25.238714  1631 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 24.
I20260709 13:03:25.240705  9958 consensus_queue.cc:1048] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:25.255738  2351 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:03:25.257862  1704 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:03:25.258363  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:25.258937  2351 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:25.259405  9833 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:25.263034  1635 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
W20260709 13:03:25.264031  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 123087us
I20260709 13:03:25.276518  2351 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 23.
W20260709 13:03:25.279565  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 142925us
I20260709 13:03:25.280241  1527 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:25.280616 10052 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:03:25.286019  9933 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:25.289959  1676 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:25.291229  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:25.293179  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:25.293826  1710 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 24.
W20260709 13:03:25.297621  1601 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:25.298251  1601 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:25.286711  9933 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:25.269930  9934 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:03:25.296775  9653 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:25.297147 10079 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.294886  9438 raft_consensus.cc:2764] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:03:25.303678  9943 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:25.324726  9564 raft_consensus.cc:493] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.325274  9564 raft_consensus.cc:515] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:25.330422  9564 leader_election.cc:290] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:25.333381  1703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:25.312410  9653 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.336803  9653 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:25.338176  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74f866a699e74063990bd566c7fb1ebf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:25.339632  1525 leader_election.cc:304] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:25.341531 10052 raft_consensus.cc:2749] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:25.341894  9933 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:25.343276  2318 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:25.344122  9922 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:25.344521  9922 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.344861  9922 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:25.350968  1710 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:25.360778  9992 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:25.319358 10079 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:25.368130 10079 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:25.369182  1635 raft_consensus.cc:1275] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:25.371573 10072 consensus_queue.cc:1048] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:25.372530  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:25.386204  9760 raft_consensus.cc:493] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.386893  9760 raft_consensus.cc:515] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:25.390784  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:25.391371  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac4670b4c614e9eb392b74198615198" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:25.392899  1525 leader_election.cc:304] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:25.398272 10052 raft_consensus.cc:2749] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:25.389436  9760 leader_election.cc:290] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:25.409582  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:25.410192  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:25.411010  2318 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:25.413656  9922 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:25.416308  2318 leader_election.cc:336] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:25.417920  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:25.443186  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:25.487164  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44029.1us
I20260709 13:03:25.418020  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:25.488080  1632 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:25.416453  9922 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:25.504397  1632 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 25.
W20260709 13:03:25.508562  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 92222.9us
I20260709 13:03:25.538480  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:25.539091  1701 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:25.510525 10079 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:25.549079  1701 raft_consensus.cc:2468] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 25.
I20260709 13:03:25.550757  2318 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:25.532042  9934 raft_consensus.cc:493] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:25.568638  9934 raft_consensus.cc:515] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:25.570847  9934 leader_election.cc:290] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:25.572288  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:25.572543  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b24efe048e6499b923f1a0215713fad" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:25.573410  9992 raft_consensus.cc:2804] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:25.578006  9992 raft_consensus.cc:697] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:25.579053  9992 consensus_queue.cc:237] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:25.548352  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:25.604678  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 56374.6us
I20260709 13:03:25.608709  1526 leader_election.cc:304] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:25.612411  9933 raft_consensus.cc:2749] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:25.432300  9653 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:25.627826  9934 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.635282  9934 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:25.637369  9934 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:25.640543  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:25.641064  2318 leader_election.cc:336] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:25.652711  1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:25.654572  9917 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:25.655086  9917 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:25.663581  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:25.664741  9917 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:25.673246  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:25.674108  2348 raft_consensus.cc:2393] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 25: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
W20260709 13:03:25.688841 10093 log.cc:927] Time spent T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:03:25.695729 10099 log.cc:927] Time spent T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:03:25.700018  9653 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:25.703624 10102 log.cc:927] Time spent T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:03:25.707500  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:25.708115  1630 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:25.713205  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:25.713889  2348 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:25.721060  1630 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 24.
I20260709 13:03:25.723409  2348 raft_consensus.cc:2468] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 24.
I20260709 13:03:25.724542  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:25.754451  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:25.761759  9884 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:25.766836  1675 leader_election.cc:336] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:25.767719  1675 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:25.770373  1635 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:25.773078  1704 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:03:25.773554  1454 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 192) took 12905 ms (client timeout 15000 ms). Trace:
I20260709 13:03:25.774425  9564 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:03:25.773942  1454 rpcz_store.cc:269] 0709 13:03:12.868200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.868571 (+   371us) service_pool.cc:224] Handling call
0709 13:03:25.773462 (+12904891us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:24.614882 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:24.807482 (+192600us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:24.807529 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:03:24.807546 (+    17us) tablet.cc:662] Decoding operations
0709 13:03:24.904249 (+ 96703us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:24.904310 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:24.904337 (+    27us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:03:24.905956 (+  1619us) tablet.cc:701] Row locks acquired
0709 13:03:24.905977 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:03:24.906118 (+   141us) write_op.cc:270] Start()
0709 13:03:24.906237 (+   119us) write_op.cc:276] Timestamp: P: 1783602204906072 usec, L: 0
0709 13:03:24.906256 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:03:24.906957 (+   701us) log.cc:844] Serialized 8735 byte log entry
0709 13:03:24.921799 (+ 14842us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b4400022b4c. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda24513c85 00007fda245146a2 00007fda272861db 00007fda27281aaf 00007fda245d00ac 00007fda2d42aab9 00007fda245ec69b 00007fda245ea93f 00007fda245ecc02 00007fda2727420b
0709 13:03:25.052431 (+130632us) op_driver.cc:464] REPLICATION: finished
0709 13:03:25.477952 (+425521us) write_op.cc:301] APPLY: starting
0709 13:03:25.478074 (+   122us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:25.691196 (+213122us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:25.691229 (+    33us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:25.741715 (+ 50486us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:25.742100 (+   385us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=0,mrs_lookups=0
0709 13:03:25.742147 (+    47us) write_op.cc:312] APPLY: finished
0709 13:03:25.742560 (+   413us) log.cc:844] Serialized 196 byte log entry
0709 13:03:25.742724 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:25.742912 (+   188us) write_op.cc:454] Released schema lock
0709 13:03:25.742964 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:25.743408 (+   444us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:25.743443 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":587,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":425220,"apply.run_cpu_time_us":29283,"apply.run_wall_time_us":265454,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96087,"num_ops":22,"prepare.queue_time_us":281246,"prepare.run_cpu_time_us":16824,"prepare.run_wall_time_us":414924,"raft.queue_time_us":34136,"raft.run_cpu_time_us":132598,"raft.run_wall_time_us":246800,"replication_time_us":145991,"spinlock_wait_cycles":30384896,"thread_start_us":476732,"threads_started":4,"wal-append.queue_time_us":101912}]]}
I20260709 13:03:25.777550  9884 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:25.779521  1676 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:25.800592  9917 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:03:25.813302  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:25.810357  9653 raft_consensus.cc:2804] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:03:25.814085  1675 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:25.818783  9653 raft_consensus.cc:697] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:03:25.840917  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:25.846424  1452 catalog_manager.cc:5697] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.862761  9653 consensus_queue.cc:237] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:25.874028  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:25.889006  9917 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:25.849136  1452 catalog_manager.cc:5697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 14 to 17, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:25.891601  1452 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.892958  1452 catalog_manager.cc:5697] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.894742  1452 catalog_manager.cc:5697] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 15 to 18, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:25.910995  1452 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 14 to 15, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.915759  1704 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 13:03:25.916280  1704 pending_rounds.cc:85] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting all ops after (but not including) 14
I20260709 13:03:25.916556  1704 pending_rounds.cc:107] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:03:25.916843  1704 raft_consensus.cc:2889] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:25.918709  9934 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:25.934839  9840 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.935367  9840 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:25.940021  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:25.942662  9840 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:25.944314  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:25.946327  1527 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:25.947264  9934 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:25.948819  1452 catalog_manager.cc:5697] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.960341  1452 catalog_manager.cc:5697] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 5 to 15, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.964016  1452 catalog_manager.cc:5697] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 14 to 18, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:25.969314  1452 catalog_manager.cc:5697] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 8 to 17, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:25.972321  2348 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:03:25.979236  9840 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:25.990813  9564 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.991356  9564 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:25.996361  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:25.999220  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:26.006879  1526 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:26.007622  9840 raft_consensus.cc:493] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.007751  9934 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:26.008107  9840 raft_consensus.cc:515] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:26.011345  9840 leader_election.cc:290] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:26.014274  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:26.016001  9934 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.016757  9934 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:26.019225  9934 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:26.021673  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696fb3c290c54f14a67f61bf78588eda" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:26.023496  9564 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:26.024003  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:26.024605  2346 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 25.
I20260709 13:03:26.025622  1526 leader_election.cc:304] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:26.027350  9564 raft_consensus.cc:2749] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:26.027347  9884 raft_consensus.cc:493] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.028105  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:26.028811  1704 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 25.
I20260709 13:03:26.028637  9884 raft_consensus.cc:515] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:26.031549  9884 leader_election.cc:290] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:26.034402  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:26.029559  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:26.039366  9564 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:26.039746  9564 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.039686  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d72a925fb22046e2be514a7913c613a4" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:26.040243  9564 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:26.043223  1527 leader_election.cc:304] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:26.045143  9840 raft_consensus.cc:2749] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:26.053752  1704 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:03:26.055613 10080 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:26.057236  9884 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.057956  9884 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:26.059230  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:26.055009  9564 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:26.059396  1704 raft_consensus.cc:2955] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 5 to 15, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.062292  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:26.072371  1710 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:26.064884  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:26.076056  1630 raft_consensus.cc:2393] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 25: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:26.066103  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:26.067279  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:26.087745  2348 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:26.088570  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:26.089466  9933 raft_consensus.cc:3060] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:26.067579  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:26.100245  2348 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 26.
I20260709 13:03:26.065564  9884 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:26.123984  1527 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1; no voters: 
I20260709 13:03:26.061470  9564 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 26 election: Requested vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:26.121006  1710 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62ce354d09a4be59c29094425d09bc1 in term 26.
I20260709 13:03:26.124804  9884 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:03:26.130285  9884 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 26 LEADER]: Becoming Leader. State: Replica: d62ce354d09a4be59c29094425d09bc1, State: Running, Role: LEADER
I20260709 13:03:26.131218  9884 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:26.107059  9933 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:26.145823  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:26.160576  9564 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.161273  9564 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:26.165292  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:26.166195  1630 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 24: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
W20260709 13:03:26.166435  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:26.169245  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:26.169512  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:26.170229  2348 raft_consensus.cc:2393] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d62ce354d09a4be59c29094425d09bc1 in current term 24: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:26.170976  1526 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:26.171550  9564 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:26.172181  9564 raft_consensus.cc:3060] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:26.183624  1630 raft_consensus.cc:1275] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:03:26.193094  9564 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:26.219751   431 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 1586)
I20260709 13:03:26.237128 10072 consensus_queue.cc:1048] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.009s
W20260709 13:03:26.256347  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:26.265398  2348 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:03:26.266829  9653 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:03:26.285365  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:26.293673  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:26.303032  1630 raft_consensus.cc:2955] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 5 to 15, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:03:26.317361  1452 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 276) took 48994 ms (client timeout 15000 ms). Trace:
I20260709 13:03:26.321527  1456 catalog_manager.cc:5697] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: config changed from index 11 to 3, term changed from 21 to 18, leader changed from <none> to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from VOTER to NON_VOTER, now has a pending config: VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194), VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 18 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:03:26.319020  1452 rpcz_store.cc:269] 0709 13:02:37.323036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.323474 (+   438us) service_pool.cc:224] Handling call
0709 13:03:26.317271 (+48993797us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:26.032981 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:26.036202 (+  3221us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:26.036246 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:03:26.036267 (+    21us) tablet.cc:662] Decoding operations
0709 13:03:26.046252 (+  9985us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:26.046337 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:26.046364 (+    27us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:03:26.047133 (+   769us) tablet.cc:701] Row locks acquired
0709 13:03:26.047152 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:03:26.047287 (+   135us) write_op.cc:270] Start()
0709 13:03:26.047388 (+   101us) write_op.cc:276] Timestamp: P: 1783602206047241 usec, L: 0
0709 13:03:26.047406 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:03:26.050501 (+  3095us) log.cc:844] Serialized 4034 byte log entry
0709 13:03:26.138495 (+ 87994us) op_driver.cc:464] REPLICATION: finished
0709 13:03:26.139198 (+   703us) write_op.cc:301] APPLY: starting
0709 13:03:26.139290 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:26.148711 (+  9421us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:26.148799 (+    88us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:26.170580 (+ 21781us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:26.170793 (+   213us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 13:03:26.170855 (+    62us) write_op.cc:312] APPLY: finished
0709 13:03:26.171129 (+   274us) log.cc:844] Serialized 97 byte log entry
0709 13:03:26.171294 (+   165us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:26.171415 (+   121us) write_op.cc:454] Released schema lock
0709 13:03:26.171477 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:26.184272 (+ 12795us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:26.184303 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1033433,"reported_tablets":48,"spinlock_wait_cycles":24192,"child_traces":[["op",{"apply.queue_time_us":451,"apply.run_cpu_time_us":12751,"apply.run_wall_time_us":45037,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41800,"num_ops":10,"prepare.queue_time_us":17933,"prepare.run_cpu_time_us":3969,"prepare.run_wall_time_us":23824,"replication_time_us":90935,"thread_start_us":21722,"threads_started":1}]]}
I20260709 13:03:26.334894  1456 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 15 to 21, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.336745  1456 catalog_manager.cc:5697] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 14 to 19, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:26.342478  1456 catalog_manager.cc:5697] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:26.343925  1456 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 15 to 16, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.346700  1456 catalog_manager.cc:5697] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 13 to 16, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.349102  1456 catalog_manager.cc:5697] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 17 to 19, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:26.352097  1456 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 15 to 16, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.353842  1456 catalog_manager.cc:5697] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 19 to 21, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 21 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:26.355623  1456 catalog_manager.cc:5697] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 15 to 16, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.358202  1456 catalog_manager.cc:5697] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 14 to 17, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 17 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:26.363284  1630 raft_consensus.cc:1275] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:03:26.364538  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:26.365180  9438 consensus_queue.cc:1048] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:26.360570  1456 catalog_manager.cc:5697] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 12 to 17, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.384841  1456 catalog_manager.cc:5697] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:26.404495  9653 raft_consensus.cc:2955] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Committing config change with OpId 18.11: config changed from index 3 to 11, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.409956  1630 raft_consensus.cc:2955] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Committing config change with OpId 18.11: config changed from index 3 to 11, 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.419977  1456 catalog_manager.cc:5697] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 16 to 21, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.425549  1456 catalog_manager.cc:5697] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 15 to 18, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.435624  1456 catalog_manager.cc:5697] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 18 to 21, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 21 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:26.437497  1456 catalog_manager.cc:5697] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.440047  9884 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.440616  9884 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:26.443125  9884 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:26.444170  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:26.439077  1456 catalog_manager.cc:5697] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 reported cstate change: term changed from 16 to 21, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.444545  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:26.448459  1526 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:26.452636  9884 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:26.529488  9933 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.530059  9933 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:26.532164  9933 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:26.535022  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:26.536834  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69b9dbe96d28487fa6d8a381ac6fc383" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:26.542069  1526 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:26.542829  9564 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:03:26.558465  1454 catalog_manager.cc:5697] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.560067  1454 catalog_manager.cc:5697] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 16 to 19, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 19 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.561971  1454 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.570613  1454 catalog_manager.cc:5697] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.572077  1454 catalog_manager.cc:5697] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 13 to 16, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.573673  1454 catalog_manager.cc:5697] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 21 to 25, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 25 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.575779  1454 catalog_manager.cc:5697] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 19 to 21, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 21 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:26.569160  1456 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 276) took 32265 ms (client timeout 15000 ms). Trace:
W20260709 13:03:26.587706  1456 rpcz_store.cc:269] 0709 13:02:54.303850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.304062 (+   212us) service_pool.cc:224] Handling call
0709 13:03:26.569061 (+32264999us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:26.462525 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:26.463233 (+   708us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:26.463280 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:03:26.463300 (+    20us) tablet.cc:662] Decoding operations
0709 13:03:26.464979 (+  1679us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:26.465045 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:26.465078 (+    33us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:03:26.468101 (+  3023us) tablet.cc:701] Row locks acquired
0709 13:03:26.468123 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:03:26.468262 (+   139us) write_op.cc:270] Start()
0709 13:03:26.468381 (+   119us) write_op.cc:276] Timestamp: P: 1783602206468218 usec, L: 0
0709 13:03:26.468408 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:03:26.469078 (+   670us) log.cc:844] Serialized 7432 byte log entry
0709 13:03:26.471541 (+  2463us) op_driver.cc:464] REPLICATION: finished
0709 13:03:26.471859 (+   318us) write_op.cc:301] APPLY: starting
0709 13:03:26.471940 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:26.480383 (+  8443us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:26.480467 (+    84us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:26.519682 (+ 39215us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:26.519949 (+   267us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 13:03:26.519998 (+    49us) write_op.cc:312] APPLY: finished
0709 13:03:26.520355 (+   357us) log.cc:844] Serialized 164 byte log entry
0709 13:03:26.520540 (+   185us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:26.520703 (+   163us) write_op.cc:454] Released schema lock
0709 13:03:26.520754 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:26.521080 (+   326us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:26.521111 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":105,"apply.run_cpu_time_us":20184,"apply.run_wall_time_us":49205,"cfile_cache_hit":72,"cfile_cache_hit_bytes":75240,"num_ops":18,"prepare.queue_time_us":243,"prepare.run_cpu_time_us":5290,"prepare.run_wall_time_us":6851,"replication_time_us":3002}]]}
I20260709 13:03:26.592921  1710 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:03:26.577368  1454 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 19 to 22, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 22 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.594007  1454 catalog_manager.cc:5697] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.595568  1454 catalog_manager.cc:5697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 10 to 20, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 20 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.596429  9933 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 13:03:26.609691  1454 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 16 to 21, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.611320  1454 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 21 to 25, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 25 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.612655  1454 catalog_manager.cc:5697] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.614452  1454 catalog_manager.cc:5697] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 21 to 24, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 24 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.616276  1454 catalog_manager.cc:5697] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.617681  1454 catalog_manager.cc:5697] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.619325  1454 catalog_manager.cc:5697] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 19 to 21, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 21 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.621891  1454 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.624145  1454 catalog_manager.cc:5697] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 15 to 18, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 18 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.627120  1454 catalog_manager.cc:5697] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 14 to 20, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 20 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.628772  1454 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.630015  1454 catalog_manager.cc:5697] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:26.677207  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:26.679597  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:26.705053  2348 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Refusing update from remote peer d62ce354d09a4be59c29094425d09bc1: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:03:26.707752  9933 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:03:26.631856  1454 catalog_manager.cc:5697] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 18 to 22, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:26.758905  1454 catalog_manager.cc:5697] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.763072  1454 catalog_manager.cc:5697] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.765775  1454 catalog_manager.cc:5697] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 24 to 26, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 26 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.768603  1454 catalog_manager.cc:5697] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 19 to 22, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:03:26.941009  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:26.956393  1675 consensus_peers.cc:597] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:27.026152  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:27.105851  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:27.151311  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:27.182190 10119 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.183213 10119 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:27.191118  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:27.198810 10119 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:27.200536  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0adeaf85877a4103bbd070927413d267" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:27.203285  1527 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:27.204602  9564 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:27.215941  1450 catalog_manager.cc:5697] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 18 to 23, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 23 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.219467  1450 catalog_manager.cc:5697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.226656  1450 catalog_manager.cc:5697] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 16 to 19, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.222852  1454 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42922 (request call id 193) took 1325 ms. Trace:
W20260709 13:03:27.230472   431 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 1586)
I20260709 13:03:27.240773  1454 rpcz_store.cc:276] 0709 13:03:25.897431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.897606 (+   175us) service_pool.cc:224] Handling call
0709 13:03:25.909960 (+ 12354us) spinlock_profiling.cc:243] Waited 6.86 ms on lock 0x7b5400179b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27644090 00007fda276e2f3a 00007fda2d4991cc 00007fda2d59737e 00007fda2ba582a4 00007fda2ba58232 00007fda2ba58161 00007fda2ba580dc 00007fda2ba57392 00007fda25a1548c 00007fda25a148c6
0709 13:03:27.222722 (+1312762us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:26.837295 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:26.838244 (+   949us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:26.838315 (+    71us) write_op.cc:435] Acquired schema lock
0709 13:03:26.838350 (+    35us) tablet.cc:662] Decoding operations
0709 13:03:26.841367 (+  3017us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:26.841495 (+   128us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:26.841529 (+    34us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:03:26.857698 (+ 16169us) tablet.cc:701] Row locks acquired
0709 13:03:26.857752 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:03:26.857955 (+   203us) write_op.cc:270] Start()
0709 13:03:26.858180 (+   225us) write_op.cc:276] Timestamp: P: 1783602206857888 usec, L: 0
0709 13:03:26.858217 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:03:26.883959 (+ 25742us) log.cc:844] Serialized 10676 byte log entry
0709 13:03:26.891883 (+  7924us) op_driver.cc:464] REPLICATION: finished
0709 13:03:26.892806 (+   923us) write_op.cc:301] APPLY: starting
0709 13:03:26.892924 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:27.036584 (+143660us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:27.036730 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:27.124296 (+ 87566us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:27.124828 (+   532us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=0,mrs_lookups=0
0709 13:03:27.124905 (+    77us) write_op.cc:312] APPLY: finished
0709 13:03:27.125586 (+   681us) log.cc:844] Serialized 236 byte log entry
0709 13:03:27.125816 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:27.126099 (+   283us) write_op.cc:454] Released schema lock
0709 13:03:27.126202 (+   103us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:27.149722 (+ 23520us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:27.149764 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9252,"reported_tablets":34,"spinlock_wait_cycles":15769984,"child_traces":[["op",{"apply.queue_time_us":548,"apply.run_cpu_time_us":42950,"apply.run_wall_time_us":256820,"cfile_cache_hit":108,"cfile_cache_hit_bytes":112860,"num_ops":27,"prepare.queue_time_us":1285,"prepare.run_cpu_time_us":9473,"prepare.run_wall_time_us":47206,"replication_time_us":33444}]]}
W20260709 13:03:27.244524  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:27.258446  1450 catalog_manager.cc:5697] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 21 to 22, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.262058  1450 catalog_manager.cc:5697] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:27.275065  1450 catalog_manager.cc:5697] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 17 to 20, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.287175  1450 catalog_manager.cc:5697] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 17 to 20, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.294615  1450 catalog_manager.cc:5697] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 9 to 19. New cstate: current_term: 19 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.314599  1450 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 18 to 22, leader changed from 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 22 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:27.317014  1450 catalog_manager.cc:5697] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 17 to 23, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 23 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.322031  1450 catalog_manager.cc:5697] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 14 to 20, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:27.332687  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:27.376276 10119 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.377045 10119 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:27.380117 10119 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:27.381791  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:27.384058  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b022b45a9b7462bb4b1a2a30ae36aa9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:27.386308  1526 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:27.387122  9564 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:27.404119  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:27.414996  9564 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.415813  9564 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:27.418251  9564 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:27.420532  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:27.423122  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b73223ab67f4b5aab700c65a70e42cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:27.434094  1454 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:27.439703  1527 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
W20260709 13:03:27.443995  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:27.445600  1454 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 19 to 23, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 23 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:27.449239 10119 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:27.460196  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:27.473140  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:27.473958  1454 catalog_manager.cc:5697] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 11, term changed from 18 to 24, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193), VOTER ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194), VOTER a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195), VOTER 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 24 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:03:27.525355  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:27.534008  9564 raft_consensus.cc:493] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.534924  9564 raft_consensus.cc:515] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:27.547827  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:27.548689  1455 catalog_manager.cc:5697] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:27.551348  1455 catalog_manager.cc:5697] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:27.552827  1455 catalog_manager.cc:5697] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 21 to 28, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 28 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:27.555076  1455 catalog_manager.cc:5697] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 16 to 19, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:27.561308  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:27.562067  9564 leader_election.cc:290] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:27.563176  1527 leader_election.cc:304] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:27.569530  9884 raft_consensus.cc:2749] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:27.572362  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:27.564915  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "637e660a66be4924866287fa6150aaea" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:27.556695  1455 catalog_manager.cc:5697] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 12 to 24. New cstate: current_term: 24 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:27.575229  1450 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 277) took 12734 ms (client timeout 15000 ms). Trace:
I20260709 13:03:27.566097 10119 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.577888 10119 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:27.581411  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:27.582829  1455 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:27.584466  1455 catalog_manager.cc:5697] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 21 to 23, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 23 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.595441  1455 catalog_manager.cc:5697] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
W20260709 13:03:27.575747  1450 rpcz_store.cc:269] 0709 13:03:14.840462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.840758 (+   296us) service_pool.cc:224] Handling call
0709 13:03:27.575113 (+12734355us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:27.351923 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:27.352530 (+   607us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:27.352578 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:03:27.352596 (+    18us) tablet.cc:662] Decoding operations
0709 13:03:27.353722 (+  1126us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:27.353790 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:27.353820 (+    30us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:03:27.355002 (+  1182us) tablet.cc:701] Row locks acquired
0709 13:03:27.355025 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:03:27.355189 (+   164us) write_op.cc:270] Start()
0709 13:03:27.355329 (+   140us) write_op.cc:276] Timestamp: P: 1783602207355143 usec, L: 0
0709 13:03:27.355352 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:03:27.356136 (+   784us) log.cc:844] Serialized 4417 byte log entry
0709 13:03:27.358834 (+  2698us) op_driver.cc:464] REPLICATION: finished
0709 13:03:27.359405 (+   571us) write_op.cc:301] APPLY: starting
0709 13:03:27.359510 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:27.369460 (+  9950us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:27.371090 (+  1630us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:27.382789 (+ 11699us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:27.383091 (+   302us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 13:03:27.419100 (+ 36009us) write_op.cc:312] APPLY: finished
0709 13:03:27.419554 (+   454us) log.cc:844] Serialized 105 byte log entry
0709 13:03:27.420526 (+   972us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:27.420723 (+   197us) write_op.cc:454] Released schema lock
0709 13:03:27.420796 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:27.421288 (+   492us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:27.421321 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21779,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":286,"apply.run_cpu_time_us":15717,"apply.run_wall_time_us":61808,"cfile_cache_hit":44,"cfile_cache_hit_bytes":45980,"num_ops":11,"prepare.queue_time_us":121,"prepare.run_cpu_time_us":4515,"prepare.run_wall_time_us":4516,"replication_time_us":3312}]]}
I20260709 13:03:27.609757  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:27.611141  1455 catalog_manager.cc:5697] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 20 to 21, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:27.614624  1526 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:27.618670  9884 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:27.616319  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:27.596910 10119 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:27.612914  1455 catalog_manager.cc:5697] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:27.694782  1455 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:27.699942  1455 catalog_manager.cc:5697] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:27.701560  1455 catalog_manager.cc:5697] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:27.722657  1455 catalog_manager.cc:5697] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 17 to 18, leader changed from a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195) to d62ce354d09a4be59c29094425d09bc1 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
W20260709 13:03:27.730942  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:27.878250  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:27.882143  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:27.892819  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:28.006994  1601 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 251 similar messages]
I20260709 13:03:28.032902 10119 raft_consensus.cc:493] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.033743 10119 raft_consensus.cc:515] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:28.037262  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:28.041278  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fb354b7d6104034b98fab8201f2719e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:28.043183  1526 leader_election.cc:304] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:28.062036  9884 raft_consensus.cc:2749] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:28.070447  9884 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.071259  9884 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:28.074157  9884 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:28.075047  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:28.076010  1455 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 276) took 4225 ms. Trace:
I20260709 13:03:28.071399 10119 leader_election.cc:290] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:28.076468  1455 rpcz_store.cc:276] 0709 13:03:23.850357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.850655 (+   298us) service_pool.cc:224] Handling call
0709 13:03:28.075917 (+4225262us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:27.751575 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:27.752164 (+   589us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:27.752205 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:03:27.752223 (+    18us) tablet.cc:662] Decoding operations
0709 13:03:27.753632 (+  1409us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:27.753690 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:27.753718 (+    28us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:03:27.754995 (+  1277us) tablet.cc:701] Row locks acquired
0709 13:03:27.755016 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:03:27.755448 (+   432us) write_op.cc:270] Start()
0709 13:03:27.755597 (+   149us) write_op.cc:276] Timestamp: P: 1783602207755382 usec, L: 0
0709 13:03:27.755625 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:03:27.756392 (+   767us) log.cc:844] Serialized 5586 byte log entry
0709 13:03:27.762926 (+  6534us) op_driver.cc:464] REPLICATION: finished
0709 13:03:27.763312 (+   386us) write_op.cc:301] APPLY: starting
0709 13:03:27.763421 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:27.795847 (+ 32426us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:27.795948 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:27.815958 (+ 20010us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:27.816192 (+   234us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=0,mrs_lookups=0
0709 13:03:27.816234 (+    42us) write_op.cc:312] APPLY: finished
0709 13:03:27.816525 (+   291us) log.cc:844] Serialized 129 byte log entry
0709 13:03:27.850785 (+ 34260us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:27.850956 (+   171us) write_op.cc:454] Released schema lock
0709 13:03:27.851113 (+   157us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:27.877794 (+ 26681us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:27.877828 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":58051,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":116,"apply.run_cpu_time_us":17729,"apply.run_wall_time_us":114429,"cfile_cache_hit":56,"cfile_cache_hit_bytes":58520,"num_ops":14,"prepare.queue_time_us":118,"prepare.run_cpu_time_us":4965,"prepare.run_wall_time_us":14109,"replication_time_us":7127}]]}
I20260709 13:03:28.088418  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:28.096887  9884 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.101516  9884 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:28.106045  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2ba61e8020a4800b9e7fdae5533800b" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:28.122421  1525 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:28.124063 10119 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:28.132341  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:28.142256 10119 raft_consensus.cc:493] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.148741 10119 raft_consensus.cc:515] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:28.152215 10119 leader_election.cc:290] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:28.155678  9564 raft_consensus.cc:493] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:28.156267  1458 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 272) took 103491 ms (client timeout 15000 ms). Trace:
W20260709 13:03:28.228583  1458 rpcz_store.cc:269] 0709 13:01:44.664487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.664850 (+   363us) service_pool.cc:224] Handling call
0709 13:03:28.155992 (+103491142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":375255,"reported_tablets":49}
I20260709 13:03:28.228593  9564 raft_consensus.cc:515] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:28.232055  9564 leader_election.cc:290] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:28.233405  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:28.163430  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:28.183444  1453 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40384 (request call id 277) took 7425 ms. Trace:
W20260709 13:03:28.197783  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:28.237764  1453 rpcz_store.cc:276] 0709 13:03:20.794519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.820424 (+ 25905us) service_pool.cc:224] Handling call
0709 13:03:21.726519 (+906095us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400179b54. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27644090 00007fda276e2f3a 00007fda2d4991cc 00007fda2d59737e 00007fda2ba582a4 00007fda2ba58232 00007fda2ba58161 00007fda2ba580dc 00007fda2ba57392 00007fda25a1548c 00007fda25a148c6
0709 13:03:28.183205 (+6456686us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68521,"reported_tablets":48,"spinlock_wait_cycles":240764160}
W20260709 13:03:28.241580   431 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 1586)
I20260709 13:03:28.249650  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:28.253432   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:28.256536   431 tablet_replica.cc:333] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.258675   431 raft_consensus.cc:2243] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.261189   431 raft_consensus.cc:2272] T 8cb801d86c874e1686eceacb9de1d0a6 P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.264007  9884 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:28.273950   431 tablet_replica.cc:333] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.274402  9564 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.275100  9564 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:28.277066   431 raft_consensus.cc:2243] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.283280   431 raft_consensus.cc:2272] T 5ba40a6e1d714431be77e8a1518ec8d6 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.278908  9564 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:28.279264  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:28.304119  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:28.304489   431 tablet_replica.cc:333] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.308043  1527 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:28.315920  9564 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:28.319242  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c211dcb5c947719413e384c6a72769" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:28.326016   431 raft_consensus.cc:2243] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.326001  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316c6fe30c8f416c942b4338f4e6eab9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:28.327378   431 raft_consensus.cc:2272] T d72a925fb22046e2be514a7913c613a4 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.333150  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a92c141f564f638497aa02dd5c7c79" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:28.336781  9564 raft_consensus.cc:493] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.337448  9564 raft_consensus.cc:515] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:28.339823  9564 leader_election.cc:290] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:28.340909   431 tablet_replica.cc:333] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
W20260709 13:03:28.345441  1451 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 275) took 25915 ms (client timeout 15000 ms). Trace:
W20260709 13:03:28.353711  1451 rpcz_store.cc:269] 0709 13:03:02.507313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.637342 (+130029us) service_pool.cc:224] Handling call
0709 13:03:28.345175 (+25707833us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":464019,"reported_tablets":49}
I20260709 13:03:28.354279  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb99ad78ebb2479fbf4715bb9d33258e" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:28.346608   431 raft_consensus.cc:2243] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:28.348131  9884 raft_consensus.cc:493] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.349275  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:28.357697   431 raft_consensus.cc:2272] T ca209b0d98c64e1da48e3027b3ea1d9a P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.357613  9884 raft_consensus.cc:515] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:28.360040  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfe7ffa6604774b0b035da58f96eb9" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:28.370599  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:28.372844   431 tablet_replica.cc:333] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.373829  1525 leader_election.cc:304] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:28.375778 10120 raft_consensus.cc:2749] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:28.376407   431 raft_consensus.cc:2243] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:03:28.386324   431 raft_consensus.cc:2272] T 43dce0aaab204cb084a73a74a80bd7a3 P d62ce354d09a4be59c29094425d09bc1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.387306  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f9010f94804fcaacae8539ddfc64a7" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:28.389137  1527 leader_election.cc:304] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:28.379482  9884 leader_election.cc:290] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:28.390208 10119 raft_consensus.cc:2749] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:28.392629   431 tablet_replica.cc:333] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.399374   431 raft_consensus.cc:2243] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.402034   431 raft_consensus.cc:2272] T 637e660a66be4924866287fa6150aaea P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.418862   431 tablet_replica.cc:333] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
W20260709 13:03:28.415246  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:28.420374  1525 leader_election.cc:304] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:28.431088 10121 raft_consensus.cc:2749] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:28.432130  1525 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:28.435251  1525 leader_election.cc:304] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:28.433017   431 raft_consensus.cc:2243] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:28.435761 10119 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:28.443717 10119 raft_consensus.cc:2749] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:28.444486   431 raft_consensus.cc:2272] T 472ed50e0865403c81123d4efe7108d7 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:28.418869  1459 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 275) took 83622 ms (client timeout 15000 ms). Trace:
W20260709 13:03:28.437603  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:28.453812  1459 rpcz_store.cc:269] 0709 13:02:04.912863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.938106 (+ 25243us) service_pool.cc:224] Handling call
0709 13:03:28.418628 (+83480522us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11602,"reported_tablets":48}
I20260709 13:03:28.468747   431 tablet_replica.cc:333] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.472097   431 raft_consensus.cc:2243] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.472898   431 raft_consensus.cc:2272] T 13debc393e8a4121883dda403cb4badf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.476394   431 tablet_replica.cc:333] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.480242   431 raft_consensus.cc:2243] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.481453   431 raft_consensus.cc:2272] T 0fb354b7d6104034b98fab8201f2719e P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.509857   431 tablet_replica.cc:333] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.536942  1450 catalog_manager.cc:5697] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:28.544495  1457 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40386 (request call id 274) took 56146 ms (client timeout 15000 ms). Trace:
W20260709 13:03:28.545605  1457 rpcz_store.cc:269] 0709 13:02:32.397887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.403723 (+  5836us) service_pool.cc:224] Handling call
0709 13:02:33.020625 (+616902us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b5400051d10. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b6640 00007fda2d48db76 00007fda2d597c3e 00007fda2ba582a4 00007fda2ba58232 00007fda2ba58161 00007fda2ba580dc 00007fda2ba57392 00007fda25a1548c 00007fda25a148c6
0709 13:03:28.544238 (+55523613us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1336762,"reported_tablets":49,"spinlock_wait_cycles":8455552}
I20260709 13:03:28.546365   431 raft_consensus.cc:2243] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.552330  1450 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:28.554288   431 pending_rounds.cc:70] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Trying to abort 2 pending ops.
I20260709 13:03:28.574404   431 pending_rounds.cc:77] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Aborting op as it isn't in flight: id { term: 19 index: 14 } timestamp: 7305634531186380800 op_type: NO_OP noop_request { }
I20260709 13:03:28.574923   431 raft_consensus.cc:2889] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:28.575269   431 pending_rounds.cc:77] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1: Aborting op as it isn't in flight: id { term: 17 index: 13 } timestamp: 7305634503190466560 op_type: NO_OP noop_request { }
I20260709 13:03:28.575680   431 raft_consensus.cc:2889] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:28.575066  1450 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:28.576025   431 raft_consensus.cc:2272] T c2ba61e8020a4800b9e7fdae5533800b P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:28.570993  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:28.569038  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:28.588032  1450 catalog_manager.cc:5697] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "d62ce354d09a4be59c29094425d09bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } }
I20260709 13:03:28.605602   431 tablet_replica.cc:333] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.627043  1710 consensus_queue.cc:237] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:28.648675   431 raft_consensus.cc:2243] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:28.650457   431 raft_consensus.cc:2272] T fb91933caa7241dc9c723195dd6950dc P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.673169   431 tablet_replica.cc:333] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.698302   431 raft_consensus.cc:2243] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.699236   431 raft_consensus.cc:2272] T 316c6fe30c8f416c942b4338f4e6eab9 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:28.705835  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:28.709686  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:28.729914   431 tablet_replica.cc:333] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.734751   431 raft_consensus.cc:2243] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:28.738862   431 raft_consensus.cc:2272] T 61e7345bb41047a2896de16e89540bbb P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.755234  2346 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:28.756407  1630 raft_consensus.cc:1275] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:28.763618   431 tablet_replica.cc:333] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.764873 10131 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:28.767427   431 raft_consensus.cc:2243] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.768571   431 raft_consensus.cc:2272] T 6b24efe048e6499b923f1a0215713fad P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.772535   431 tablet_replica.cc:333] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.772642  9633 consensus_queue.cc:1048] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:28.780282   431 raft_consensus.cc:2243] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.781603   431 raft_consensus.cc:2272] T 37bfe7ffa6604774b0b035da58f96eb9 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.801520 10121 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.802533 10121 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } attrs { replace: true } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:28.806988  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:28.827303 10121 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:28.828682  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:28.830252  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8785cfce0d0042d3855cf9a7edaf1f7d" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:28.830875  1527 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:28.834163 10120 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:28.835325   431 tablet_replica.cc:333] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.836861   431 raft_consensus.cc:2243] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:28.849262  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:28.851748 10129 raft_consensus.cc:2955] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 3 to 11, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:28.855355   431 raft_consensus.cc:2272] T 8516bd3d9b8948f0a0a32c2e93d4673e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.855190  1632 raft_consensus.cc:2955] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 3 to 11, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:28.894275   431 tablet_replica.cc:333] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.893913  2346 raft_consensus.cc:2955] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 3 to 11, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
W20260709 13:03:28.900471  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:28.906807  1445 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8785cfce0d0042d3855cf9a7edaf1f7d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:28.895479   431 raft_consensus.cc:2243] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.910825   431 raft_consensus.cc:2272] T 9ac4670b4c614e9eb392b74198615198 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.926133  1457 catalog_manager.cc:5697] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: config changed from index 3 to 11, VOTER d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) evicted. New cstate: current_term: 16 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } } }
I20260709 13:03:28.934577   431 tablet_replica.cc:333] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:28.936004   431 raft_consensus.cc:2243] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:28.937088   431 pending_rounds.cc:70] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Trying to abort 1 pending ops.
W20260709 13:03:28.945604  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:28.937340   431 pending_rounds.cc:77] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1: Aborting op as it isn't in flight: id { term: 21 index: 12 } timestamp: 7305634555629412352 op_type: NO_OP noop_request { }
I20260709 13:03:28.966284   431 raft_consensus.cc:2889] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:28.966850   431 raft_consensus.cc:2272] T 77e952b7f0f34618a5ce435fa48e0448 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:28.968618  1450 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 8785cfce0d0042d3855cf9a7edaf1f7d with cas_config_opid_index 3: aborting the task: latest config opid_index 11; task opid_index 3
I20260709 13:03:28.971081  1450 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40398 (request call id 278) took 1080 ms. Trace:
I20260709 13:03:28.973675  1450 rpcz_store.cc:276] 0709 13:03:27.891006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.891170 (+   164us) service_pool.cc:224] Handling call
0709 13:03:28.970972 (+1079802us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:28.669317 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:28.670225 (+   908us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:28.670281 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:03:28.670314 (+    33us) tablet.cc:662] Decoding operations
0709 13:03:28.671309 (+   995us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:28.671433 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:28.671479 (+    46us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:03:28.672076 (+   597us) tablet.cc:701] Row locks acquired
0709 13:03:28.672105 (+    29us) write_op.cc:260] PREPARE: finished
0709 13:03:28.672328 (+   223us) write_op.cc:270] Start()
0709 13:03:28.672505 (+   177us) write_op.cc:276] Timestamp: P: 1783602208672259 usec, L: 0
0709 13:03:28.672553 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 13:03:28.690481 (+ 17928us) log.cc:844] Serialized 1706 byte log entry
0709 13:03:28.751757 (+ 61276us) op_driver.cc:464] REPLICATION: finished
0709 13:03:28.812952 (+ 61195us) write_op.cc:301] APPLY: starting
0709 13:03:28.813059 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:28.815943 (+  2884us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:28.816072 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:28.845552 (+ 29480us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:28.845848 (+   296us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:03:28.845916 (+    68us) write_op.cc:312] APPLY: finished
0709 13:03:28.855776 (+  9860us) log.cc:844] Serialized 49 byte log entry
0709 13:03:28.856833 (+  1057us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:28.856980 (+   147us) write_op.cc:454] Released schema lock
0709 13:03:28.857062 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:28.857705 (+   643us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:28.857738 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":51453,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":60890,"apply.run_cpu_time_us":8234,"apply.run_wall_time_us":44726,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20847,"num_ops":4,"prepare.queue_time_us":74419,"prepare.run_cpu_time_us":4797,"prepare.run_wall_time_us":21580,"replication_time_us":79017,"thread_start_us":127857,"threads_started":2}]]}
I20260709 13:03:29.019004   431 tablet_replica.cc:333] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.032176 10120 raft_consensus.cc:493] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.032932 10120 raft_consensus.cc:515] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:29.035521 10120 leader_election.cc:290] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:29.035836   431 raft_consensus.cc:2243] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.036712   431 raft_consensus.cc:2272] T 3897029c4b5f443296641c39af287e98 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.039702  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:29.056178  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "946eea23a2334d678895281941dd9926" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:29.056238  9564 raft_consensus.cc:493] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.058100  9564 raft_consensus.cc:515] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } attrs { replace: true } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:29.060940  9564 leader_election.cc:290] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:29.062237  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:29.066951  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:29.067586  9564 raft_consensus.cc:493] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.074249  9564 raft_consensus.cc:515] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:29.071165   431 tablet_replica.cc:333] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.078608   431 raft_consensus.cc:2243] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:29.076678  9564 leader_election.cc:290] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:29.079988   431 raft_consensus.cc:2272] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.081660  1526 leader_election.cc:304] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8
I20260709 13:03:29.082823 10120 raft_consensus.cc:2749] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:29.087833  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:29.092461  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61a6c336f56b4c248aac90993fb477cf" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:29.092932  1445 catalog_manager.cc:4755] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8785cfce0d0042d3855cf9a7edaf1f7d: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:03:29.095819  1525 leader_election.cc:304] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:29.096858 10137 raft_consensus.cc:2749] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:29.097520  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18511a70f641409b83d4cde6d4e56009" candidate_uuid: "d62ce354d09a4be59c29094425d09bc1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:29.115674   431 tablet_replica.cc:333] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.117198   431 raft_consensus.cc:2243] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.118172   431 raft_consensus.cc:2272] T 22f9010f94804fcaacae8539ddfc64a7 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.120467  1525 leader_election.cc:304] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d62ce354d09a4be59c29094425d09bc1; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:29.130322 10120 raft_consensus.cc:2749] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:29.151706   431 tablet_replica.cc:333] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.152998   431 raft_consensus.cc:2243] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.153777   431 pending_rounds.cc:70] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Trying to abort 1 pending ops.
I20260709 13:03:29.154080   431 pending_rounds.cc:77] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1: Aborting op as it isn't in flight: id { term: 25 index: 19 } timestamp: 7305634536770588672 op_type: NO_OP noop_request { }
I20260709 13:03:29.154500   431 raft_consensus.cc:2889] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:29.154827   431 raft_consensus.cc:2272] T 946eea23a2334d678895281941dd9926 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.186977   431 tablet_replica.cc:333] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.188158   431 raft_consensus.cc:2243] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:03:29.189612  1527 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.198073   431 raft_consensus.cc:2272] T e6edcffcaa5a46d2ba7c824073943b82 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.211261   431 tablet_replica.cc:333] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.212388   431 raft_consensus.cc:2243] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:03:29.218729  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:29.238617   431 raft_consensus.cc:2272] T 0cb744c0184549918fb0bb586614bb7f P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.257617   431 tablet_replica.cc:333] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.258769   431 raft_consensus.cc:2243] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.259569   431 raft_consensus.cc:2272] T eb99ad78ebb2479fbf4715bb9d33258e P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.293648   431 tablet_replica.cc:333] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.294795   431 raft_consensus.cc:2243] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.296895   431 raft_consensus.cc:2272] T 26a92c141f564f638497aa02dd5c7c79 P d62ce354d09a4be59c29094425d09bc1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.308445   431 tablet_replica.cc:333] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.320779   431 raft_consensus.cc:2243] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:29.322206   431 raft_consensus.cc:2272] T 05b3e259298f4617a9f83f3e7cf738ac P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.338959   431 tablet_replica.cc:333] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.340219   431 raft_consensus.cc:2243] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:29.353135   431 raft_consensus.cc:2272] T c96b268f1f734ec793355ab247e28a75 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.359364   431 tablet_replica.cc:333] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.360267   431 raft_consensus.cc:2243] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.361089   431 raft_consensus.cc:2272] T 74f866a699e74063990bd566c7fb1ebf P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:29.364486  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:29.388059   431 tablet_replica.cc:333] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.392694   431 raft_consensus.cc:2243] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:29.394230   431 raft_consensus.cc:2272] T 9a353430d15945ee9524da3d2b667c90 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.415211   431 tablet_replica.cc:333] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.416229   431 raft_consensus.cc:2243] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.416980   431 raft_consensus.cc:2272] T 696fb3c290c54f14a67f61bf78588eda P d62ce354d09a4be59c29094425d09bc1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.428097   431 tablet_replica.cc:333] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.429118   431 raft_consensus.cc:2243] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:03:29.430418   431 raft_consensus.cc:2272] T 4e2094a808984c84823c1e0dad0a5de0 P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.439107  9653 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:29.439827  9653 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:29.445607  9653 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:29.446935  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:29.455871  1675 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:29.456456  1675 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:29.457545 10129 raft_consensus.cc:2749] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:29.464178  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:29.475907   431 tablet_replica.cc:333] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.477133   431 raft_consensus.cc:2243] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:03:29.478386   431 raft_consensus.cc:2272] T 563a5aa85e754fe48eaf5615d947ab9a P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.481737   431 tablet_replica.cc:333] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.498159   431 raft_consensus.cc:2243] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.498963   431 raft_consensus.cc:2272] T 18511a70f641409b83d4cde6d4e56009 P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.500880 10124 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:29.509410 10124 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:29.511778   431 tablet_replica.cc:333] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.512945   431 raft_consensus.cc:2243] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.513646   431 raft_consensus.cc:2272] T e6c211dcb5c947719413e384c6a72769 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.517107   431 tablet_replica.cc:333] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.518095   431 raft_consensus.cc:2243] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:29.518491  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:29.519481   431 raft_consensus.cc:2272] T 169e9f6ed9684060b36e93c778925461 P d62ce354d09a4be59c29094425d09bc1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.520511 10124 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:29.524528   431 tablet_replica.cc:333] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.525447   431 raft_consensus.cc:2243] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.526063   431 pending_rounds.cc:70] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Trying to abort 1 pending ops.
I20260709 13:03:29.526351   431 pending_rounds.cc:77] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1: Aborting op as it isn't in flight: id { term: 15 index: 12 } timestamp: 7305634527491182592 op_type: NO_OP noop_request { }
I20260709 13:03:29.526758   431 raft_consensus.cc:2889] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:29.527130   431 raft_consensus.cc:2272] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.531097   431 tablet_replica.cc:333] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.532667   431 raft_consensus.cc:2243] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.533463   431 raft_consensus.cc:2272] T 61a6c336f56b4c248aac90993fb477cf P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.538378   431 tablet_replica.cc:333] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.539287   431 raft_consensus.cc:2243] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.540210   431 pending_rounds.cc:70] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Trying to abort 1 pending ops.
I20260709 13:03:29.540460   431 pending_rounds.cc:77] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1: Aborting op as it isn't in flight: id { term: 22 index: 15 } timestamp: 7305634530946269184 op_type: NO_OP noop_request { }
I20260709 13:03:29.540879   431 raft_consensus.cc:2889] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:29.541261   431 raft_consensus.cc:2272] T e6c08bb7c6294cf9b33ad3b85e34831d P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.544806   431 tablet_replica.cc:333] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.545734   431 raft_consensus.cc:2243] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.546358   431 raft_consensus.cc:2272] T 69b9dbe96d28487fa6d8a381ac6fc383 P d62ce354d09a4be59c29094425d09bc1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.549877   431 tablet_replica.cc:333] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.550771   431 raft_consensus.cc:2243] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.551433   431 raft_consensus.cc:2272] T 10315b15aaee40e0a1110e5ce8cf0abf P d62ce354d09a4be59c29094425d09bc1 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:29.556510  1601 leader_election.cc:336] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:29.557153  1601 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:29.558609 10124 raft_consensus.cc:2749] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:29.556507   431 tablet_replica.cc:333] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.560971   431 raft_consensus.cc:2243] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.562001   431 raft_consensus.cc:2272] T 29b074786b944dd19df72b17c4fbb001 P d62ce354d09a4be59c29094425d09bc1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.610235   431 tablet_replica.cc:333] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.611303   431 raft_consensus.cc:2243] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.618693   431 raft_consensus.cc:2272] T 8785cfce0d0042d3855cf9a7edaf1f7d P d62ce354d09a4be59c29094425d09bc1 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.622336   431 tablet_replica.cc:333] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.623208   431 raft_consensus.cc:2243] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 26 LEADER]: Raft consensus shutting down.
W20260709 13:03:29.634546  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:29.650143   431 raft_consensus.cc:2272] T 939d7b92438a4b20bd5f2e81d2291c03 P d62ce354d09a4be59c29094425d09bc1 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:29.659693  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:29.678058   431 tablet_replica.cc:333] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.679209   431 raft_consensus.cc:2243] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.679883   431 pending_rounds.cc:70] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Trying to abort 1 pending ops.
I20260709 13:03:29.680148   431 pending_rounds.cc:77] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1: Aborting op as it isn't in flight: id { term: 21 index: 11 } timestamp: 7305634531603230720 op_type: NO_OP noop_request { }
I20260709 13:03:29.680580   431 raft_consensus.cc:2889] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:29.680965   431 raft_consensus.cc:2272] T f5aa381f739e4c71869e157bdba8aa97 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.698407   431 tablet_replica.cc:333] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.699363   431 raft_consensus.cc:2243] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.699971   431 pending_rounds.cc:70] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Trying to abort 1 pending ops.
I20260709 13:03:29.700222   431 pending_rounds.cc:77] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1: Aborting op as it isn't in flight: id { term: 17 index: 12 } timestamp: 7305634519503437824 op_type: NO_OP noop_request { }
I20260709 13:03:29.700606   431 raft_consensus.cc:2889] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:29.700973   431 raft_consensus.cc:2272] T 6b73223ab67f4b5aab700c65a70e42cf P d62ce354d09a4be59c29094425d09bc1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.704480   431 tablet_replica.cc:333] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.705418   431 raft_consensus.cc:2243] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:29.706888   431 raft_consensus.cc:2272] T 2f47909c83d54b699ec2b1c2402084b4 P d62ce354d09a4be59c29094425d09bc1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:29.710472   431 tablet_replica.cc:333] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1: stopping tablet replica
I20260709 13:03:29.711253   431 raft_consensus.cc:2243] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:29.711820   431 raft_consensus.cc:2272] T 0adeaf85877a4103bbd070927413d267 P d62ce354d09a4be59c29094425d09bc1 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:29.726742  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:29.778419  9633 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:29.779047  9633 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:29.787178  9633 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:29.793274  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:29.803920 10122 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:29.806649 10122 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:29.812135 10122 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:29.809935  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:29.830749  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:29.831561  1710 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
W20260709 13:03:29.833060  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:29.833881  1675 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:29.834554  1675 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:29.836896  9633 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:03:29.838575 10123 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:29.839254 10123 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:29.839781  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:29.842244 10123 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:29.844225  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:29.845034  1710 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 27.
W20260709 13:03:29.847661  2318 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:29.850895  1601 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:29.852028  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:29.852952 10123 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:03:29.853377 10123 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:29.853798 10123 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:03:29.860400  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:29.861505 10072 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:29.861927 10072 raft_consensus.cc:493] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:29.862290 10072 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:29.869364  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:29.887989  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:29.890408 10072 raft_consensus.cc:515] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:29.892849 10072 leader_election.cc:290] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:29.902709  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca209b0d98c64e1da48e3027b3ea1d9a" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:29.903513  1710 raft_consensus.cc:3060] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:29.905218 10123 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:29.907505  2318 leader_election.cc:336] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:29.911412  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43dce0aaab204cb084a73a74a80bd7a3" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:29.912228  1701 raft_consensus.cc:3060] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:03:29.919816  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:29.927659  9653 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:03:29.926931  1601 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:29.928265  9653 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:29.929984  1701 raft_consensus.cc:2468] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 28.
I20260709 13:03:29.933516  1601 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:29.934554 10124 raft_consensus.cc:2804] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:03:29.935132 10124 raft_consensus.cc:697] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 28 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:29.936161 10124 consensus_queue.cc:237] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:29.937718  1710 raft_consensus.cc:2468] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22.
I20260709 13:03:29.941053 10072 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:29.941660  2318 leader_election.cc:304] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:29.941736 10072 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:29.907529 10123 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 28 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:29.945803 10122 raft_consensus.cc:2804] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:29.946439 10122 raft_consensus.cc:697] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:29.947458 10122 consensus_queue.cc:237] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:29.951287  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:29.968804  9653 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:29.971869  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:29.972692  1635 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:29.953020 10072 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:29.975580  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:29.977509  9633 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:29.977926  9633 raft_consensus.cc:493] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:29.978276  9633 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:29.979847  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:29.985492  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:29.986732  2318 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:29.993831  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:29.995085 10140 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:29.986920  1675 leader_election.cc:336] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:30.005856  9633 raft_consensus.cc:515] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:30.010126  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472ed50e0865403c81123d4efe7108d7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:30.011118  1630 raft_consensus.cc:3060] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:30.014778  1455 catalog_manager.cc:5697] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 21 to 22, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 22 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:30.031304  1675 leader_election.cc:336] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:30.035986  1630 raft_consensus.cc:2468] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 24.
I20260709 13:03:30.039314  1674 leader_election.cc:304] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.041604 10141 raft_consensus.cc:2804] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:30.042448 10141 raft_consensus.cc:697] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:30.043589 10141 consensus_queue.cc:237] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.10, Last appended by leader: 10, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:30.045624  9633 leader_election.cc:290] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:30.045255  1450 catalog_manager.cc:5697] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 27 to 28, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 28 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:30.079280 10124 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.080036 10124 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.085044 10124 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:30.088447  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:30.089360  2346 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 20.
W20260709 13:03:30.090987  1601 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:30.093214  1602 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.094561 10124 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:30.095016 10124 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.095443 10124 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:30.096558  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:30.096695  1457 catalog_manager.cc:5697] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 23 to 24, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 24 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:30.114651 10124 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.117419 10122 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.118218 10122 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.122656  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:30.125526  2318 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:30.126389  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.127362 10140 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:30.127615 10122 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:30.134256  1601 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:30.140456  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb91933caa7241dc9c723195dd6950dc" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:30.141265  2346 raft_consensus.cc:3060] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:30.138842 10124 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:30.163012  2346 raft_consensus.cc:2468] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
I20260709 13:03:30.167251  1602 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.169524 10124 raft_consensus.cc:2804] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:30.170392 10124 raft_consensus.cc:697] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 21 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:30.172741 10124 consensus_queue.cc:237] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.225654 10124 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.226290 10124 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.237214  9633 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.238191  9633 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.246300  9633 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:30.256771  1601 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.258565 10124 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:30.258780  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:30.259498  2346 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
I20260709 13:03:30.260797  1602 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.262583 10124 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:30.263036 10124 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.263522 10124 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:30.270918 10123 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.271703 10123 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:30.275640 10123 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:30.275974 10122 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.278784  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:30.278842 10122 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.283006 10122 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:30.287823  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:30.290027  1601 leader_election.cc:336] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.290596  1601 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.292093 10123 raft_consensus.cc:2749] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:30.311122  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:30.311582  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:30.311935  2318 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:30.319670  1675 leader_election.cc:336] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.320259  1675 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.322389 10141 raft_consensus.cc:2749] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:30.327337  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.329205 10122 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:30.332625 10124 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.337780 10123 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.350541 10123 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.352874 10123 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:30.354982  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:03:30.357228  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:30.358796  1601 leader_election.cc:336] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.359382  1601 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.367300 10124 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:30.371589  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:30.360724 10123 raft_consensus.cc:2749] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:30.374411  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61e7345bb41047a2896de16e89540bbb" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:30.375276  2346 raft_consensus.cc:3060] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:30.389593  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:30.393070  2346 raft_consensus.cc:2468] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 23.
W20260709 13:03:30.396322  1445 catalog_manager.cc:4755] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8785cfce0d0042d3855cf9a7edaf1f7d: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.400810  1602 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:30.404281 10156 raft_consensus.cc:2804] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:03:30.404838  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:30.405140 10156 raft_consensus.cc:697] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 23 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
W20260709 13:03:30.405524  1601 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.406175 10156 consensus_queue.cc:237] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.412949  1457 catalog_manager.cc:5697] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 20 to 21, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 21 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:30.465538  1457 catalog_manager.cc:5697] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 22 to 23, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 23 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:30.468320  1630 raft_consensus.cc:1275] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 23 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 13:03:30.475428 10141 consensus_queue.cc:1048] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:03:30.477835  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:30.479219 10122 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.479866 10122 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:30.482093  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:30.482224 10122 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:30.487934  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:30.488698  1710 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 23.
W20260709 13:03:30.491597  2318 leader_election.cc:336] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.492991  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.494696 10140 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:30.495118 10140 raft_consensus.cc:493] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.495510 10140 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:30.507572  1710 raft_consensus.cc:1275] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:30.509552 10122 consensus_queue.cc:1048] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:30.510890  1710 raft_consensus.cc:1275] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:03:30.512753 10156 consensus_queue.cc:1048] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:03:30.534075  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:30.544442  9653 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.545047  9653 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:30.550026  9653 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:30.552611  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:30.553694 10122 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:03:30.554538  1675 leader_election.cc:336] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.554303 10122 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.557147 10122 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:30.558486 10072 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.559286 10129 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.505409 10140 raft_consensus.cc:515] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.559899 10129 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.562541 10129 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:30.559051 10072 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.565317 10072 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:30.567104  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:30.567912  1635 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
I20260709 13:03:30.568496  1674 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.571247  2317 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:30.578163 10141 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:30.591580  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:30.598063 10165 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:30.598685 10165 raft_consensus.cc:493] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.599094 10165 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:30.615278 10140 leader_election.cc:290] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:30.618310  1675 leader_election.cc:336] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.610298 10123 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.620388 10123 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:30.647250 10165 raft_consensus.cc:515] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.653753 10141 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.666517 10136 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.669888 10129 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.670535 10129 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:30.672854 10129 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:30.667285 10136 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:03:30.659593 10159 log.cc:927] Time spent T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:03:30.674513  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:30.674114  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:30.688890  1630 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:30.674062 10123 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:30.666862 10141 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:30.690717  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:30.694234  9633 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:30.694785  9633 raft_consensus.cc:493] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.695302  9633 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:30.711091  9633 raft_consensus.cc:515] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:30.731053  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a61af0ea7d47c1bc0ee6a94a2d1a7f" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:30.732292  1701 raft_consensus.cc:3060] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:30.749249  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:30.750010  1630 raft_consensus.cc:3060] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:30.758524  1630 raft_consensus.cc:2468] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 24.
I20260709 13:03:30.762328  2346 raft_consensus.cc:1275] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:30.763743  1704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb744c0184549918fb0bb586614bb7f" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:30.764997  1704 raft_consensus.cc:2393] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba0548b7d2a94e8598475a46c279fe0e in current term 24: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:30.746912  9633 leader_election.cc:290] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:30.777534  9633 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.778402  9633 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.798978  1701 raft_consensus.cc:2468] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 24.
I20260709 13:03:30.803707  9653 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.806981  1674 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:30.804411  9653 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:30.803017  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 106334us
I20260709 13:03:30.839780  9633 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:30.841012  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:30.821373 10169 consensus_queue.cc:1048] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.070s
I20260709 13:03:30.850163 10179 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.854382 10179 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:30.856046 10136 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:30.850193 10141 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:30.840394 10165 leader_election.cc:290] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:30.848058 10176 raft_consensus.cc:2804] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:30.851061  2318 leader_election.cc:304] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:03:30.878700  1601 leader_election.cc:336] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.879230  1601 leader_election.cc:304] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba0548b7d2a94e8598475a46c279fe0e; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:30.883775  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:30.886329 10170 raft_consensus.cc:2749] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:30.886370 10166 raft_consensus.cc:2804] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:30.887653 10166 raft_consensus.cc:697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:30.888557 10166 consensus_queue.cc:237] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.889158 10173 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.889740 10173 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.895875 10179 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:30.915701  2318 leader_election.cc:336] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.916432  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.917696 10165 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:03:30.917713  2318 leader_election.cc:336] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:30.922453  1676 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:30.931939 10180 raft_consensus.cc:2749] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:30.933327  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:30.934020  2348 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 25.
I20260709 13:03:30.934692 10165 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.935209 10165 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:30.937112 10165 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:30.941900  2318 leader_election.cc:336] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:30.942736  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:30.950724  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:30.951354  1710 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 21.
W20260709 13:03:30.954314  2318 leader_election.cc:336] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:30.954984  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:30.955395  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:30.957350  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:30.958189 10180 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:30.958581 10180 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:30.958941 10180 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:30.962343  9653 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:30.964867 10173 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:30.966171  1630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:30.966796  1630 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
I20260709 13:03:30.973949  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2094a808984c84823c1e0dad0a5de0" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:30.974515  1632 raft_consensus.cc:3060] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:30.986770  1632 raft_consensus.cc:2468] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:03:30.990315  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:30.991056  1710 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 18.
W20260709 13:03:31.002291  1601 leader_election.cc:336] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.002756  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:31.006148  9633 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:31.006511 10165 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:31.006688  9633 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:31.007095 10165 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:31.009158  9633 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:31.009935 10165 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:31.013289 10164 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:31.015010 10164 raft_consensus.cc:493] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:31.015450 10164 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:31.023638  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:31.024498  2351 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:31.030010  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:31.030652  2351 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 22.
W20260709 13:03:31.033782  2318 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.040179  2351 raft_consensus.cc:1275] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:03:31.040649  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:31.044692  1632 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 18.
W20260709 13:03:31.051263  2318 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.051303 10176 raft_consensus.cc:697] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:31.064448 10176 consensus_queue.cc:237] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:31.027351 10164 raft_consensus.cc:515] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:31.068109  1457 catalog_manager.cc:5697] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 23 to 24, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 24 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:31.069883 10164 leader_election.cc:290] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:03:31.054205  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44437.3us
I20260709 13:03:31.072978  1674 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:31.073199  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:31.074134 10177 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:31.074349 10183 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:31.074844 10183 raft_consensus.cc:493] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:31.075246 10183 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:31.074635 10177 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:31.075917 10177 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:31.082739  1628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6edcffcaa5a46d2ba7c824073943b82" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:31.083454  1628 raft_consensus.cc:3060] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:31.084802  1675 leader_election.cc:336] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:31.085832  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:31.086284  1675 leader_election.cc:336] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.090355 10183 raft_consensus.cc:515] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:31.092985  1628 raft_consensus.cc:2468] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:31.056300  2317 leader_election.cc:304] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:31.058398  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:31.097167  1674 leader_election.cc:304] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:31.098202 10187 raft_consensus.cc:2804] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:31.098351 10165 raft_consensus.cc:2804] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:31.098775 10187 raft_consensus.cc:697] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:31.099123 10165 raft_consensus.cc:697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:31.099733 10187 consensus_queue.cc:237] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:31.060066  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:31.093214  1675 leader_election.cc:336] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.101866  1710 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22.
I20260709 13:03:31.102052 10178 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
W20260709 13:03:31.102288  1675 leader_election.cc:336] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.102651 10178 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:31.092941 10183 leader_election.cc:290] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:31.104872 10178 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:31.094120  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563a5aa85e754fe48eaf5615d947ab9a" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e"
I20260709 13:03:31.105695  1632 raft_consensus.cc:3060] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:31.100513 10165 consensus_queue.cc:237] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:31.117385  2318 leader_election.cc:336] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.118433 10180 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:31.119611  1632 raft_consensus.cc:2468] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 19.
I20260709 13:03:31.121097  1674 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:31.122136 10187 raft_consensus.cc:2804] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:31.122887 10187 raft_consensus.cc:697] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
W20260709 13:03:31.126133  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 52990.9us
I20260709 13:03:31.054751  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:03:31.127553  1675 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:31.129899  1675 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.127836 10187 consensus_queue.cc:237] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:31.146857 10183 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:31.147397 10183 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:31.147809 10183 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:31.155858 10183 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:31.158406  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 66765.6us
I20260709 13:03:31.159583 10179 consensus_queue.cc:1048] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:03:31.158314  1457 catalog_manager.cc:5697] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 18 to 19, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 19 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:31.163291  1450 catalog_manager.cc:5697] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 23 to 24, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 24 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:31.146715  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 92004.5us
I20260709 13:03:31.165575  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:31.165838  1601 leader_election.cc:304] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:31.166594 10164 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:31.166771  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:31.167061 10164 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:31.167582 10164 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:31.167619 10165 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:31.168077  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:31.121156 10180 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:31.169121 10165 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:31.169545 10165 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:31.169658 10179 raft_consensus.cc:2764] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:03:31.170027 10165 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:31.176055 10183 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:31.178725  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:03:31.179286 10183 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:31.179430  1705 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 26.
I20260709 13:03:31.179841 10183 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:31.181944 10183 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:31.197008  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a353430d15945ee9524da3d2b667c90" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:31.197752  2351 raft_consensus.cc:3060] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:03:31.200299  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:31.207111 10164 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:31.211602 10164 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:31.214655  1601 leader_election.cc:336] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.221649 10177 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:31.223805 10177 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:31.227002  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b3e259298f4617a9f83f3e7cf738ac" candidate_uuid: "ba0548b7d2a94e8598475a46c279fe0e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:31.227674  1707 raft_consensus.cc:3060] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:31.237671  1707 raft_consensus.cc:2468] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba0548b7d2a94e8598475a46c279fe0e in term 22.
W20260709 13:03:31.238052  1675 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:31.238850  1675 leader_election.cc:336] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:31.239658  1675 leader_election.cc:336] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:31.240276  1675 leader_election.cc:336] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:31.240885  1675 leader_election.cc:336] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:31.244935   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 1582)
I20260709 13:03:31.263245  2351 raft_consensus.cc:2468] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 26.
I20260709 13:03:31.262400  1601 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e; no voters: 
I20260709 13:03:31.268708  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c96b268f1f734ec793355ab247e28a75" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:31.268975  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:31.269320  2351 raft_consensus.cc:3060] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:31.271971  1675 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:31.272526  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:31.276224  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:31.276813  2349 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 25.
I20260709 13:03:31.272733  1450 catalog_manager.cc:5697] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 22 to 23, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 23 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:31.273772  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:31.280325  1450 catalog_manager.cc:5697] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 18 to 19, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:31.282761  2351 raft_consensus.cc:2468] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 24.
W20260709 13:03:31.284422  1675 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.278823 10170 raft_consensus.cc:2804] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:31.287927 10170 raft_consensus.cc:697] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 22 LEADER]: Becoming Leader. State: Replica: ba0548b7d2a94e8598475a46c279fe0e, State: Running, Role: LEADER
I20260709 13:03:31.289173 10170 consensus_queue.cc:237] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:31.280426 10165 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:31.298949 10178 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:03:31.299409 10178 raft_consensus.cc:493] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:31.300714  2346 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2ba7fc60f9641cdac25f1e016ef91d8 in current term 23: Already voted for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in this term.
I20260709 13:03:31.299794 10178 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:03:31.302369  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:31.306034 10165 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:03:31.309999  1601 leader_election.cc:336] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.343801  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:31.344709  1707 raft_consensus.cc:2393] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in current term 23: Already voted for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in this term.
I20260709 13:03:31.345806  1676 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:31.346573 10180 raft_consensus.cc:2804] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:03:31.347043 10180 raft_consensus.cc:697] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:31.347922 10180 consensus_queue.cc:237] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:31.350469  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:31.351423 10183 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:31.351795 10183 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Starting leader election (detected failure of leader d62ce354d09a4be59c29094425d09bc1)
I20260709 13:03:31.352212 10183 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:31.355094 10178 raft_consensus.cc:515] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:31.371317  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "939d7b92438a4b20bd5f2e81d2291c03" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:03:31.372016  1705 raft_consensus.cc:3060] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:03:31.357236 10178 leader_election.cc:290] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:31.381012  1450 catalog_manager.cc:5697] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e reported cstate change: term changed from 21 to 22, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194). New cstate: current_term: 22 leader_uuid: "ba0548b7d2a94e8598475a46c279fe0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:31.384851  1457 catalog_manager.cc:5697] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 25 to 26, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 26 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:31.393994 10183 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:31.374966  2318 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.398763  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:31.400385  1675 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:31.402038  1705 raft_consensus.cc:2468] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 27.
I20260709 13:03:31.404896 10171 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:31.407920 10183 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:31.351799  1676 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
W20260709 13:03:31.409426  1676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 57677.1us
I20260709 13:03:31.412309  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:31.410211 10183 raft_consensus.cc:2804] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:31.413233 10180 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:31.414043 10183 raft_consensus.cc:697] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:31.414897 10183 consensus_queue.cc:237] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:31.425204  2318 leader_election.cc:304] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
W20260709 13:03:31.428248  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:31.430768  1705 raft_consensus.cc:1275] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:03:31.431646 10171 raft_consensus.cc:2804] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:03:31.432183 10171 raft_consensus.cc:697] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 27 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:03:31.433036 10171 consensus_queue.cc:237] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:31.446326 10166 consensus_queue.cc:1048] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:03:31.448964  2318 leader_election.cc:336] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:31.449769  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:31.452346  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f47909c83d54b699ec2b1c2402084b4" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:31.452931  2349 raft_consensus.cc:3060] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:31.462330  1628 raft_consensus.cc:1275] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:03:31.464617 10187 consensus_queue.cc:1048] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:31.469075  2349 raft_consensus.cc:2468] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 26.
I20260709 13:03:31.485845  1455 catalog_manager.cc:5697] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 26 to 27, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 27 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:31.499850  1676 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:31.500931 10180 raft_consensus.cc:2804] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:03:31.501619 10180 raft_consensus.cc:697] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:31.502496 10180 consensus_queue.cc:237] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:31.503196  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:31.505949  1632 raft_consensus.cc:1275] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:31.507866 10166 consensus_queue.cc:1048] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:31.512579  1635 raft_consensus.cc:1275] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:03:31.514930  9633 consensus_queue.cc:1048] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:03:31.519840  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:31.551707  1455 catalog_manager.cc:5697] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 23 to 24, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 24 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:31.578915  1445 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 90277.4us
W20260709 13:03:31.620617  1445 catalog_manager.cc:4755] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8785cfce0d0042d3855cf9a7edaf1f7d: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:31.624367  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:31.625206  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:31.641439  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:31.658720  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:31.659353  1630 raft_consensus.cc:1275] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:31.665694 10164 consensus_queue.cc:1048] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:03:31.665145  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:31.668957  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:31.687008  1457 catalog_manager.cc:5697] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 25 to 26, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 26 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:31.693608  1707 raft_consensus.cc:1275] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Refusing update from remote peer ba0548b7d2a94e8598475a46c279fe0e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:03:31.695732 10123 consensus_queue.cc:1048] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:31.734390  2349 raft_consensus.cc:1275] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:03:31.737511 10164 consensus_queue.cc:1048] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:03:31.825062  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:31.849687  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:31.853976  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:31.973582  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:31.982834  1705 raft_consensus.cc:1275] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
W20260709 13:03:31.985846  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:31.988476  2349 raft_consensus.cc:1275] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:03:31.993690 10209 consensus_queue.cc:1048] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 13:03:31.993863 10187 consensus_queue.cc:1048] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:03:31.999540  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.100250  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:32.120965  2349 raft_consensus.cc:1275] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:03:32.123908 10164 consensus_queue.cc:1048] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:03:32.245397   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:32.305418  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.313898  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.355980  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:32.356979  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.362931  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.408193  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.441545  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.451972  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.546574  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.555785  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.558058  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.605099  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:32.816967  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:32.918963  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:32.919821  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.950762  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:33.012077  1601 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111) [suppressed 421 similar messages]
W20260709 13:03:33.023810  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:33.121815  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:33.124841  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:33.205612  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:33.211943  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:33.215410  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:33.246165   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:33.251288  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:33.277379  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:33.365094  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:03:33.367653 10164 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.368331 10164 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:33.371260 10164 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:33.374622  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:33.375391  2346 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 23.
I20260709 13:03:33.377290  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:33.378953 10164 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:33.379429 10164 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.379858 10164 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:33.397696  1675 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:33.453637 10164 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:33.464386  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169e9f6ed9684060b36e93c778925461" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb"
I20260709 13:03:33.465140  2346 raft_consensus.cc:3060] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:33.474053 10164 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Requested vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:03:33.474557  2346 raft_consensus.cc:2468] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ba7fc60f9641cdac25f1e016ef91d8 in term 24.
I20260709 13:03:33.479166  1676 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:03:33.480590 10183 raft_consensus.cc:2804] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:33.481195 10183 raft_consensus.cc:697] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Becoming Leader. State: Replica: a2ba7fc60f9641cdac25f1e016ef91d8, State: Running, Role: LEADER
I20260709 13:03:33.482275 10183 consensus_queue.cc:237] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:03:33.501986  1675 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:33.514180  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:33.549464  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:33.557927  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43780.5us
I20260709 13:03:33.620884  1459 catalog_manager.cc:5697] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 reported cstate change: term changed from 22 to 24, leader changed from d62ce354d09a4be59c29094425d09bc1 (127.0.107.193) to a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195). New cstate: current_term: 24 leader_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:33.785527  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:33.868076  1445 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 54594.9us
W20260709 13:03:33.888335  1445 catalog_manager.cc:4755] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8785cfce0d0042d3855cf9a7edaf1f7d: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:33.924983  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:33.969417  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:34.034566  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:34.080488  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:34.091914  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:34.130155  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:34.150053  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:03:34.155375  2348 raft_consensus.cc:1275] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Refusing update from remote peer a2ba7fc60f9641cdac25f1e016ef91d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:03:34.156116  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:34.156718  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:34.165263  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:34.170878 10231 consensus_queue.cc:1048] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:03:34.241175  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:34.247277   431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:34.308557  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:34.362531  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:34.377454  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:34.470356  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:34.501724  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:34.502664  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:34.533864  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:34.741542  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:34.753732  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:34.776553  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:34.803505  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:34.865532  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:34.876204  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:34.909929  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:34.959993  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:35.074391  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:35.108174  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:35.117578  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:35.225263  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:35.237819  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:35.247814   431 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:35.274529  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:35.304032  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.480850  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:35.546044  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:35.572379  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:35.600301  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:35.777738  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:35.794219  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.831617  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:35.963020  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.966266  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:35.974479  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.994481  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.996557  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:36.163643  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:36.248462   431 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:36.355543  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:36.406229  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:36.444492  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 57766.5us
W20260709 13:03:36.445770  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:36.497346  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:36.505697  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:36.520967  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:36.557775  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:36.629050  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:36.679230  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:36.774564  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:36.778162  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:36.790584  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:37.090893  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:37.111285  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:37.112461  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:37.159049  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 107270us
W20260709 13:03:37.179912  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:37.210762  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.249341   431 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:37.348534  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:37.358397  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.394012  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.416906  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:37.452994  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.453742  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.473723  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.512164  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:37.528337  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:37.546803  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.551265  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.590039  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.620271  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.775187  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.874337  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:37.929881  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:37.958437  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:37.987273  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:38.012497  1675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111) [suppressed 457 similar messages]
W20260709 13:03:38.039309  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:38.066993  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:38.071832  1445 catalog_manager.cc:4755] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8785cfce0d0042d3855cf9a7edaf1f7d: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:38.093961  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:38.127584  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:38.140178  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 73629us
W20260709 13:03:38.249969   431 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:38.267419  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:38.410305  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:38.427059  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:38.474850  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:38.500953  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:38.562963  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:38.610679  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:38.633695  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:38.654964  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:38.675968  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:38.880733  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:38.896503  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:38.936360  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:39.024887  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:39.127832  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:39.195191  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:39.237385  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:39.250527   431 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:39.375748  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:39.403337  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:39.448256  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:39.468770  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:39.539108  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:39.564970  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:39.607422  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:39.623454  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:39.642103  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:39.693715  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:39.825892  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:39.885035  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:39.885710  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:39.959827  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:39.975662  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:39.991792  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:40.016820  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:40.061800  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:40.110836  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:40.122707  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:40.154402  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:40.251143   431 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:40.252871  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:40.306735  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:40.356765  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:40.386341  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:40.418848  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:40.472169  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:40.487877  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:40.568182  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:40.760218  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:40.787056  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:40.825289  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:41.012562  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:41.039808  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:41.068645  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:41.091926  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:41.156953  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:41.203840  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:41.251732   431 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:41.256863  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:41.373327  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:41.373960  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:41.439366  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:41.474531  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:41.479926  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:41.488003  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:41.529217  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:41.585561  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:41.694552  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:41.835433  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:41.863307  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:42.013024  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:42.050863  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:42.122179  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:42.134778  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.197130  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:42.243805  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:42.252226   431 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:42.362329  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.394629  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:42.403585  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.411998  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.412483  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.465416  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.518016  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52628.8us
W20260709 13:03:42.604642  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:42.637534  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:42.672086  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.698457  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.736992  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.774942  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:42.775522  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.776069  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.793123  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:42.821588  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.859241  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:42.912109  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 53435.3us
W20260709 13:03:43.004352  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:43.017750  1675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111) [suppressed 470 similar messages]
W20260709 13:03:43.084621  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:43.154211  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:43.157081  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:43.191088  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:43.252789   431 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:43.319890  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:43.508381  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:43.579866  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:43.815007  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:43.874787  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 44429.5us
W20260709 13:03:43.945118  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 55453.5us
W20260709 13:03:43.955855  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:44.010875  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:44.070734  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:44.095463  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:44.175412  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 79975us
W20260709 13:03:44.176579  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:44.177749  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:44.238062  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:44.253366   431 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:44.265808  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:44.302778  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:44.336856  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:44.338591  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:44.339165  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:44.339756  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:44.344529  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:44.384272  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:44.468626  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 43898.9us
W20260709 13:03:44.541917  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:44.567293  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 75106.9us
W20260709 13:03:44.626710  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:44.670003  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:44.685171  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:44.853089  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:44.895692  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:44.936233  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:44.950353  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.034019  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.035914  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.054790  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:45.058117  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.117285  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.129065  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.197113  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.207348  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:45.246452  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.254053   431 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:45.291688  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:45.326172  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:45.380182  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:45.394094  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.394719  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.464424  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:45.614851  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.646056  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:45.657814  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.686911  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:45.851373  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:45.939297  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:46.023473  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:46.099116  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:46.104154  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:46.255312   431 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:46.294420  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:46.330085  1445 catalog_manager.cc:4755] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8785cfce0d0042d3855cf9a7edaf1f7d: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:03:46.351436  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:46.449993  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:46.534283  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:46.536440  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 71460.4us
W20260709 13:03:46.598533  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:46.614377  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:46.646944  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:46.648733  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:46.667052  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:46.675604  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:46.681980  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:46.682451  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:46.706871  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:46.820935  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:47.015869  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:47.042691  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.073841  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:47.131280  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 49196.3us
W20260709 13:03:47.133160  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.194109  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:47.222805  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:47.256111   431 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:47.264400  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:47.349643  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 62358.4us
W20260709 13:03:47.453636  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.454880  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.458932  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.506090  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 63758.4us
W20260709 13:03:47.519937  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.612046  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.625245  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:47.644289  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:47.645020  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:47.700870  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.738356  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.760893  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.761838  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.792263  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:47.801231  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.820600  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.911896  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:47.966811  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:47.997406  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:48.026647  1675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111) [suppressed 464 similar messages]
W20260709 13:03:48.076589  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:48.196221  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:48.256740   431 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:48.302011  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:48.336977  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:48.432528  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:48.463110  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:48.582758  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:48.775075  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:48.870996  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:48.989598  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 61826.7us
W20260709 13:03:49.052624  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 44786.4us
W20260709 13:03:49.058137  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:49.068028  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:49.068812  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:49.085664  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:49.197044  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:49.249361  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:49.250473  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:49.256336  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:49.257325   431 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:49.267150  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:49.278824  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:49.279497  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:49.282411  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:49.376777  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:49.394256  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:49.530755  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:49.548008  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:49.572710  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:49.573454  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:49.739329  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:49.778024  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:49.794387  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:49.886745  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:49.887814  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45431, user_credentials={real_user=slave}} blocked reactor thread for 62319us
W20260709 13:03:49.910756  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:49.947067  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:49.974509  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:49.997212  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:50.026861  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:50.034720  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:50.103708  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:50.161644  1657 debug-util.cc:398] Leaking SignalData structure 0x7b0800166b00 after lost signal to thread 434
W20260709 13:03:50.163298  1657 debug-util.cc:398] Leaking SignalData structure 0x7b08003c2e00 after lost signal to thread 1493
W20260709 13:03:50.181670  1657 debug-util.cc:398] Leaking SignalData structure 0x7b08001784e0 after lost signal to thread 1660
W20260709 13:03:50.185976  1657 debug-util.cc:398] Leaking SignalData structure 0x7b080005b840 after lost signal to thread 1734
W20260709 13:03:50.191540  1657 debug-util.cc:398] Leaking SignalData structure 0x7b080021dc20 after lost signal to thread 2377
W20260709 13:03:50.259287   431 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:50.306278  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:50.356885  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:50.400434  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:50.412371  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:50.414943  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:50.414346  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:50.417773  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:50.436826  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:50.483604  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:50.573401  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:50.729246  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:50.736294  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:50.857700  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:50.979362  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:51.024101  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:51.054605  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:51.215891  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:51.259908   431 thread.cc:527] Waited for 21000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:51.333868  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:51.334745  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:51.567087  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:51.605273  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:51.623469  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:51.686738  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:51.731616  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:51.753172  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:51.824582  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:51.827309  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:51.913265  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:51.943360  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:51.945830  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:51.996269  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 52968.8us
W20260709 13:03:52.019763  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:52.057689  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:52.059242  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:52.081650  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:52.121920  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:52.249562  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:52.257088  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:52.261906   431 thread.cc:527] Waited for 22000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:52.354449  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:52.355101  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:52.383116  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:52.425772  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:52.477052  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:52.496181  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:52.524701  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:52.604746  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:52.668062  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:52.747331  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:52.757709  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:52.786480  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:52.810926  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:52.887501  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:52.938227  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:52.955142  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:53.029289  1675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111) [suppressed 461 similar messages]
W20260709 13:03:53.084208  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:53.214331  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:53.219087  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:53.222564  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:53.236707  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:53.262979  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:53.264701   431 thread.cc:527] Waited for 23000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:53.315057  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:54.022575  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 121593us
W20260709 13:03:54.168665  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:54.234505  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:54.266757   431 thread.cc:527] Waited for 24000ms trying to join with diag-logger (tid 1582)
I20260709 13:03:54.267495 10306 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:54.268666 10306 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:03:54.273782 10306 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:03:54.278740  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b074786b944dd19df72b17c4fbb001" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:54.281689  1675 leader_election.cc:336] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:03:54.282517  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:03:54.283883 10306 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:54.303988  1602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:36257, user_credentials={real_user=slave}} blocked reactor thread for 129992us
I20260709 13:03:54.309074 10322 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:03:54.311755 10322 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:54.317271 10322 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:03:54.332516  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
W20260709 13:03:54.339560  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:54.356580  1490 debug-util.cc:398] Leaking SignalData structure 0x7b0800174600 after lost signal to thread 434
W20260709 13:03:54.374765  1490 debug-util.cc:398] Leaking SignalData structure 0x7b0800372580 after lost signal to thread 1493
W20260709 13:03:54.376366  1490 debug-util.cc:398] Leaking SignalData structure 0x7b08001a9120 after lost signal to thread 1660
W20260709 13:03:54.377647  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 68879.9us
W20260709 13:03:54.377899  1490 debug-util.cc:398] Leaking SignalData structure 0x7b0800415440 after lost signal to thread 1734
W20260709 13:03:54.379706  1490 debug-util.cc:398] Leaking SignalData structure 0x7b0800219080 after lost signal to thread 2377
I20260709 13:03:54.348980 10306 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:03:54.386335 10306 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:03:54.397774  1632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba0548b7d2a94e8598475a46c279fe0e" is_pre_election: true
I20260709 13:03:54.407356  1705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1457f31daeb646a9824ac8068949eddd" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
W20260709 13:03:54.412449  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:03:54.415387 10306 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
W20260709 13:03:54.426790  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:54.427382  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:54.433195  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:03:54.435319  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:54.436456 10330 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:54.437439  2350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:03:54.453298  1676 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:03:54.456228 10310 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:54.554104  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:54.644740  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:54.760459  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:54.761696  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:54.773321  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:54.794356  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:54.797818  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:54.842129  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:54.906811  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:54.953683  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:54.985918  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:55.001061  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:55.121311  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:55.159896  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:55.239730  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:55.267294   431 thread.cc:527] Waited for 25000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:55.446620  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:55.450393  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:55.516916  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:55.519284  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:55.572575  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:55.578120  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:55.616737  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:55.641932  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:55.669817  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:55.735333  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:55.773548  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:55.786654  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:55.813915  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:55.843051  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:55.866948  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:55.906349  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:56.210820  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:56.212092  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:56.267905   431 thread.cc:527] Waited for 26000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:56.302588  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:56.309427  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:56.336511  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:56.342314  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:56.349087  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:56.359220  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:56.370873  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:56.672869  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:56.719421  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:56.793859  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:56.858176  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:57.051131  2374 debug-util.cc:398] Leaking SignalData structure 0x7b08003974e0 after lost signal to thread 434
W20260709 13:03:57.052523  2374 debug-util.cc:398] Leaking SignalData structure 0x7b08003977e0 after lost signal to thread 1493
W20260709 13:03:57.054068  2374 debug-util.cc:398] Leaking SignalData structure 0x7b080014ae20 after lost signal to thread 1660
W20260709 13:03:57.055401  2374 debug-util.cc:398] Leaking SignalData structure 0x7b08002ba4a0 after lost signal to thread 1734
W20260709 13:03:57.056766  2374 debug-util.cc:398] Leaking SignalData structure 0x7b0800216760 after lost signal to thread 2377
W20260709 13:03:57.099577  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:57.103178  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:57.120805  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:57.233342  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:57.252100  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:57.319850   431 thread.cc:527] Waited for 27000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:57.362504  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 71680.8us
W20260709 13:03:57.440162  1601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 48452.4us
W20260709 13:03:57.483815  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:57.511504  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:57.550843  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:57.610235  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:57.619297  2317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 88110.7us
W20260709 13:03:57.621924  1674 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:37097, user_credentials={real_user=slave}} blocked reactor thread for 84234.7us
W20260709 13:03:57.623854  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:57.643740  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:57.682896  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:57.691617  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:57.692193  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:57.691640  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:57.788220  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:57.816715  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:57.913528  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:57.987538  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.039740  1601 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111) [suppressed 402 similar messages]
W20260709 13:03:58.078919  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:58.177958  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.192257  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.212929  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:58.219213  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.267912  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:58.316181  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.321295   431 thread.cc:527] Waited for 28000ms trying to join with diag-logger (tid 1582)
W20260709 13:03:58.332484  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:58.336196  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:58.345656  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.394825  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.455441  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.519881  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.521842  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:58.540315  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:03:58.556339  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:58.642185  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.670354  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.794457  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:58.837607  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:58.856510  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:58.872965  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:59.081358  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:59.135524  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:59.159924  1601 consensus_peers.cc:597] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:59.301271  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:03:59.317596  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:03:59.318104  1601 consensus_peers.cc:597] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:59.857288  1601 consensus_peers.cc:597] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:00.126204  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 48280.3us
W20260709 13:04:00.127504  1601 consensus_peers.cc:597] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:04:00.135192  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:00.135926  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:00.148491  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:00.149209  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:00.149739  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:00.157068  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:00.162588  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:00.163551  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:00.325838  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:00.330272  1601 consensus_peers.cc:597] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:04:00.375262  1601 consensus_peers.cc:597] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:00.421420  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:00.456367  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:00.464027  1657 debug-util.cc:398] Leaking SignalData structure 0x7b08000d1760 after lost signal to thread 434
W20260709 13:04:00.465328  1657 debug-util.cc:398] Leaking SignalData structure 0x7b080013e320 after lost signal to thread 1493
W20260709 13:04:00.466796  1657 debug-util.cc:398] Leaking SignalData structure 0x7b08003693a0 after lost signal to thread 1660
W20260709 13:04:00.468024  1657 debug-util.cc:398] Leaking SignalData structure 0x7b08003b15e0 after lost signal to thread 1734
W20260709 13:04:00.469242  1657 debug-util.cc:398] Leaking SignalData structure 0x7b08001c0100 after lost signal to thread 2377
W20260709 13:04:00.458418  1601 consensus_peers.cc:597] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:04:00.554852  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:00.568161  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:00.706557  1601 consensus_peers.cc:597] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:04:00.765416   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:04:00.782372  1601 consensus_peers.cc:597] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:00.855507  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:00.883908  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:00.884913  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:00.910607  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:00.914505  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:00.920943  1601 consensus_peers.cc:597] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:04:00.931216  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:00.973047  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:01.071302  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:01.158655  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:01.187515  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:04:01.227245   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:04:01.239035  1674 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.253867  2317 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.294039  1674 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.301513  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:01.303041  2317 consensus_peers.cc:597] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.313884  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:01.317167  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:04:01.325582   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:01.327899   431 tablet_replica.cc:333] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.329348   431 raft_consensus.cc:2243] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.331297   431 raft_consensus.cc:2272] T 25feb0fae8094dd29fea235dd1af7a14 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.334852   431 tablet_replica.cc:333] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.336074   431 raft_consensus.cc:2243] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.336977   431 raft_consensus.cc:2272] T ac931d9355284a168be64d7debd009ae P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.340330   431 tablet_replica.cc:333] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.341600   431 raft_consensus.cc:2243] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.342734   431 raft_consensus.cc:2272] T 8cb801d86c874e1686eceacb9de1d0a6 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.345863   431 tablet_replica.cc:333] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
W20260709 13:04:01.352552  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:04:01.357293  2317 consensus_peers.cc:597] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.357028  1674 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.362676  1674 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.365350   431 raft_consensus.cc:2243] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.366833   431 raft_consensus.cc:2272] T d72a925fb22046e2be514a7913c613a4 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.370019   431 tablet_replica.cc:333] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.371167   431 raft_consensus.cc:2243] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.372119   431 raft_consensus.cc:2272] T 4e2094a808984c84823c1e0dad0a5de0 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.383119  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:01.385267  1674 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.389979   431 tablet_replica.cc:333] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.391008   431 raft_consensus.cc:2243] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.392632   431 raft_consensus.cc:2272] T 43dce0aaab204cb084a73a74a80bd7a3 P ba0548b7d2a94e8598475a46c279fe0e [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.392618  1601 consensus_peers.cc:597] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:04:01.395790   431 tablet_replica.cc:333] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.396999   431 raft_consensus.cc:2243] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.398149   431 raft_consensus.cc:2272] T 637e660a66be4924866287fa6150aaea P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.400983   431 tablet_replica.cc:333] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.401858   431 raft_consensus.cc:2243] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.403749   431 raft_consensus.cc:2272] T 13debc393e8a4121883dda403cb4badf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.405092  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:01.404249  2317 consensus_peers.cc:597] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.408614   431 tablet_replica.cc:333] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
W20260709 13:04:01.415146  1674 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.409560   431 raft_consensus.cc:2243] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.416950   431 raft_consensus.cc:2272] T 472ed50e0865403c81123d4efe7108d7 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.419973   431 tablet_replica.cc:333] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.420848   431 raft_consensus.cc:2243] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.422061   431 raft_consensus.cc:2272] T 61e7345bb41047a2896de16e89540bbb P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.430891  1674 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.439971  1674 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.440146  2317 consensus_peers.cc:597] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.440644  1674 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.454896  1674 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.458357  1674 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.441859   431 tablet_replica.cc:333] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.459939   431 raft_consensus.cc:2243] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.460985   431 raft_consensus.cc:2272] T 6b24efe048e6499b923f1a0215713fad P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.464001  2317 consensus_peers.cc:597] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.467775  1674 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.474135  1674 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.475325   431 tablet_replica.cc:333] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
W20260709 13:04:01.475878  2317 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.480363  1674 consensus_peers.cc:597] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.481945   431 raft_consensus.cc:2243] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.485702   431 raft_consensus.cc:2272] T c2ba61e8020a4800b9e7fdae5533800b P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.489279   431 tablet_replica.cc:333] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.490690   431 raft_consensus.cc:2243] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.492367   431 raft_consensus.cc:2272] T fb91933caa7241dc9c723195dd6950dc P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.495048   431 tablet_replica.cc:333] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
W20260709 13:04:01.495594  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:01.495885   431 raft_consensus.cc:2243] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.496693   431 raft_consensus.cc:2272] T 0835fee76231431da297ec1cf367640d P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.497678  2317 consensus_peers.cc:597] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.500721   431 tablet_replica.cc:333] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.501700   431 raft_consensus.cc:2243] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.502767   431 raft_consensus.cc:2272] T 316c6fe30c8f416c942b4338f4e6eab9 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.505427   431 tablet_replica.cc:333] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.506279   431 raft_consensus.cc:2243] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.507599   431 raft_consensus.cc:2272] T 37bfe7ffa6604774b0b035da58f96eb9 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.510190   431 tablet_replica.cc:333] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
W20260709 13:04:01.514070  1674 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.515450   431 raft_consensus.cc:2243] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.516170   431 raft_consensus.cc:2272] T 0fb354b7d6104034b98fab8201f2719e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.518877   431 tablet_replica.cc:333] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.522562   431 raft_consensus.cc:2243] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.525794   431 raft_consensus.cc:2272] T 8516bd3d9b8948f0a0a32c2e93d4673e P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.531780   431 tablet_replica.cc:333] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.536962   431 raft_consensus.cc:2243] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.537904   431 raft_consensus.cc:2272] T 9ac4670b4c614e9eb392b74198615198 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.540818   431 tablet_replica.cc:333] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.541599   431 raft_consensus.cc:2243] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.542454   431 raft_consensus.cc:2272] T 77e952b7f0f34618a5ce435fa48e0448 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.545492   431 tablet_replica.cc:333] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.546522   431 raft_consensus.cc:2243] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.547369   431 raft_consensus.cc:2272] T 3897029c4b5f443296641c39af287e98 P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.549944   431 tablet_replica.cc:333] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.550952   431 raft_consensus.cc:2243] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.551542   431 raft_consensus.cc:2272] T b5838267321348eb83df088988da6449 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.554174   431 tablet_replica.cc:333] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.554924   431 raft_consensus.cc:2243] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.555914   431 raft_consensus.cc:2272] T 946eea23a2334d678895281941dd9926 P ba0548b7d2a94e8598475a46c279fe0e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.559211   431 tablet_replica.cc:333] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.559978   431 raft_consensus.cc:2243] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.560806   431 raft_consensus.cc:2272] T e6edcffcaa5a46d2ba7c824073943b82 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.563633   431 tablet_replica.cc:333] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.564445   431 raft_consensus.cc:2243] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.565651   431 raft_consensus.cc:2272] T ce507f8325ba41dda6865b9492bfd0a7 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.568960   431 tablet_replica.cc:333] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.569805   431 raft_consensus.cc:2243] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.570509   431 raft_consensus.cc:2272] T bb0b8480ae5646bea5cd44d499b6bcaa P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.573441   431 tablet_replica.cc:333] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.574259   431 raft_consensus.cc:2243] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.575093   431 raft_consensus.cc:2272] T 26a92c141f564f638497aa02dd5c7c79 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.584477  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:01.587078  1674 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.590372   431 tablet_replica.cc:333] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.591487   431 raft_consensus.cc:2243] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.596886   431 raft_consensus.cc:2272] T 05b3e259298f4617a9f83f3e7cf738ac P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.597299  2317 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.598589  2317 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.600677   431 tablet_replica.cc:333] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.601745   431 raft_consensus.cc:2243] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.602910   431 raft_consensus.cc:2272] T 74f866a699e74063990bd566c7fb1ebf P ba0548b7d2a94e8598475a46c279fe0e [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.605266  1674 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.605860  2317 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.606853  2317 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.615975   431 tablet_replica.cc:333] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.617098   431 raft_consensus.cc:2243] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.617875   431 raft_consensus.cc:2272] T e6c211dcb5c947719413e384c6a72769 P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.620327  2317 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.621542   431 tablet_replica.cc:333] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.622601   431 raft_consensus.cc:2243] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.623632   431 raft_consensus.cc:2272] T 0cb744c0184549918fb0bb586614bb7f P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.627074   431 tablet_replica.cc:333] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.628099   431 raft_consensus.cc:2243] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.629592   431 raft_consensus.cc:2272] T eb99ad78ebb2479fbf4715bb9d33258e P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.633057   431 tablet_replica.cc:333] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.634023   431 raft_consensus.cc:2243] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.634881   431 raft_consensus.cc:2272] T 1457f31daeb646a9824ac8068949eddd P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.638190  1674 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.640151   431 tablet_replica.cc:333] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.641220   431 raft_consensus.cc:2243] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.642372   431 raft_consensus.cc:2272] T 61a6c336f56b4c248aac90993fb477cf P ba0548b7d2a94e8598475a46c279fe0e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.645866   431 tablet_replica.cc:333] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.646790   431 raft_consensus.cc:2243] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.647742   431 raft_consensus.cc:2272] T bc25e0c8bb834899a98a40efe1e66bba P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.651491   431 tablet_replica.cc:333] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.652323   431 raft_consensus.cc:2243] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.654816   431 raft_consensus.cc:2272] T 5ba40a6e1d714431be77e8a1518ec8d6 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.659337   431 tablet_replica.cc:333] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.660301   431 raft_consensus.cc:2243] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.661154   431 raft_consensus.cc:2272] T 8785cfce0d0042d3855cf9a7edaf1f7d P ba0548b7d2a94e8598475a46c279fe0e [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.666208  2317 consensus_peers.cc:597] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.666481   431 tablet_replica.cc:333] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.667573   431 raft_consensus.cc:2243] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.668974   431 raft_consensus.cc:2272] T 18511a70f641409b83d4cde6d4e56009 P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.678581   431 tablet_replica.cc:333] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.679731   431 raft_consensus.cc:2243] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.680678   431 raft_consensus.cc:2272] T 7db3e87d77f44ba19ad756277b557c45 P ba0548b7d2a94e8598475a46c279fe0e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.691512   431 tablet_replica.cc:333] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.692559   431 raft_consensus.cc:2243] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.693609   431 raft_consensus.cc:2272] T f5aa381f739e4c71869e157bdba8aa97 P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.697875  1674 consensus_peers.cc:597] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.700870   431 tablet_replica.cc:333] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.702261   431 raft_consensus.cc:2243] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.703751   431 raft_consensus.cc:2272] T 6b73223ab67f4b5aab700c65a70e42cf P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.707226   431 tablet_replica.cc:333] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.708166   431 raft_consensus.cc:2243] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.709520   431 raft_consensus.cc:2272] T 2c3a7d726b2f43c28355175906793af9 P ba0548b7d2a94e8598475a46c279fe0e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.712641   431 tablet_replica.cc:333] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.713578   431 raft_consensus.cc:2243] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.715636   431 raft_consensus.cc:2272] T 69b9dbe96d28487fa6d8a381ac6fc383 P ba0548b7d2a94e8598475a46c279fe0e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.718972   431 tablet_replica.cc:333] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.719964   431 raft_consensus.cc:2243] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.721032   431 raft_consensus.cc:2272] T 10315b15aaee40e0a1110e5ce8cf0abf P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.725052  1674 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.734213   431 tablet_replica.cc:333] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.743498   431 raft_consensus.cc:2243] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:01.745270   431 raft_consensus.cc:2272] T 29b074786b944dd19df72b17c4fbb001 P ba0548b7d2a94e8598475a46c279fe0e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.748953   431 tablet_replica.cc:333] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.766983   431 raft_consensus.cc:2243] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.769236   431 raft_consensus.cc:2272] T 696fb3c290c54f14a67f61bf78588eda P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.806201   431 tablet_replica.cc:333] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.807137   431 raft_consensus.cc:2243] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.808316   431 raft_consensus.cc:2272] T 563a5aa85e754fe48eaf5615d947ab9a P ba0548b7d2a94e8598475a46c279fe0e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.829906   431 tablet_replica.cc:333] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.831370   431 raft_consensus.cc:2243] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.832063   431 raft_consensus.cc:2272] T 70981cbf9c1143309f0f78706fe5931d P ba0548b7d2a94e8598475a46c279fe0e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.835852   431 tablet_replica.cc:333] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e: stopping tablet replica
I20260709 13:04:01.836679   431 raft_consensus.cc:2243] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.844506   431 raft_consensus.cc:2272] T e6c08bb7c6294cf9b33ad3b85e34831d P ba0548b7d2a94e8598475a46c279fe0e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:01.848065  2317 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:01.869930  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:04:01.890170  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:01.893110  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:02.373754  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 395843us
W20260709 13:04:02.433283  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 345478us
I20260709 13:04:02.478924 10401 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.479666 10401 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:04:02.484512  1675 consensus_peers.cc:597] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:04:02.484714 10401 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:02.485378  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:02.489423  1675 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:02.497941 10380 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.499681 10380 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:02.503571 10380 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:02.554224  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:04:02.554651  2317 connection.cc:769] client connection to 127.0.107.194:37097 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:04:02.555570  2318 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:02.556417  2318 leader_election.cc:336] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:02.561329  2317 leader_election.cc:336] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:02.562604  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:02.561966  2317 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:02.566972 10393 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:02.568504  2317 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:02.569062  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:02.570045 10393 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:02.595378  1675 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:04:02.629393 10405 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.630056 10405 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:02.632893 10405 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:04:02.634186  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:04:02.637028  1674 leader_election.cc:336] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:02.637499  1674 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:04:02.637856 10393 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.638422 10393 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:02.639204 10400 raft_consensus.cc:2749] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:02.639993 10400 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.640467 10400 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:04:02.640843 10393 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:02.647950  1675 leader_election.cc:336] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:02.650568  2318 leader_election.cc:336] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:02.651189 10400 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:02.655561  2317 leader_election.cc:336] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:02.656253  2317 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:02.657231 10403 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:02.657426  1674 leader_election.cc:336] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:02.657982  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:02.659209 10400 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:02.684559 10403 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.685248 10403 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:02.687690 10403 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:04:02.689512  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:04:02.689591 10403 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.690340  1707 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 20.
I20260709 13:04:02.690409 10403 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:02.693181 10403 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:04:02.695892  2318 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:04:02.696918 10403 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:02.697400 10403 raft_consensus.cc:493] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.697889 10403 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:02.698170  2317 leader_election.cc:336] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:02.702657  2317 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:02.706833  2318 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:02.707486  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:02.708000 10400 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.708460 10393 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:02.708626 10400 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:04:02.711819 10403 raft_consensus.cc:515] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:02.715066 10403 leader_election.cc:290] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:04:02.716202  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25feb0fae8094dd29fea235dd1af7a14" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:04:02.716902  1707 raft_consensus.cc:3060] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:02.718283  1675 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:02.719389 10405 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.719950 10405 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:02.722412 10405 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:04:02.725752  2351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
I20260709 13:04:02.727298  1707 raft_consensus.cc:2468] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
W20260709 13:04:02.729142  1674 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:02.729652  1674 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:02.731601 10400 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:04:02.732391 10405 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:02.736212  2318 leader_election.cc:304] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:04:02.737195 10393 raft_consensus.cc:2804] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:02.737807 10393 raft_consensus.cc:697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
W20260709 13:04:02.738072  1674 leader_election.cc:336] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:02.739923  1674 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:04:02.741645 10405 raft_consensus.cc:2749] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:02.738930 10393 consensus_queue.cc:237] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:04:02.748135  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:02.760993 10403 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
W20260709 13:04:02.761824  2317 leader_election.cc:336] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:02.761816 10403 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:02.767513 10403 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:04:02.771772  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:04:02.772641  1707 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 21.
I20260709 13:04:02.774041  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 
I20260709 13:04:02.775141 10393 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:02.775565 10393 raft_consensus.cc:493] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.776105 10393 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:02.788717  1675 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:02.778164  1445 catalog_manager.cc:4755] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8785cfce0d0042d3855cf9a7edaf1f7d: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:02.795697  2317 leader_election.cc:336] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:02.797788 10405 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.798475 10405 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
W20260709 13:04:02.807386  1674 leader_election.cc:336] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:02.808741 10405 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:02.828768  1675 leader_election.cc:336] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:02.829566  1459 catalog_manager.cc:5697] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 20 to 21, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 21 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:02.834038  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:02.834873 10405 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:02.835177 10393 raft_consensus.cc:515] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:04:02.845620  2317 leader_election.cc:336] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:02.847594 10393 leader_election.cc:290] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:04:02.848425  1707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce507f8325ba41dda6865b9492bfd0a7" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:04:02.849233  1707 raft_consensus.cc:3060] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:02.859366  1707 raft_consensus.cc:2468] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 22.
I20260709 13:04:02.860952  2318 leader_election.cc:304] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:04:02.862450 10393 raft_consensus.cc:2804] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:02.863380 10393 raft_consensus.cc:697] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:04:02.865213 10393 consensus_queue.cc:237] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:02.907995  1459 catalog_manager.cc:5697] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 21 to 22, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 22 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:02.931041  1675 consensus_peers.cc:597] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:02.950209 10400 raft_consensus.cc:493] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.950837 10400 raft_consensus.cc:515] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:04:02.952978 10400 leader_election.cc:290] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:02.962707  1675 leader_election.cc:336] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:02.967126  1674 leader_election.cc:336] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:02.967700  1674 leader_election.cc:304] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:02.968927 10406 raft_consensus.cc:2749] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:02.973690 10406 raft_consensus.cc:493] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:02.975035 10406 raft_consensus.cc:515] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:04:02.980827 10406 leader_election.cc:290] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:02.996030  1675 leader_election.cc:336] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:02.996651  1675 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.018069  1675 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.027083  1674 leader_election.cc:336] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:03.028415  1674 leader_election.cc:304] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:03.029280 10406 raft_consensus.cc:2749] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:03.050063  2317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111) [suppressed 552 similar messages]
I20260709 13:04:03.063690 10406 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:03.064316 10406 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:03.067291 10406 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196:36257)
I20260709 13:04:03.070370 10406 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:03.071018 10406 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:04:03.076529 10406 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:03.079474  1675 leader_election.cc:336] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:03.083376  1674 leader_election.cc:336] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:03.086074  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" is_pre_election: true
W20260709 13:04:03.087625  1674 leader_election.cc:336] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:03.088161  1676 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:04:03.088162  1674 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:03.089406 10400 raft_consensus.cc:2749] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:03.089658 10406 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:04:03.125337  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.148898  1675 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:03.167033 10403 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:03.168579 10403 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:03.170781 10403 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:04:03.174019  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" is_pre_election: true
I20260709 13:04:03.175132  1701 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 18.
W20260709 13:04:03.176460  2317 leader_election.cc:336] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:03.177582 10403 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:03.177827  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:04:03.179143 10393 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:03.179634 10393 raft_consensus.cc:493] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting leader election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:03.180111 10393 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.178297 10403 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:03.182914 10403 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:03.191586  2317 leader_election.cc:336] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:03.193354  2318 leader_election.cc:336] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:03.193945  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:03.194309 10393 raft_consensus.cc:515] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:03.195055 10403 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:03.197014 10393 leader_election.cc:290] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Requested vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:04:03.198698  1701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3a7d726b2f43c28355175906793af9" candidate_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8"
I20260709 13:04:03.199295  1701 raft_consensus.cc:3060] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:03.203025  2317 leader_election.cc:336] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:03.207500  1701 raft_consensus.cc:2468] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc6a4cf5874b7eb91280dffcd3b3fb in term 19.
I20260709 13:04:03.208912  2318 leader_election.cc:304] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb, a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:04:03.209817 10393 raft_consensus.cc:2804] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:03.210402 10393 raft_consensus.cc:697] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Becoming Leader. State: Replica: 95fc6a4cf5874b7eb91280dffcd3b3fb, State: Running, Role: LEADER
I20260709 13:04:03.211328 10393 consensus_queue.cc:237] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:03.227173  1459 catalog_manager.cc:5697] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb reported cstate change: term changed from 18 to 19, leader changed from ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194) to 95fc6a4cf5874b7eb91280dffcd3b3fb (127.0.107.196). New cstate: current_term: 19 leader_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:03.292685  2317 consensus_peers.cc:597] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:03.314870  1731 debug-util.cc:398] Leaking SignalData structure 0x7b08000ddca0 after lost signal to thread 434
W20260709 13:04:03.316448  1731 debug-util.cc:398] Leaking SignalData structure 0x7b08000f62e0 after lost signal to thread 1493
W20260709 13:04:03.317236  1731 debug-util.cc:398] Leaking SignalData structure 0x7b0800109c00 after lost signal to thread 1660
W20260709 13:04:03.318656  1731 debug-util.cc:398] Leaking SignalData structure 0x7b0800107040 after lost signal to thread 1734
W20260709 13:04:03.320259  1731 debug-util.cc:398] Leaking SignalData structure 0x7b08001a1440 after lost signal to thread 2377
I20260709 13:04:03.322099  1701 raft_consensus.cc:1275] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:04:03.323962 10393 consensus_queue.cc:1048] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:03.339751  1707 raft_consensus.cc:1275] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:04:03.342825 10393 consensus_queue.cc:1048] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:04:03.455298  1675 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.455886  1675 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.498423  1675 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.531872  1675 consensus_peers.cc:597] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.599735  1675 consensus_peers.cc:597] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:03.618644  2317 consensus_peers.cc:597] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:03.646855  1675 consensus_peers.cc:597] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:03.701743  1675 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.711517  2317 consensus_peers.cc:597] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:03.731609  1707 raft_consensus.cc:1275] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Refusing update from remote peer 95fc6a4cf5874b7eb91280dffcd3b3fb: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
W20260709 13:04:03.737628  1675 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.759018  2317 consensus_peers.cc:597] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:03.769721 10423 consensus_queue.cc:1048] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:04:03.779220  1674 consensus_peers.cc:597] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:03.814486  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:03.854650  1675 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.880462  2318 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 51601.3us
W20260709 13:04:03.885407  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.881968 10424 log.cc:927] Time spent T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:04:03.909572  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:04:03.914143  1675 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.916874  1675 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:03.930307  1674 consensus_peers.cc:597] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:03.941044  1674 consensus_peers.cc:597] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:03.948228  1674 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:03.950178  2317 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:03.972704  1674 consensus_peers.cc:597] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:03.978142  1674 consensus_peers.cc:597] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.014580  2317 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.028017  2317 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.030443  1674 consensus_peers.cc:597] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.031644  1674 consensus_peers.cc:597] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.033381  1675 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:04.055076  1674 consensus_peers.cc:597] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.061131  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:04.087584  2317 consensus_peers.cc:597] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:04.142661   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:04:04.208154  1675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33149, user_credentials={real_user=slave}} blocked reactor thread for 93140.4us
I20260709 13:04:04.227604 10430 raft_consensus.cc:493] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:04.243237 10430 raft_consensus.cc:515] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:04:04.268056 10430 leader_election.cc:290] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:04.271945  2317 consensus_peers.cc:597] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:04.283551 10435 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:04.285703 10435 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:04.288013 10435 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:04.290448  2317 consensus_peers.cc:597] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.292018  1675 consensus_peers.cc:597] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:04.292850  1675 consensus_peers.cc:597] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:04.305639  1674 consensus_peers.cc:597] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.308036  1675 leader_election.cc:336] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:04.315668  2317 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.319798  2317 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.320278  2317 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:04.320773  2317 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.324898  2317 consensus_peers.cc:597] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.328361  1674 leader_election.cc:336] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:04.328929  1674 leader_election.cc:304] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
W20260709 13:04:04.338960  1674 consensus_peers.cc:597] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:04.341377 10430 raft_consensus.cc:2749] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:04.344659  1674 consensus_peers.cc:597] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.348753  1674 consensus_peers.cc:597] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.360960  1674 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.361232  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:04.381454  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:04.382109  2318 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:04.382774  2318 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:04.384164 10437 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:04.387302 10437 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:04.387867 10437 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:04.391014 10437 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:04.397866  2317 leader_election.cc:336] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:04.400389  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:04.403679  2318 leader_election.cc:336] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:04.414917  2317 consensus_peers.cc:597] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:04.416225  2318 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:04.417282 10423 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:04.483428  1675 consensus_peers.cc:597] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:04.501035  1674 consensus_peers.cc:597] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.504381  1674 consensus_peers.cc:597] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:04.519955 10423 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:04.520789 10423 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:04:04.534819  2317 leader_election.cc:336] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:04.544040 10423 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:04.550412  2318 leader_election.cc:336] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:04.550936  2318 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:04.553113 10423 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:04.547374   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:04:04.557453 10433 raft_consensus.cc:493] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:04.558465 10433 raft_consensus.cc:515] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:04:04.561408 10433 leader_election.cc:290] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:04.570071  1674 consensus_peers.cc:597] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.589573  1674 leader_election.cc:336] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:04.592571  1674 consensus_peers.cc:597] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.603021  1674 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.627458  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:04:04.630945  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:04.634050  1675 leader_election.cc:336] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:04.634865  1675 leader_election.cc:304] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:04.636425 10433 raft_consensus.cc:2749] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:04.643500  2318 consensus_peers.cc:597] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:04.644570  2318 consensus_peers.cc:597] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:04.645267  2318 consensus_peers.cc:597] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:04.645980  2318 consensus_peers.cc:597] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:04.654446  2318 consensus_peers.cc:597] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:04.663062  1674 consensus_peers.cc:597] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:04.681463  2317 consensus_peers.cc:597] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:04.684785   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:04.692399   431 tablet_replica.cc:333] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.694906   431 raft_consensus.cc:2243] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.695659   431 raft_consensus.cc:2272] T 25feb0fae8094dd29fea235dd1af7a14 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.699429   431 tablet_replica.cc:333] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.703536 10423 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:04.704278 10423 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:04.707170 10423 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:04:04.709256   431 raft_consensus.cc:2243] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.710210   431 raft_consensus.cc:2272] T ac931d9355284a168be64d7debd009ae P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.710810 10433 raft_consensus.cc:493] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:04.711385 10433 raft_consensus.cc:515] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:04:04.714083 10433 leader_election.cc:290] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:04:04.719182   431 tablet_replica.cc:333] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.720350   431 raft_consensus.cc:2243] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.721414   431 raft_consensus.cc:2272] T ca209b0d98c64e1da48e3027b3ea1d9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:04.721941  1674 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:04.724834   431 tablet_replica.cc:333] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
W20260709 13:04:04.724970  1675 leader_election.cc:336] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:04.726099   431 raft_consensus.cc:2243] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.725701  1675 leader_election.cc:304] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:04.727685   431 raft_consensus.cc:2272] T 8cb801d86c874e1686eceacb9de1d0a6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.731056 10436 raft_consensus.cc:2749] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:04:04.728071  2317 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:04.734433   431 tablet_replica.cc:333] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.735538   431 raft_consensus.cc:2243] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.736902   431 raft_consensus.cc:2272] T 5ba40a6e1d714431be77e8a1518ec8d6 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.740434   431 tablet_replica.cc:333] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.741662   431 raft_consensus.cc:2243] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.743228   431 raft_consensus.cc:2272] T 472ed50e0865403c81123d4efe7108d7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.747651   431 tablet_replica.cc:333] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.750519   431 raft_consensus.cc:2243] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.751430   431 raft_consensus.cc:2272] T 43dce0aaab204cb084a73a74a80bd7a3 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.759711   431 tablet_replica.cc:333] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.760679   431 raft_consensus.cc:2243] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.761767   431 raft_consensus.cc:2272] T 637e660a66be4924866287fa6150aaea P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.764878   431 tablet_replica.cc:333] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.765910   431 raft_consensus.cc:2243] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.767866   431 raft_consensus.cc:2272] T 316c6fe30c8f416c942b4338f4e6eab9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:04.764426  2318 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:04.771126   431 tablet_replica.cc:333] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.772166   431 raft_consensus.cc:2243] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.768606  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:04.773406   431 raft_consensus.cc:2272] T 6b24efe048e6499b923f1a0215713fad P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.774755 10423 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:04.776638   431 tablet_replica.cc:333] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.777688   431 raft_consensus.cc:2243] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.778375   431 raft_consensus.cc:2272] T 8516bd3d9b8948f0a0a32c2e93d4673e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.781333   431 tablet_replica.cc:333] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
W20260709 13:04:04.781705  2317 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:04.782263   431 raft_consensus.cc:2243] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.783608   431 raft_consensus.cc:2272] T 9ac4670b4c614e9eb392b74198615198 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.788054   431 tablet_replica.cc:333] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.789096   431 raft_consensus.cc:2243] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.789906   431 raft_consensus.cc:2272] T 0835fee76231431da297ec1cf367640d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.793509   431 tablet_replica.cc:333] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.800787   431 raft_consensus.cc:2243] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.802309   431 raft_consensus.cc:2272] T 0fb354b7d6104034b98fab8201f2719e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:04.804642  2318 consensus_peers.cc:597] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:04.805830   431 tablet_replica.cc:333] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.806782   431 raft_consensus.cc:2243] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.808193   431 raft_consensus.cc:2272] T 77e952b7f0f34618a5ce435fa48e0448 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.811565   431 tablet_replica.cc:333] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.812543   431 raft_consensus.cc:2243] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.813896   431 raft_consensus.cc:2272] T 3897029c4b5f443296641c39af287e98 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.816931   431 tablet_replica.cc:333] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.817848   431 raft_consensus.cc:2243] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.818825   431 raft_consensus.cc:2272] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.822513   431 tablet_replica.cc:333] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.823361   431 raft_consensus.cc:2243] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.824100   431 raft_consensus.cc:2272] T b5838267321348eb83df088988da6449 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.827690   431 tablet_replica.cc:333] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.828678   431 raft_consensus.cc:2243] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.830098   431 raft_consensus.cc:2272] T 0cb744c0184549918fb0bb586614bb7f P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.833652   431 tablet_replica.cc:333] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.834586   431 raft_consensus.cc:2243] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.835319   431 raft_consensus.cc:2272] T eb99ad78ebb2479fbf4715bb9d33258e P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.838645   431 tablet_replica.cc:333] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.839656   431 raft_consensus.cc:2243] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.840407   431 raft_consensus.cc:2272] T 22f9010f94804fcaacae8539ddfc64a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:04.840988  2317 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:04.843681   431 tablet_replica.cc:333] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.844637   431 raft_consensus.cc:2243] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.846038   431 raft_consensus.cc:2272] T e6edcffcaa5a46d2ba7c824073943b82 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.849095   431 tablet_replica.cc:333] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.850061   431 raft_consensus.cc:2243] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.850731   431 raft_consensus.cc:2272] T ce507f8325ba41dda6865b9492bfd0a7 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.853574   431 tablet_replica.cc:333] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.854413   431 raft_consensus.cc:2243] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.855016   431 raft_consensus.cc:2272] T bb0b8480ae5646bea5cd44d499b6bcaa P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.858052   431 tablet_replica.cc:333] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.858916   431 raft_consensus.cc:2243] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.860392   431 raft_consensus.cc:2272] T 26a92c141f564f638497aa02dd5c7c79 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.863720   431 tablet_replica.cc:333] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.864543   431 raft_consensus.cc:2243] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.865280   431 raft_consensus.cc:2272] T 05b3e259298f4617a9f83f3e7cf738ac P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.868486   431 tablet_replica.cc:333] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.869614   431 raft_consensus.cc:2243] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.871021   431 raft_consensus.cc:2272] T c96b268f1f734ec793355ab247e28a75 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.874325   431 tablet_replica.cc:333] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.875236   431 raft_consensus.cc:2243] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.876560   431 raft_consensus.cc:2272] T 74f866a699e74063990bd566c7fb1ebf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.879983   431 tablet_replica.cc:333] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.881330   431 raft_consensus.cc:2243] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.882845   431 raft_consensus.cc:2272] T 9a353430d15945ee9524da3d2b667c90 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:04.879983  1675 consensus_peers.cc:597] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:04:04.886121   431 tablet_replica.cc:333] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.887154   431 raft_consensus.cc:2243] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.889183   431 raft_consensus.cc:2272] T 0adeaf85877a4103bbd070927413d267 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.892592   431 tablet_replica.cc:333] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.893832   431 raft_consensus.cc:2243] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.895231   431 raft_consensus.cc:2272] T 169e9f6ed9684060b36e93c778925461 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.898444   431 tablet_replica.cc:333] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.899365   431 raft_consensus.cc:2243] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:04.899767  2318 consensus_peers.cc:597] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:04.900068   431 raft_consensus.cc:2272] T bc25e0c8bb834899a98a40efe1e66bba P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.903203   431 tablet_replica.cc:333] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.904127   431 raft_consensus.cc:2243] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.905040   431 raft_consensus.cc:2272] T 939d7b92438a4b20bd5f2e81d2291c03 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.907987   431 tablet_replica.cc:333] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.909354   431 raft_consensus.cc:2243] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.910633   431 raft_consensus.cc:2272] T f5aa381f739e4c71869e157bdba8aa97 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.913522   431 tablet_replica.cc:333] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.913645 10426 raft_consensus.cc:493] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:04.914392   431 raft_consensus.cc:2243] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.914203 10426 raft_consensus.cc:515] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } }
I20260709 13:04:04.915715   431 raft_consensus.cc:2272] T 18511a70f641409b83d4cde6d4e56009 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.916519 10426 leader_election.cc:290] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:04:04.918705   431 tablet_replica.cc:333] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.919648   431 raft_consensus.cc:2243] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.920457   431 raft_consensus.cc:2272] T 7db3e87d77f44ba19ad756277b557c45 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:04.923261  1674 leader_election.cc:336] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:04.923615   431 tablet_replica.cc:333] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.924506   431 raft_consensus.cc:2243] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.926049   431 raft_consensus.cc:2272] T 2f47909c83d54b699ec2b1c2402084b4 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:04.926437  2318 consensus_peers.cc:597] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:04.929108   431 tablet_replica.cc:333] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.930058   431 raft_consensus.cc:2243] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.931288   431 raft_consensus.cc:2272] T 8785cfce0d0042d3855cf9a7edaf1f7d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.934302   431 tablet_replica.cc:333] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.935164   431 raft_consensus.cc:2243] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.936412   431 raft_consensus.cc:2272] T 1457f31daeb646a9824ac8068949eddd P a2ba7fc60f9641cdac25f1e016ef91d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.939474   431 tablet_replica.cc:333] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.940313   431 raft_consensus.cc:2243] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.941188   431 raft_consensus.cc:2272] T 61a6c336f56b4c248aac90993fb477cf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.944281   431 tablet_replica.cc:333] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
W20260709 13:04:04.944808  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:04:04.945058   431 raft_consensus.cc:2243] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.947095   431 raft_consensus.cc:2272] T 2c3a7d726b2f43c28355175906793af9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:04.949023  1675 leader_election.cc:336] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:04.953758  1675 leader_election.cc:304] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2ba7fc60f9641cdac25f1e016ef91d8; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:04.957528 10426 raft_consensus.cc:2749] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:04.956748   431 tablet_replica.cc:333] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.966738   431 raft_consensus.cc:2243] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.967705   431 raft_consensus.cc:2272] T 69b9dbe96d28487fa6d8a381ac6fc383 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.970937   431 tablet_replica.cc:333] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.971830   431 raft_consensus.cc:2243] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:04.973230   431 raft_consensus.cc:2272] T 10315b15aaee40e0a1110e5ce8cf0abf P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.986136   431 tablet_replica.cc:333] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:04.986996   431 raft_consensus.cc:2243] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.987630   431 raft_consensus.cc:2272] T 29b074786b944dd19df72b17c4fbb001 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:04.988432  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:04.989538  2318 consensus_peers.cc:597] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:04.990422   431 tablet_replica.cc:333] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
W20260709 13:04:04.990465  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:04.991220   431 raft_consensus.cc:2243] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:05.001906   431 raft_consensus.cc:2272] T 563a5aa85e754fe48eaf5615d947ab9a P a2ba7fc60f9641cdac25f1e016ef91d8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:05.005177   431 tablet_replica.cc:333] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:05.006107   431 raft_consensus.cc:2243] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:05.007524   431 raft_consensus.cc:2272] T 696fb3c290c54f14a67f61bf78588eda P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:05.010888   431 tablet_replica.cc:333] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:05.011718   431 raft_consensus.cc:2243] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:05.012374   431 raft_consensus.cc:2272] T 70981cbf9c1143309f0f78706fe5931d P a2ba7fc60f9641cdac25f1e016ef91d8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:05.015618   431 tablet_replica.cc:333] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8: stopping tablet replica
I20260709 13:04:05.016661   431 raft_consensus.cc:2243] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:04:05.018064   431 raft_consensus.cc:2272] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P a2ba7fc60f9641cdac25f1e016ef91d8 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:05.018838  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:05.034991  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:05.037257  2318 consensus_peers.cc:597] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:05.041621 10423 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:05.042346 10423 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:05.044838 10423 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:05.054211  2317 leader_election.cc:336] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:05.066829  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:05.077965  2318 leader_election.cc:336] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:05.078647  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:05.079561 10423 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:05.301798  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:05.434512  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:05.758265  2318 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:04:05.828701  2317 consensus_peers.cc:597] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:05.858584 10437 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:05.859282 10437 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:05.862515 10437 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:04:05.872890  2317 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:05.874588  2318 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
I20260709 13:04:05.875168  2318 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:04:05.876111 10431 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:05.878037  2318 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:06.018006 10431 raft_consensus.cc:493] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:06.018702 10431 raft_consensus.cc:515] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:06.021175 10431 leader_election.cc:290] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:04:06.030280  2318 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
W20260709 13:04:06.031167  2318 leader_election.cc:336] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:06.031564  2318 leader_election.cc:304] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:06.032366 10431 raft_consensus.cc:2749] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:06.059788  2317 consensus_peers.cc:597] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:06.076319 10437 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:06.076979 10437 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:06.079331 10437 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:04:06.082587  2317 consensus_peers.cc:597] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:06.090375  2318 leader_election.cc:336] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:06.091488  2318 leader_election.cc:336] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
I20260709 13:04:06.092231  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:06.093055 10437 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:06.100167 10431 raft_consensus.cc:493] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:06.100816 10431 raft_consensus.cc:515] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:06.104331 10431 leader_election.cc:290] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:06.110311  2317 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:06.114686  2318 leader_election.cc:336] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
I20260709 13:04:06.115309  2318 leader_election.cc:304] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:04:06.116109 10431 raft_consensus.cc:2749] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:06.132634 10431 raft_consensus.cc:493] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:06.133316 10431 raft_consensus.cc:515] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:06.135748 10431 leader_election.cc:290] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:04:06.141803  2318 leader_election.cc:336] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:06.142268 10431 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:06.142885 10431 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:04:06.144909  2318 leader_election.cc:336] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
I20260709 13:04:06.145234 10437 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:06.145574  2318 leader_election.cc:304] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:06.145895 10437 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:06.146617 10439 raft_consensus.cc:2749] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:06.148291 10437 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:04:06.151795 10431 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:04:06.155959  2318 leader_election.cc:336] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:06.157258  2318 leader_election.cc:336] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
I20260709 13:04:06.157745  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:06.158738 10431 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:04:06.161692  2318 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:06.162650  2318 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
I20260709 13:04:06.163056  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:06.163884 10431 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:04:06.182528   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 1731)
W20260709 13:04:06.186483  2318 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
I20260709 13:04:06.249814 10437 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:06.250569 10437 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:06.253705 10437 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:04:06.258984  2318 leader_election.cc:336] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
W20260709 13:04:06.259697  2317 leader_election.cc:336] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:06.260236  2317 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:04:06.261066 10437 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:06.318449 10431 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:06.319056 10431 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:06.321360 10431 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:04:06.334188  2318 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:06.335124  2318 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
I20260709 13:04:06.335608  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:06.336485 10431 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:06.352777 10431 raft_consensus.cc:493] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:06.353410 10431 raft_consensus.cc:515] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:06.358554 10431 leader_election.cc:290] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:06.364259  2318 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:06.365768  2317 leader_election.cc:336] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:06.366580  2317 leader_election.cc:304] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:06.368283 10439 raft_consensus.cc:2749] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:06.406232  2317 consensus_peers.cc:597] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.445261  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:04:06.456367  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:04:06.457858  2317 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.519277  2317 consensus_peers.cc:597] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.604883  2317 consensus_peers.cc:597] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.605955  2318 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:04:06.608021  2317 consensus_peers.cc:597] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.617871  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:04:06.639201 10439 raft_consensus.cc:493] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:06.639940 10439 raft_consensus.cc:515] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:06.642737 10439 leader_election.cc:290] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:06.648512  2317 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:06.650049  2318 leader_election.cc:336] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:06.650594  2318 leader_election.cc:304] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:06.651392 10439 raft_consensus.cc:2749] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:06.721647  2317 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.773402  2317 consensus_peers.cc:597] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.831877  2317 consensus_peers.cc:597] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.857753  2317 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.869233  2318 consensus_peers.cc:597] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:04:06.891265  2317 consensus_peers.cc:597] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.933279  2317 consensus_peers.cc:597] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.948773  2317 consensus_peers.cc:597] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:06.976384  2318 consensus_peers.cc:597] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.014370  2318 consensus_peers.cc:597] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.052940  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:04:07.053236  2317 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:07.059391  2318 consensus_peers.cc:597] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:07.090611  2318 consensus_peers.cc:597] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.121565  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:07.176888 10431 raft_consensus.cc:493] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:07.177583 10431 raft_consensus.cc:515] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:07.182947 10431 leader_election.cc:290] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:07.186326  2317 leader_election.cc:336] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:07.189159  2318 consensus_peers.cc:597] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.189949  2318 leader_election.cc:336] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:07.190388  2318 leader_election.cc:304] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:07.191187 10431 raft_consensus.cc:2749] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:07.214864  2318 consensus_peers.cc:597] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.238785  2317 consensus_peers.cc:597] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:07.267010 10439 raft_consensus.cc:493] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:07.267709 10439 raft_consensus.cc:515] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:04:07.269120  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:07.270113 10439 leader_election.cc:290] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
I20260709 13:04:07.282526 10441 raft_consensus.cc:493] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba0548b7d2a94e8598475a46c279fe0e)
I20260709 13:04:07.283149 10441 raft_consensus.cc:515] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:04:07.283883  2317 leader_election.cc:336] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:07.285895 10441 leader_election.cc:290] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097)
W20260709 13:04:07.287015  2318 consensus_peers.cc:597] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.287688  2318 consensus_peers.cc:597] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.288637  2318 leader_election.cc:336] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:07.289108  2318 leader_election.cc:304] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:07.289980 10441 raft_consensus.cc:2749] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:07.292975  2317 leader_election.cc:336] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:07.296106  2318 consensus_peers.cc:597] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.298513  2318 leader_election.cc:336] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:07.299088  2318 leader_election.cc:304] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:07.299962 10431 raft_consensus.cc:2749] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:07.316787  2318 consensus_peers.cc:597] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.347314  2318 consensus_peers.cc:597] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.363519  2317 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Couldn't send request to peer ba0548b7d2a94e8598475a46c279fe0e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:07.402171  2318 consensus_peers.cc:597] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.484053  2318 consensus_peers.cc:597] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.495224  2318 consensus_peers.cc:597] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:04:07.504163  2318 consensus_peers.cc:597] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.559371  2318 consensus_peers.cc:597] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Couldn't send request to peer a2ba7fc60f9641cdac25f1e016ef91d8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:07.578209  2318 consensus_peers.cc:597] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:04:07.583865   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:04:07.625011 10441 raft_consensus.cc:493] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:07.626456 10441 raft_consensus.cc:515] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } attrs { replace: true } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:04:07.635296  2317 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
W20260709 13:04:07.638056  2318 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:07.639403  2318 leader_election.cc:304] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: ba0548b7d2a94e8598475a46c279fe0e, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:07.640705 10431 raft_consensus.cc:2749] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:07.650853 10441 leader_election.cc:290] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:04:07.663657  2318 leader_election.cc:336] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
W20260709 13:04:07.703969  2318 consensus_peers.cc:597] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb -> Peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Couldn't send request to peer d62ce354d09a4be59c29094425d09bc1. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:04:07.710171 10441 raft_consensus.cc:493] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:07.711233 10441 raft_consensus.cc:515] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:07.714653 10441 leader_election.cc:290] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
W20260709 13:04:07.725430  2318 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:07.726557  2318 leader_election.cc:336] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
I20260709 13:04:07.727109  2318 leader_election.cc:304] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:07.728107 10441 raft_consensus.cc:2749] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:07.790223 10431 raft_consensus.cc:493] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:07.791167 10431 raft_consensus.cc:515] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:07.794793 10431 leader_election.cc:290] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:04:07.799829   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
W20260709 13:04:07.805020  2318 leader_election.cc:336] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
W20260709 13:04:07.808302  2318 leader_election.cc:336] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:07.809387 10431 raft_consensus.cc:493] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:07.810331 10431 raft_consensus.cc:515] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ba0548b7d2a94e8598475a46c279fe0e" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37097 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:07.812932 10431 leader_election.cc:290] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:04:07.808995  2318 leader_election.cc:304] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:07.817008 10441 raft_consensus.cc:2749] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:07.826820  2317 leader_election.cc:336] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ba0548b7d2a94e8598475a46c279fe0e (127.0.107.194:37097): Network error: Client connection negotiation failed: client connection to 127.0.107.194:37097: connect: Connection refused (error 111)
I20260709 13:04:07.828410 10445 raft_consensus.cc:493] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:07.829305 10445 raft_consensus.cc:515] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
W20260709 13:04:07.838529  2318 leader_election.cc:336] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
I20260709 13:04:07.839282  2318 leader_election.cc:304] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, ba0548b7d2a94e8598475a46c279fe0e
I20260709 13:04:07.838635 10445 leader_election.cc:290] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:04:07.840186 10441 raft_consensus.cc:2749] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:07.852602  2318 leader_election.cc:336] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:07.854074  2318 leader_election.cc:336] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
I20260709 13:04:07.854609  2318 leader_election.cc:304] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:07.855502 10441 raft_consensus.cc:2749] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:07.893057   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:07.895772   431 tablet_replica.cc:333] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.897261   431 raft_consensus.cc:2243] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.899382   431 raft_consensus.cc:2272] T 316c6fe30c8f416c942b4338f4e6eab9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.903507   431 tablet_replica.cc:333] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.904603   431 raft_consensus.cc:2243] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.908265   431 raft_consensus.cc:2272] T 61a6c336f56b4c248aac90993fb477cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.909560 10441 raft_consensus.cc:493] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2ba7fc60f9641cdac25f1e016ef91d8)
I20260709 13:04:07.910254 10441 raft_consensus.cc:515] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d62ce354d09a4be59c29094425d09bc1" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33149 } } peers { permanent_uuid: "a2ba7fc60f9641cdac25f1e016ef91d8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45431 } } peers { permanent_uuid: "95fc6a4cf5874b7eb91280dffcd3b3fb" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 36257 } attrs { promote: false } }
I20260709 13:04:07.912551 10441 leader_election.cc:290] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149), a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431)
I20260709 13:04:07.912847   431 tablet_replica.cc:333] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.914435   431 raft_consensus.cc:2243] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.916390   431 raft_consensus.cc:2272] T 8785cfce0d0042d3855cf9a7edaf1f7d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:07.918603  2318 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:07.919519  2318 leader_election.cc:336] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a2ba7fc60f9641cdac25f1e016ef91d8 (127.0.107.195:45431): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111)
I20260709 13:04:07.919931  2318 leader_election.cc:304] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95fc6a4cf5874b7eb91280dffcd3b3fb; no voters: a2ba7fc60f9641cdac25f1e016ef91d8, d62ce354d09a4be59c29094425d09bc1
I20260709 13:04:07.920671 10441 raft_consensus.cc:2749] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:07.921226   431 tablet_replica.cc:333] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.922392   431 raft_consensus.cc:2243] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.924942   431 raft_consensus.cc:2272] T 637e660a66be4924866287fa6150aaea P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.928915   431 tablet_replica.cc:333] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.930099   431 raft_consensus.cc:2243] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.930998   431 raft_consensus.cc:2272] T 1457f31daeb646a9824ac8068949eddd P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.935460   431 tablet_replica.cc:333] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.936453   431 raft_consensus.cc:2243] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.938633   431 raft_consensus.cc:2272] T 83a61af0ea7d47c1bc0ee6a94a2d1a7f P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.943367   431 tablet_replica.cc:333] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.944567   431 raft_consensus.cc:2243] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.947510   431 raft_consensus.cc:2272] T e6c08bb7c6294cf9b33ad3b85e34831d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.951598   431 tablet_replica.cc:333] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.952474   431 raft_consensus.cc:2243] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.954171   431 raft_consensus.cc:2272] T 4e2094a808984c84823c1e0dad0a5de0 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.957839   431 tablet_replica.cc:333] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.958688   431 raft_consensus.cc:2243] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.960438   431 raft_consensus.cc:2272] T e6c211dcb5c947719413e384c6a72769 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.964107   431 tablet_replica.cc:333] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.965216   431 raft_consensus.cc:2243] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.967173   431 raft_consensus.cc:2272] T 70981cbf9c1143309f0f78706fe5931d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.971097   431 tablet_replica.cc:333] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.972213   431 raft_consensus.cc:2243] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.973956   431 raft_consensus.cc:2272] T 22f9010f94804fcaacae8539ddfc64a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.977960   431 tablet_replica.cc:333] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.978886   431 raft_consensus.cc:2243] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.980072   431 raft_consensus.cc:2272] T 2f47909c83d54b699ec2b1c2402084b4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.984124   431 tablet_replica.cc:333] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.985129   431 raft_consensus.cc:2243] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.986203   431 raft_consensus.cc:2272] T 13debc393e8a4121883dda403cb4badf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.990622   431 tablet_replica.cc:333] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.991578   431 raft_consensus.cc:2243] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.993364   431 raft_consensus.cc:2272] T 7db3e87d77f44ba19ad756277b557c45 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.996897   431 tablet_replica.cc:333] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:07.997766   431 raft_consensus.cc:2243] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.999565   431 raft_consensus.cc:2272] T 25feb0fae8094dd29fea235dd1af7a14 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.002910   431 tablet_replica.cc:333] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.003754   431 raft_consensus.cc:2243] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.005390   431 raft_consensus.cc:2272] T ca209b0d98c64e1da48e3027b3ea1d9a P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.008628   431 tablet_replica.cc:333] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.009517   431 raft_consensus.cc:2243] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:08.010291   431 raft_consensus.cc:2272] T 8b022b45a9b7462bb4b1a2a30ae36aa9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.014263   431 tablet_replica.cc:333] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.015149   431 raft_consensus.cc:2243] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.016916   431 raft_consensus.cc:2272] T 2c3a7d726b2f43c28355175906793af9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.020897   431 tablet_replica.cc:333] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.021901   431 raft_consensus.cc:2243] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.024055   431 raft_consensus.cc:2272] T 939d7b92438a4b20bd5f2e81d2291c03 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.028199   431 tablet_replica.cc:333] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.029240   431 raft_consensus.cc:2243] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:08.030194   431 raft_consensus.cc:2272] T c96b268f1f734ec793355ab247e28a75 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.033895   431 tablet_replica.cc:333] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.034885   431 raft_consensus.cc:2243] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:08.035620   431 raft_consensus.cc:2272] T 9a353430d15945ee9524da3d2b667c90 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.039292   431 tablet_replica.cc:333] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.040283   431 raft_consensus.cc:2243] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:08.041265   431 raft_consensus.cc:2272] T c2ba61e8020a4800b9e7fdae5533800b P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.045120   431 tablet_replica.cc:333] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.046047   431 raft_consensus.cc:2243] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.047646   431 raft_consensus.cc:2272] T bb0b8480ae5646bea5cd44d499b6bcaa P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.051270   431 tablet_replica.cc:333] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.052145   431 raft_consensus.cc:2243] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:08.053110   431 raft_consensus.cc:2272] T 61e7345bb41047a2896de16e89540bbb P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:08.053498  2318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45431: connect: Connection refused (error 111) [suppressed 615 similar messages]
I20260709 13:04:08.057168   431 tablet_replica.cc:333] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.058247   431 raft_consensus.cc:2243] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.059708   431 raft_consensus.cc:2272] T 37bfe7ffa6604774b0b035da58f96eb9 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.063292   431 tablet_replica.cc:333] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.064111   431 raft_consensus.cc:2243] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:08.065124   431 raft_consensus.cc:2272] T 169e9f6ed9684060b36e93c778925461 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.068321   431 tablet_replica.cc:333] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.069119   431 raft_consensus.cc:2243] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.071178   431 raft_consensus.cc:2272] T d72a925fb22046e2be514a7913c613a4 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.074572   431 tablet_replica.cc:333] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.075423   431 raft_consensus.cc:2243] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.077368   431 raft_consensus.cc:2272] T bc25e0c8bb834899a98a40efe1e66bba P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.082340   431 tablet_replica.cc:333] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.083266   431 raft_consensus.cc:2243] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:08.084287   431 raft_consensus.cc:2272] T 0adeaf85877a4103bbd070927413d267 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.087667   431 tablet_replica.cc:333] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.088501   431 raft_consensus.cc:2243] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.090008   431 raft_consensus.cc:2272] T b5838267321348eb83df088988da6449 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.093720   431 tablet_replica.cc:333] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.094663   431 raft_consensus.cc:2243] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.096269   431 raft_consensus.cc:2272] T ac931d9355284a168be64d7debd009ae P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.099591   431 tablet_replica.cc:333] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.100419   431 raft_consensus.cc:2243] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.102200   431 raft_consensus.cc:2272] T ce507f8325ba41dda6865b9492bfd0a7 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.105461   431 tablet_replica.cc:333] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.106258   431 raft_consensus.cc:2243] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.107997   431 raft_consensus.cc:2272] T 946eea23a2334d678895281941dd9926 P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.111209   431 tablet_replica.cc:333] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.111938   431 raft_consensus.cc:2243] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:08.112982   431 raft_consensus.cc:2272] T 6b73223ab67f4b5aab700c65a70e42cf P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.116120   431 tablet_replica.cc:333] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.116820   431 raft_consensus.cc:2243] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:08.118078   431 raft_consensus.cc:2272] T fb91933caa7241dc9c723195dd6950dc P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.121398   431 tablet_replica.cc:333] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb: stopping tablet replica
I20260709 13:04:08.122047   431 raft_consensus.cc:2243] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:08.123523   431 raft_consensus.cc:2272] T 0835fee76231431da297ec1cf367640d P 95fc6a4cf5874b7eb91280dffcd3b3fb [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.542966   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:04:08.637025   431 master.cc:562] Master@127.0.107.254:43857 shutting down...
W20260709 13:04:08.682313  1507 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:04:08.682736  1507 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:04:08.682991  1507 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:04:35.600318  1445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
W20260709 13:04:35.604668  1445 catalog_manager.cc:4755] TS d62ce354d09a4be59c29094425d09bc1 (127.0.107.193:33149): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8785cfce0d0042d3855cf9a7edaf1f7d: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33149: connect: Connection refused (error 111)
I20260709 13:04:41.865547   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.867393   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.867812   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8f622dabf84e4c0595473d2c01f108d0: stopping tablet replica
I20260709 13:04:42.116503   431 master.cc:584] Master@127.0.107.254:43857 shutdown complete.
I20260709 13:04:42.186710   431 test_util.cc:182] -----------------------------------------------
I20260709 13:04:42.187005   431 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601924925497-431-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (331760 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:04:42.191689   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:34253
I20260709 13:04:42.193100   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:42.200426 10451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.200431 10452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.200755 10454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:42.202788   431 server_base.cc:1061] running on GCE node
I20260709 13:04:42.207294   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:42.207567   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:42.207752   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602282207740 us; error 0 us; skew 500 ppm
I20260709 13:04:42.208424   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:42.211979   431 webserver.cc:533] Webserver started at http://127.0.107.254:38017/ using document root <none> and password file <none>
I20260709 13:04:42.212657   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:42.212867   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:42.213223   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:42.214784   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/master-0-root/instance:
uuid: "c599466512fc46e28facaef651433292"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-l8g5"
I20260709 13:04:42.221052   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:04:42.225775 10459 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.226794   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:04:42.227128   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/master-0-root
uuid: "c599466512fc46e28facaef651433292"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-l8g5"
I20260709 13:04:42.227443   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:42.281340   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:42.282950   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:42.330699   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:34253
I20260709 13:04:42.330834 10510 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:34253 every 8 connection(s)
I20260709 13:04:42.335562 10511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.350298 10511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292: Bootstrap starting.
I20260709 13:04:42.356288 10511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.362205 10511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292: No bootstrap required, opened a new log
I20260709 13:04:42.365039 10511 raft_consensus.cc:359] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c599466512fc46e28facaef651433292" member_type: VOTER }
I20260709 13:04:42.365653 10511 raft_consensus.cc:385] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.365875 10511 raft_consensus.cc:740] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c599466512fc46e28facaef651433292, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.366613 10511 consensus_queue.cc:260] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c599466512fc46e28facaef651433292" member_type: VOTER }
I20260709 13:04:42.367141 10511 raft_consensus.cc:399] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:42.367497 10511 raft_consensus.cc:493] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:42.368089 10511 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.374724 10511 raft_consensus.cc:515] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c599466512fc46e28facaef651433292" member_type: VOTER }
I20260709 13:04:42.375415 10511 leader_election.cc:304] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c599466512fc46e28facaef651433292; no voters: 
I20260709 13:04:42.376837 10511 leader_election.cc:290] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:42.377231 10514 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:42.378892 10514 raft_consensus.cc:697] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [term 1 LEADER]: Becoming Leader. State: Replica: c599466512fc46e28facaef651433292, State: Running, Role: LEADER
I20260709 13:04:42.379716 10514 consensus_queue.cc:237] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c599466512fc46e28facaef651433292" member_type: VOTER }
I20260709 13:04:42.380803 10511 sys_catalog.cc:565] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:42.389520 10515 sys_catalog.cc:455] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c599466512fc46e28facaef651433292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c599466512fc46e28facaef651433292" member_type: VOTER } }
I20260709 13:04:42.389504 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c599466512fc46e28facaef651433292. Latest consensus state: current_term: 1 leader_uuid: "c599466512fc46e28facaef651433292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c599466512fc46e28facaef651433292" member_type: VOTER } }
I20260709 13:04:42.390692 10515 sys_catalog.cc:458] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:42.391083 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:42.393299 10524 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:42.402261 10524 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:42.403484   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:42.414156 10524 catalog_manager.cc:1383] Generated new cluster ID: a6c900d57ba44f15b2c26c8b087d5d79
I20260709 13:04:42.414520 10524 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:42.445279 10524 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:42.446905 10524 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:42.466946 10524 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292: Generated new TSK 0
I20260709 13:04:42.467866 10524 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:42.535920   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:42.544564 10532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.545521 10533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.547550 10535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:42.549304   431 server_base.cc:1061] running on GCE node
I20260709 13:04:42.553033   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:42.553308   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:42.553615   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602282553587 us; error 0 us; skew 500 ppm
I20260709 13:04:42.554347   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:42.557291   431 webserver.cc:533] Webserver started at http://127.0.107.193:33297/ using document root <none> and password file <none>
I20260709 13:04:42.557946   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:42.558149   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:42.558444   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:42.559928   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-0-root/instance:
uuid: "bef3584ebb2c4a2ea619936c71900230"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-l8g5"
I20260709 13:04:42.565352   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 13:04:42.569765 10540 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.570705   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:04:42.571045   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-0-root
uuid: "bef3584ebb2c4a2ea619936c71900230"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-l8g5"
I20260709 13:04:42.571410   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:42.589466   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:42.591022   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:42.592859   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:42.595567   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:42.595806   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.596056   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:42.596257   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.648237   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:44795
I20260709 13:04:42.648379 10602 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:44795 every 8 connection(s)
I20260709 13:04:42.654773   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:42.665525 10607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.667349 10608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.668949 10610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:42.670969   431 server_base.cc:1061] running on GCE node
I20260709 13:04:42.675642   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:42.675973   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:42.676203   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602282676172 us; error 0 us; skew 500 ppm
I20260709 13:04:42.676925   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:42.678828 10603 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34253
I20260709 13:04:42.679318 10603 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:42.680428 10603 heartbeater.cc:507] Master 127.0.107.254:34253 requested a full tablet report, sending...
I20260709 13:04:42.680781   431 webserver.cc:533] Webserver started at http://127.0.107.194:35783/ using document root <none> and password file <none>
I20260709 13:04:42.681862   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:42.682336   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:42.682776   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:42.683882 10476 ts_manager.cc:194] Registered new tserver with Master: bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:42.685060   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-1-root/instance:
uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-l8g5"
I20260709 13:04:42.687062 10476 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37066
I20260709 13:04:42.693327   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:42.700896 10615 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.702227   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:04:42.702656   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-1-root
uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-l8g5"
I20260709 13:04:42.702965   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:42.720563   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:42.722343   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:42.724237   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:42.726871   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:42.727138   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.727421   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:42.727619   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.784437   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:43755
I20260709 13:04:42.784569 10677 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:43755 every 8 connection(s)
I20260709 13:04:42.790311   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:42.799434 10681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.802109 10682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:42.803859   431 server_base.cc:1061] running on GCE node
I20260709 13:04:42.804989 10678 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34253
W20260709 13:04:42.805229 10684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:42.805410 10678 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:42.806457   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:42.806697   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:42.806605 10678 heartbeater.cc:507] Master 127.0.107.254:34253 requested a full tablet report, sending...
I20260709 13:04:42.807057   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602282807023 us; error 0 us; skew 500 ppm
I20260709 13:04:42.807833   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:42.809862 10476 ts_manager.cc:194] Registered new tserver with Master: 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:42.811474   431 webserver.cc:533] Webserver started at http://127.0.107.195:34507/ using document root <none> and password file <none>
I20260709 13:04:42.811815 10476 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37078
I20260709 13:04:42.812151   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:42.812458   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:42.812830   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:42.814415   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-2-root/instance:
uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-l8g5"
I20260709 13:04:42.819916   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:04:42.824193 10689 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.825162   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:04:42.825634   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-2-root
uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-l8g5"
I20260709 13:04:42.825968   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:42.843962   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:42.845706   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:42.847651   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:42.850477   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:42.850761   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.851042   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:42.851253   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.913874   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:34475
I20260709 13:04:42.913993 10751 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:34475 every 8 connection(s)
I20260709 13:04:42.920315   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:42.931381 10755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.937233 10756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:42.940704 10758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:42.940754 10752 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34253
I20260709 13:04:42.941633   431 server_base.cc:1061] running on GCE node
I20260709 13:04:42.941677 10752 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:42.942914   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:42.943253   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:42.943018 10752 heartbeater.cc:507] Master 127.0.107.254:34253 requested a full tablet report, sending...
I20260709 13:04:42.943671   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602282943631 us; error 0 us; skew 500 ppm
I20260709 13:04:42.944502   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:42.946640 10476 ts_manager.cc:194] Registered new tserver with Master: 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:42.948177   431 webserver.cc:533] Webserver started at http://127.0.107.196:35553/ using document root <none> and password file <none>
I20260709 13:04:42.948818 10476 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37090
I20260709 13:04:42.948949   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:42.949348   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:42.949733   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:42.951381   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-3-root/instance:
uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-l8g5"
I20260709 13:04:42.958019   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:04:42.962636 10763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.964313   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:04:42.964737   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-3-root
uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-l8g5"
I20260709 13:04:42.965531   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:42.982043   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:42.983714   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:42.985419   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:42.988503   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:42.988950   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:42.989300   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:42.989516   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:43.049571   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:38707
I20260709 13:04:43.049703 10825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:38707 every 8 connection(s)
I20260709 13:04:43.066530 10826 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34253
I20260709 13:04:43.067171 10826 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:43.069429 10826 heartbeater.cc:507] Master 127.0.107.254:34253 requested a full tablet report, sending...
I20260709 13:04:43.072011 10476 ts_manager.cc:194] Registered new tserver with Master: 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:43.072382   431 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.01828898s
I20260709 13:04:43.074234 10476 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37092
I20260709 13:04:43.110306 10476 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37094:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:43.395744 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:43.410328 10791 tablet_service.cc:1511] Processing CreateTablet for tablet 33e1dc0bb8f54bd2ba02310d2d9d3bdc (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:04:43.412482 10791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e1dc0bb8f54bd2ba02310d2d9d3bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.414752 10568 tablet_service.cc:1511] Processing CreateTablet for tablet 33e1dc0bb8f54bd2ba02310d2d9d3bdc (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:04:43.416908 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e1dc0bb8f54bd2ba02310d2d9d3bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.419271 10567 tablet_service.cc:1511] Processing CreateTablet for tablet 5a09eebcf668422aa3c84e1950676dcb (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:04:43.423194 10566 tablet_service.cc:1511] Processing CreateTablet for tablet b5349592564d4f4d863236517a765d59 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:04:43.414964 10790 tablet_service.cc:1511] Processing CreateTablet for tablet b5349592564d4f4d863236517a765d59 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:04:43.426091 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5349592564d4f4d863236517a765d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.425819 10643 tablet_service.cc:1511] Processing CreateTablet for tablet 33e1dc0bb8f54bd2ba02310d2d9d3bdc (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:04:43.427380 10643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33e1dc0bb8f54bd2ba02310d2d9d3bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.421380 10567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a09eebcf668422aa3c84e1950676dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.429075 10642 tablet_service.cc:1511] Processing CreateTablet for tablet 5a09eebcf668422aa3c84e1950676dcb (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:04:43.433387 10717 tablet_service.cc:1511] Processing CreateTablet for tablet 5a09eebcf668422aa3c84e1950676dcb (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:04:43.433635 10716 tablet_service.cc:1511] Processing CreateTablet for tablet b5349592564d4f4d863236517a765d59 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:04:43.434262 10715 tablet_service.cc:1511] Processing CreateTablet for tablet 756a5f26bcd6404b9b9df57d2c62ee79 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:04:43.434516 10789 tablet_service.cc:1511] Processing CreateTablet for tablet 756a5f26bcd6404b9b9df57d2c62ee79 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:04:43.435241 10717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a09eebcf668422aa3c84e1950676dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.436080 10715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756a5f26bcd6404b9b9df57d2c62ee79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.436934 10716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5349592564d4f4d863236517a765d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.436089 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756a5f26bcd6404b9b9df57d2c62ee79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.437933 10565 tablet_service.cc:1511] Processing CreateTablet for tablet 72c64556f12543bfbe8833b32eaf8366 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:04:43.438577 10641 tablet_service.cc:1511] Processing CreateTablet for tablet 756a5f26bcd6404b9b9df57d2c62ee79 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:04:43.429587 10566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5349592564d4f4d863236517a765d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.441771 10565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c64556f12543bfbe8833b32eaf8366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.444505 10642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a09eebcf668422aa3c84e1950676dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.446596 10641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756a5f26bcd6404b9b9df57d2c62ee79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.457646 10640 tablet_service.cc:1511] Processing CreateTablet for tablet 72c64556f12543bfbe8833b32eaf8366 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:04:43.459514 10640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c64556f12543bfbe8833b32eaf8366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.465991 10714 tablet_service.cc:1511] Processing CreateTablet for tablet 72c64556f12543bfbe8833b32eaf8366 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:04:43.467773 10714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c64556f12543bfbe8833b32eaf8366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.490927 10564 tablet_service.cc:1511] Processing CreateTablet for tablet e97c60086a204747ab7fc8b714daa4ea (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:04:43.491238 10788 tablet_service.cc:1511] Processing CreateTablet for tablet e97c60086a204747ab7fc8b714daa4ea (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:04:43.492890 10564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97c60086a204747ab7fc8b714daa4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.494608 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97c60086a204747ab7fc8b714daa4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.509567 10639 tablet_service.cc:1511] Processing CreateTablet for tablet e97c60086a204747ab7fc8b714daa4ea (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:04:43.512153 10639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97c60086a204747ab7fc8b714daa4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.511993 10713 tablet_service.cc:1511] Processing CreateTablet for tablet 46a5186a491b4b87b1ab0c692ba8c7f2 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:04:43.513724 10713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46a5186a491b4b87b1ab0c692ba8c7f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.537964 10787 tablet_service.cc:1511] Processing CreateTablet for tablet 46a5186a491b4b87b1ab0c692ba8c7f2 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:04:43.540628 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46a5186a491b4b87b1ab0c692ba8c7f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.562500 10568 tablet_service.cc:1511] Processing CreateTablet for tablet 46a5186a491b4b87b1ab0c692ba8c7f2 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:04:43.564808 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46a5186a491b4b87b1ab0c692ba8c7f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.594067 10638 tablet_service.cc:1511] Processing CreateTablet for tablet 6224db6d69b24c2ba435311454b28dc9 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:04:43.607066 10638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6224db6d69b24c2ba435311454b28dc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.623134 10846 tablet_bootstrap.cc:492] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:43.625275 10847 tablet_bootstrap.cc:492] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:43.636314 10716 tablet_service.cc:1511] Processing CreateTablet for tablet 6224db6d69b24c2ba435311454b28dc9 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:04:43.640610 10846 tablet_bootstrap.cc:654] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:43.648136 10716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6224db6d69b24c2ba435311454b28dc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.656497 10787 tablet_service.cc:1511] Processing CreateTablet for tablet 6224db6d69b24c2ba435311454b28dc9 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:04:43.657186 10566 tablet_service.cc:1511] Processing CreateTablet for tablet 5ec45eaa322240d2af46cc177f87d32f (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:04:43.658447 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6224db6d69b24c2ba435311454b28dc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.659248 10566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec45eaa322240d2af46cc177f87d32f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.669601 10637 tablet_service.cc:1511] Processing CreateTablet for tablet 5ec45eaa322240d2af46cc177f87d32f (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:04:43.670672 10713 tablet_service.cc:1511] Processing CreateTablet for tablet 5ec45eaa322240d2af46cc177f87d32f (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:04:43.671360 10637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec45eaa322240d2af46cc177f87d32f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.672155 10713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec45eaa322240d2af46cc177f87d32f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.677838 10788 tablet_service.cc:1511] Processing CreateTablet for tablet fee952b378da49509e7ced417a0093cc (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:04:43.679407 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee952b378da49509e7ced417a0093cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.681982 10565 tablet_service.cc:1511] Processing CreateTablet for tablet fee952b378da49509e7ced417a0093cc (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:04:43.683504 10565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee952b378da49509e7ced417a0093cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.690470 10636 tablet_service.cc:1511] Processing CreateTablet for tablet fee952b378da49509e7ced417a0093cc (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:04:43.691326 10603 heartbeater.cc:499] Master 127.0.107.254:34253 was elected leader, sending a full tablet report...
I20260709 13:04:43.692155 10636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee952b378da49509e7ced417a0093cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.696305 10846 tablet_bootstrap.cc:492] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:43.697021 10846 ts_tablet_manager.cc:1403] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.074s	user 0.007s	sys 0.008s
I20260709 13:04:43.697872 10848 tablet_bootstrap.cc:492] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:43.701215 10846 raft_consensus.cc:359] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:43.702430 10846 raft_consensus.cc:385] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:43.702826 10846 raft_consensus.cc:740] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:43.704432 10846 consensus_queue.cc:260] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:43.709707 10850 tablet_bootstrap.cc:492] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:43.711356 10715 tablet_service.cc:1511] Processing CreateTablet for tablet dbf062bca1cd416181f4c017fecb396c (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:04:43.713011 10715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf062bca1cd416181f4c017fecb396c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.713567 10848 tablet_bootstrap.cc:654] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:43.718801 10790 tablet_service.cc:1511] Processing CreateTablet for tablet dbf062bca1cd416181f4c017fecb396c (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:04:43.720441 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf062bca1cd416181f4c017fecb396c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.740813 10568 tablet_service.cc:1511] Processing CreateTablet for tablet dbf062bca1cd416181f4c017fecb396c (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:04:43.742591 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf062bca1cd416181f4c017fecb396c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.746937 10846 ts_tablet_manager.cc:1434] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.003s
I20260709 13:04:43.752913 10846 tablet_bootstrap.cc:492] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:43.766908 10850 tablet_bootstrap.cc:654] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:43.769639 10641 tablet_service.cc:1511] Processing CreateTablet for tablet 9f26ffe11b8143728804f33b884ed36a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:04:43.779072 10641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f26ffe11b8143728804f33b884ed36a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.770843 10847 tablet_bootstrap.cc:654] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:43.780462 10717 tablet_service.cc:1511] Processing CreateTablet for tablet 9f26ffe11b8143728804f33b884ed36a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:04:43.791985 10717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f26ffe11b8143728804f33b884ed36a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.791755 10568 tablet_service.cc:1511] Processing CreateTablet for tablet b02d8d78f8e0419186c6c09d9cc307d7 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:04:43.785089 10848 tablet_bootstrap.cc:492] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:43.793677 10848 ts_tablet_manager.cc:1403] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.100s	user 0.026s	sys 0.018s
I20260709 13:04:43.788820 10788 tablet_service.cc:1511] Processing CreateTablet for tablet 9f26ffe11b8143728804f33b884ed36a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:04:43.794358 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b02d8d78f8e0419186c6c09d9cc307d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.794994 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f26ffe11b8143728804f33b884ed36a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.803633 10846 tablet_bootstrap.cc:654] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:43.809736 10636 tablet_service.cc:1511] Processing CreateTablet for tablet b02d8d78f8e0419186c6c09d9cc307d7 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:04:43.811497 10636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b02d8d78f8e0419186c6c09d9cc307d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.815048 10678 heartbeater.cc:499] Master 127.0.107.254:34253 was elected leader, sending a full tablet report...
I20260709 13:04:43.820053 10713 tablet_service.cc:1511] Processing CreateTablet for tablet b02d8d78f8e0419186c6c09d9cc307d7 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:04:43.821909 10713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b02d8d78f8e0419186c6c09d9cc307d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.826416 10848 raft_consensus.cc:359] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:43.827442 10848 raft_consensus.cc:385] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:43.827764 10848 raft_consensus.cc:740] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:43.829645 10790 tablet_service.cc:1511] Processing CreateTablet for tablet 33dcc19b71a2421eab2c5d20d5affc10 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:04:43.831303 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33dcc19b71a2421eab2c5d20d5affc10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.848582 10847 tablet_bootstrap.cc:492] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:43.848704 10637 tablet_service.cc:1511] Processing CreateTablet for tablet 33dcc19b71a2421eab2c5d20d5affc10 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:04:43.849820 10847 ts_tablet_manager.cc:1403] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.225s	user 0.021s	sys 0.016s
I20260709 13:04:43.850780 10637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33dcc19b71a2421eab2c5d20d5affc10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.840564 10848 consensus_queue.cc:260] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:43.848528 10566 tablet_service.cc:1511] Processing CreateTablet for tablet 33dcc19b71a2421eab2c5d20d5affc10 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:04:43.859421 10566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33dcc19b71a2421eab2c5d20d5affc10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.866492 10847 raft_consensus.cc:359] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:43.867421 10847 raft_consensus.cc:385] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:43.867724 10847 raft_consensus.cc:740] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:43.878757 10846 tablet_bootstrap.cc:492] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:43.881008 10846 ts_tablet_manager.cc:1403] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.128s	user 0.010s	sys 0.007s
I20260709 13:04:43.884727 10846 raft_consensus.cc:359] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:43.887806 10851 raft_consensus.cc:493] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.887990 10846 raft_consensus.cc:385] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:43.872536 10715 tablet_service.cc:1511] Processing CreateTablet for tablet d6ecb24ddae546da9915163c05742a7e (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:04:43.886433 10787 tablet_service.cc:1511] Processing CreateTablet for tablet d6ecb24ddae546da9915163c05742a7e (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:04:43.888427 10851 raft_consensus.cc:515] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:43.889438 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6ecb24ddae546da9915163c05742a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.890892 10567 tablet_service.cc:1511] Processing CreateTablet for tablet d6ecb24ddae546da9915163c05742a7e (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:04:43.892364 10567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6ecb24ddae546da9915163c05742a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.892593 10851 leader_election.cc:290] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:43.887794 10850 tablet_bootstrap.cc:492] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:43.897004 10850 ts_tablet_manager.cc:1403] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.188s	user 0.030s	sys 0.014s
I20260709 13:04:43.893759 10715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6ecb24ddae546da9915163c05742a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.900743 10850 raft_consensus.cc:359] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:43.888422 10846 raft_consensus.cc:740] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:43.868563 10847 consensus_queue.cc:260] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
W20260709 13:04:43.904399 10604 tablet.cc:2406] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:04:43.905697 10826 heartbeater.cc:499] Master 127.0.107.254:34253 was elected leader, sending a full tablet report...
I20260709 13:04:43.919914 10846 consensus_queue.cc:260] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:43.918881 10850 raft_consensus.cc:385] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:43.924867 10850 raft_consensus.cc:740] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:43.926426 10850 consensus_queue.cc:260] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:43.926934 10846 ts_tablet_manager.cc:1434] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.000s
I20260709 13:04:43.929872 10846 tablet_bootstrap.cc:492] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:43.946081 10856 raft_consensus.cc:493] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.947017 10856 raft_consensus.cc:515] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:43.952210 10752 heartbeater.cc:499] Master 127.0.107.254:34253 was elected leader, sending a full tablet report...
I20260709 13:04:43.965947 10846 tablet_bootstrap.cc:654] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:43.968387 10847 ts_tablet_manager.cc:1434] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.118s	user 0.011s	sys 0.004s
I20260709 13:04:43.969885 10856 leader_election.cc:290] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:43.969412 10636 tablet_service.cc:1511] Processing CreateTablet for tablet 5b4c2675dde04ec982395518aa83bfb0 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:04:43.963397 10848 ts_tablet_manager.cc:1434] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.169s	user 0.010s	sys 0.006s
I20260709 13:04:43.983244 10847 tablet_bootstrap.cc:492] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:43.984357 10848 tablet_bootstrap.cc:492] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:43.983284 10636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b4c2675dde04ec982395518aa83bfb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:43.992579 10848 tablet_bootstrap.cc:654] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.017241 10850 ts_tablet_manager.cc:1434] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.120s	user 0.013s	sys 0.005s
I20260709 13:04:44.019069 10717 tablet_service.cc:1511] Processing CreateTablet for tablet 5b4c2675dde04ec982395518aa83bfb0 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:04:44.020718 10717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b4c2675dde04ec982395518aa83bfb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.021147 10850 tablet_bootstrap.cc:492] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:44.023810 10787 tablet_service.cc:1511] Processing CreateTablet for tablet 5b4c2675dde04ec982395518aa83bfb0 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:04:44.024139 10566 tablet_service.cc:1511] Processing CreateTablet for tablet f95de1505ade4f5389a46d7aca7d7136 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:04:44.025439 10566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95de1505ade4f5389a46d7aca7d7136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.025535 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b4c2675dde04ec982395518aa83bfb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.028231 10847 tablet_bootstrap.cc:654] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.036798 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:44.038173 10801 raft_consensus.cc:2468] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:44.040663 10544 leader_election.cc:304] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:44.042479 10851 raft_consensus.cc:2804] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:44.046187 10636 tablet_service.cc:1511] Processing CreateTablet for tablet f95de1505ade4f5389a46d7aca7d7136 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:04:44.043654 10851 raft_consensus.cc:493] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.047884 10636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95de1505ade4f5389a46d7aca7d7136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.048074 10851 raft_consensus.cc:3060] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:44.066087 10715 tablet_service.cc:1511] Processing CreateTablet for tablet f95de1505ade4f5389a46d7aca7d7136 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:04:44.067850 10715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95de1505ade4f5389a46d7aca7d7136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.070787 10790 tablet_service.cc:1511] Processing CreateTablet for tablet 6e2988fb83d740c6a6b692c5fc8d8105 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:04:44.072441 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2988fb83d740c6a6b692c5fc8d8105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.073700 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:44.074930 10727 raft_consensus.cc:2468] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:44.075588 10567 tablet_service.cc:1511] Processing CreateTablet for tablet 6e2988fb83d740c6a6b692c5fc8d8105 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:04:44.076855 10764 leader_election.cc:304] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:44.078218 10856 raft_consensus.cc:2804] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:44.078622 10856 raft_consensus.cc:493] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.078950 10856 raft_consensus.cc:3060] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:44.081573 10567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2988fb83d740c6a6b692c5fc8d8105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.082755 10637 tablet_service.cc:1511] Processing CreateTablet for tablet 6e2988fb83d740c6a6b692c5fc8d8105 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:04:44.083498 10713 tablet_service.cc:1511] Processing CreateTablet for tablet a5f96bbe7d674a5485410f0aa8ec40e6 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:04:44.084460 10637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2988fb83d740c6a6b692c5fc8d8105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.084954 10713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5f96bbe7d674a5485410f0aa8ec40e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.086287 10850 tablet_bootstrap.cc:654] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.092703 10788 tablet_service.cc:1511] Processing CreateTablet for tablet a5f96bbe7d674a5485410f0aa8ec40e6 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:04:44.097903 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5f96bbe7d674a5485410f0aa8ec40e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.100977 10565 tablet_service.cc:1511] Processing CreateTablet for tablet a5f96bbe7d674a5485410f0aa8ec40e6 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:04:44.102653 10565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5f96bbe7d674a5485410f0aa8ec40e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.110796 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:44.111641 10727 raft_consensus.cc:2468] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:44.114732 10846 tablet_bootstrap.cc:492] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:44.116756 10846 ts_tablet_manager.cc:1403] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.187s	user 0.028s	sys 0.019s
I20260709 13:04:44.120348 10641 tablet_service.cc:1511] Processing CreateTablet for tablet 12b99c17dd874ae2bd745e4ed1ba0330 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:04:44.126644 10641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b99c17dd874ae2bd745e4ed1ba0330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.128525 10716 tablet_service.cc:1511] Processing CreateTablet for tablet 12b99c17dd874ae2bd745e4ed1ba0330 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:04:44.130753 10851 raft_consensus.cc:515] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:44.133234 10851 leader_election.cc:290] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:44.134480 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:44.135214 10568 tablet_service.cc:1511] Processing CreateTablet for tablet 44e728368868473181686e35396f3ae5 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:04:44.138857 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e728368868473181686e35396f3ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.138617 10789 tablet_service.cc:1511] Processing CreateTablet for tablet 12b99c17dd874ae2bd745e4ed1ba0330 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:04:44.137200 10846 raft_consensus.cc:359] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:44.140123 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b99c17dd874ae2bd745e4ed1ba0330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.141690 10638 tablet_service.cc:1511] Processing CreateTablet for tablet 44e728368868473181686e35396f3ae5 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:04:44.153087 10791 tablet_service.cc:1511] Processing CreateTablet for tablet 9323c8a136794ad692b7dbff074333f3 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:04:44.144548 10564 tablet_service.cc:1511] Processing CreateTablet for tablet 9323c8a136794ad692b7dbff074333f3 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:04:44.153880 10786 tablet_service.cc:1511] Processing CreateTablet for tablet a99af7813c7b40da8a055799f327ac72 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:04:44.154775 10791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9323c8a136794ad692b7dbff074333f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.154917 10564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9323c8a136794ad692b7dbff074333f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.157183 10643 tablet_service.cc:1511] Processing CreateTablet for tablet 9323c8a136794ad692b7dbff074333f3 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:04:44.157210 10642 tablet_service.cc:1511] Processing CreateTablet for tablet ec71cde09fe64d089f95382f32269db5 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:04:44.153720 10638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e728368868473181686e35396f3ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.160383 10563 tablet_service.cc:1511] Processing CreateTablet for tablet a99af7813c7b40da8a055799f327ac72 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:04:44.161911 10642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec71cde09fe64d089f95382f32269db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.163654 10714 tablet_service.cc:1511] Processing CreateTablet for tablet 44e728368868473181686e35396f3ae5 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:04:44.165140 10714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e728368868473181686e35396f3ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.166793 10711 tablet_service.cc:1511] Processing CreateTablet for tablet ec71cde09fe64d089f95382f32269db5 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:04:44.168949 10561 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4d2b26b86c4f22976db2fc47a3ec1a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:04:44.163712 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:44.170399 10727 raft_consensus.cc:3060] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:44.170974 10643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9323c8a136794ad692b7dbff074333f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.172658 10846 raft_consensus.cc:385] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.173316 10846 raft_consensus.cc:740] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.182736 10639 tablet_service.cc:1511] Processing CreateTablet for tablet e2bb2563ed2a407c9c219a632057e54a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:04:44.184279 10639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2bb2563ed2a407c9c219a632057e54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.186532 10640 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4d2b26b86c4f22976db2fc47a3ec1a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:04:44.187994 10640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4d2b26b86c4f22976db2fc47a3ec1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.190842 10635 tablet_service.cc:1511] Processing CreateTablet for tablet 8cfb8e7fc7e448f8b02ff6992503bc35 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:04:44.183831 10846 consensus_queue.cc:260] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:44.192415 10635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cfb8e7fc7e448f8b02ff6992503bc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.194964 10634 tablet_service.cc:1511] Processing CreateTablet for tablet eca28e86e81447cb899fbb41f02a7f03 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:04:44.198971 10785 tablet_service.cc:1511] Processing CreateTablet for tablet ec71cde09fe64d089f95382f32269db5 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:04:44.201116 10846 ts_tablet_manager.cc:1434] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:04:44.173694 10563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a99af7813c7b40da8a055799f327ac72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.175601 10712 tablet_service.cc:1511] Processing CreateTablet for tablet a99af7813c7b40da8a055799f327ac72 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:04:44.166882 10562 tablet_service.cc:1511] Processing CreateTablet for tablet e2bb2563ed2a407c9c219a632057e54a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:04:44.176548 10786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a99af7813c7b40da8a055799f327ac72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.177068 10716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b99c17dd874ae2bd745e4ed1ba0330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.208302 10634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eca28e86e81447cb899fbb41f02a7f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.207983 10783 tablet_service.cc:1511] Processing CreateTablet for tablet a65b76e958854f86b7ccf675dfeb85db (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:04:44.201815 10784 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4d2b26b86c4f22976db2fc47a3ec1a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:04:44.212566 10851 raft_consensus.cc:493] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.213155 10851 raft_consensus.cc:515] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:44.214351 10784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4d2b26b86c4f22976db2fc47a3ec1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.215127 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec71cde09fe64d089f95382f32269db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.213282 10782 tablet_service.cc:1511] Processing CreateTablet for tablet 8cfb8e7fc7e448f8b02ff6992503bc35 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:04:44.216285 10851 leader_election.cc:290] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:44.216552 10782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cfb8e7fc7e448f8b02ff6992503bc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.212162 10711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec71cde09fe64d089f95382f32269db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.220500 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e1dc0bb8f54bd2ba02310d2d9d3bdc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:44.224715 10544 leader_election.cc:343] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.226816 10560 tablet_service.cc:1511] Processing CreateTablet for tablet a65b76e958854f86b7ccf675dfeb85db (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:04:44.227376 10846 tablet_bootstrap.cc:492] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:44.220860 10712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a99af7813c7b40da8a055799f327ac72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.229807 10559 tablet_service.cc:1511] Processing CreateTablet for tablet eca28e86e81447cb899fbb41f02a7f03 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:04:44.234289 10562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2bb2563ed2a407c9c219a632057e54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.234800 10783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a65b76e958854f86b7ccf675dfeb85db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.239979 10709 tablet_service.cc:1511] Processing CreateTablet for tablet a65b76e958854f86b7ccf675dfeb85db (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:04:44.240671 10710 tablet_service.cc:1511] Processing CreateTablet for tablet e2bb2563ed2a407c9c219a632057e54a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:04:44.241849 10709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a65b76e958854f86b7ccf675dfeb85db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.241984 10561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4d2b26b86c4f22976db2fc47a3ec1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.245966 10710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2bb2563ed2a407c9c219a632057e54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.246447 10708 tablet_service.cc:1511] Processing CreateTablet for tablet 8cfb8e7fc7e448f8b02ff6992503bc35 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:04:44.248394 10708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cfb8e7fc7e448f8b02ff6992503bc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.249821 10560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a65b76e958854f86b7ccf675dfeb85db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.253957 10559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eca28e86e81447cb899fbb41f02a7f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.331777 10851 raft_consensus.cc:493] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.332054 10863 raft_consensus.cc:493] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.332757 10851 raft_consensus.cc:515] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:44.332844 10863 raft_consensus.cc:515] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:44.206737 10856 raft_consensus.cc:515] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:44.339675 10789 tablet_service.cc:1511] Processing CreateTablet for tablet 95882d5aa57c46ba86be3160648fc85a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:04:44.342044 10801 raft_consensus.cc:2393] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bef3584ebb2c4a2ea619936c71900230 in current term 1: Already voted for candidate 5a12f1c20c674e7f8242df41a9acf5dc in this term.
I20260709 13:04:44.365509 10715 tablet_service.cc:1511] Processing CreateTablet for tablet eca28e86e81447cb899fbb41f02a7f03 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:04:44.367424 10715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eca28e86e81447cb899fbb41f02a7f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.375311 10564 tablet_service.cc:1511] Processing CreateTablet for tablet da58d0f3541543dc8678d61df75295f9 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:04:44.376540 10727 raft_consensus.cc:2468] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
I20260709 13:04:44.383049 10846 tablet_bootstrap.cc:654] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.387223 10851 leader_election.cc:290] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:44.389071 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95882d5aa57c46ba86be3160648fc85a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.391557 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:44.392727 10578 raft_consensus.cc:2393] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 1: Already voted for candidate bef3584ebb2c4a2ea619936c71900230 in this term.
W20260709 13:04:44.405272 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:44.408774 10564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da58d0f3541543dc8678d61df75295f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.428653 10636 tablet_service.cc:1511] Processing CreateTablet for tablet 95882d5aa57c46ba86be3160648fc85a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:04:44.430325 10636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95882d5aa57c46ba86be3160648fc85a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.430925 10787 tablet_service.cc:1511] Processing CreateTablet for tablet 46823d8cc8df4f4bafb078e496b9a7d2 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:04:44.432762 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46823d8cc8df4f4bafb078e496b9a7d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.444821 10634 tablet_service.cc:1511] Processing CreateTablet for tablet a48d84eceb174792919f6256ee4fbb21 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:04:44.447088 10634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48d84eceb174792919f6256ee4fbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.448652 10716 tablet_service.cc:1511] Processing CreateTablet for tablet 3bb134b566b6443f887986585ebc296c (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:04:44.454774 10637 tablet_service.cc:1511] Processing CreateTablet for tablet da58d0f3541543dc8678d61df75295f9 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:04:44.456416 10637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da58d0f3541543dc8678d61df75295f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.458346 10566 tablet_service.cc:1511] Processing CreateTablet for tablet 95882d5aa57c46ba86be3160648fc85a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:04:44.468653 10541 leader_election.cc:304] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:44.470320 10851 raft_consensus.cc:2804] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:44.473337 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a09eebcf668422aa3c84e1950676dcb" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:44.474792 10716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bb134b566b6443f887986585ebc296c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.490075 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:44.491851 10856 leader_election.cc:290] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:44.495568 10848 tablet_bootstrap.cc:492] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:44.496696 10848 ts_tablet_manager.cc:1403] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.513s	user 0.020s	sys 0.008s
W20260709 13:04:44.497272 10542 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.0.107.254:34253, user_credentials={real_user=slave}} blocked reactor thread for 72548.2us
I20260709 13:04:44.501614 10863 leader_election.cc:290] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:44.507344 10848 raft_consensus.cc:359] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:44.508314 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:44.517408 10640 tablet_service.cc:1511] Processing CreateTablet for tablet bad0f5b06cbf47c5a6ae75b972ee23a6 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:04:44.519830 10640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bad0f5b06cbf47c5a6ae75b972ee23a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.489400 10567 tablet_service.cc:1511] Processing CreateTablet for tablet 3bb134b566b6443f887986585ebc296c (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:04:44.508652 10848 raft_consensus.cc:385] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.520311 10727 raft_consensus.cc:2393] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 1: Already voted for candidate bef3584ebb2c4a2ea619936c71900230 in this term.
I20260709 13:04:44.545768 10563 tablet_service.cc:1511] Processing CreateTablet for tablet c676741447624a8080294db4e3b3cd4c (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
W20260709 13:04:44.549562 10541 leader_election.cc:343] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.476508 10717 tablet_service.cc:1511] Processing CreateTablet for tablet a48d84eceb174792919f6256ee4fbb21 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:04:44.570371 10568 tablet_service.cc:1511] Processing CreateTablet for tablet bad0f5b06cbf47c5a6ae75b972ee23a6 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:04:44.570647 10786 tablet_service.cc:1511] Processing CreateTablet for tablet bad0f5b06cbf47c5a6ae75b972ee23a6 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:04:44.572407 10786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bad0f5b06cbf47c5a6ae75b972ee23a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.579152 10711 tablet_service.cc:1511] Processing CreateTablet for tablet 46823d8cc8df4f4bafb078e496b9a7d2 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:04:44.530462 10848 raft_consensus.cc:740] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.582006 10783 tablet_service.cc:1511] Processing CreateTablet for tablet a48d84eceb174792919f6256ee4fbb21 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:04:44.582707 10848 consensus_queue.cc:260] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:44.583894 10783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48d84eceb174792919f6256ee4fbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.517994 10562 tablet_service.cc:1511] Processing CreateTablet for tablet 46823d8cc8df4f4bafb078e496b9a7d2 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:04:44.594383 10713 tablet_service.cc:1511] Processing CreateTablet for tablet da58d0f3541543dc8678d61df75295f9 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:04:44.596177 10850 tablet_bootstrap.cc:492] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:44.596855 10850 ts_tablet_manager.cc:1403] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.576s	user 0.011s	sys 0.022s
I20260709 13:04:44.557862 10566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95882d5aa57c46ba86be3160648fc85a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.585289 10848 ts_tablet_manager.cc:1434] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.088s	user 0.010s	sys 0.004s
W20260709 13:04:44.622300 10462 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 48556.3us
I20260709 13:04:44.627297 10714 tablet_service.cc:1511] Processing CreateTablet for tablet c676741447624a8080294db4e3b3cd4c (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:04:44.627786 10634 tablet_service.cc:1511] Processing CreateTablet for tablet fb17118da73149cbb340865c5c6959c5 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:04:44.629205 10634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb17118da73149cbb340865c5c6959c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.635166 10641 tablet_service.cc:1511] Processing CreateTablet for tablet 413b57e5acb54f9b86a0741e6f235ad3 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:04:44.636847 10559 tablet_service.cc:1511] Processing CreateTablet for tablet 5b14050b5d4a4fb4aa74de480c7d9cdc (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:04:44.605670 10851 raft_consensus.cc:697] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:04:44.612552 10709 tablet_service.cc:1511] Processing CreateTablet for tablet fb17118da73149cbb340865c5c6959c5 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:04:44.662144 10785 tablet_service.cc:1511] Processing CreateTablet for tablet fb17118da73149cbb340865c5c6959c5 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:04:44.664212 10790 tablet_service.cc:1511] Processing CreateTablet for tablet 3bb134b566b6443f887986585ebc296c (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:04:44.569295 10638 tablet_service.cc:1511] Processing CreateTablet for tablet c676741447624a8080294db4e3b3cd4c (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:04:44.673938 10635 tablet_service.cc:1511] Processing CreateTablet for tablet 9887fc1ac0ac404e9f590987c2f0b0be (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:04:44.676729 10561 tablet_service.cc:1511] Processing CreateTablet for tablet add2b1482b6e41b3ba139426c7da8f53 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:04:44.736694 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e1dc0bb8f54bd2ba02310d2d9d3bdc" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:44.678869 10641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413b57e5acb54f9b86a0741e6f235ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.737545 10801 raft_consensus.cc:2468] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:44.738332 10563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c676741447624a8080294db4e3b3cd4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.739223 10619 leader_election.cc:304] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:04:44.681921 10846 tablet_bootstrap.cc:492] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:44.740547 10863 raft_consensus.cc:2804] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:44.740659 10846 ts_tablet_manager.cc:1403] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.514s	user 0.032s	sys 0.012s
I20260709 13:04:44.740989 10863 raft_consensus.cc:493] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.741397 10863 raft_consensus.cc:3060] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:44.744753 10846 raft_consensus.cc:359] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:44.745997 10846 raft_consensus.cc:385] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.694700 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e1dc0bb8f54bd2ba02310d2d9d3bdc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:44.695796 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a09eebcf668422aa3c84e1950676dcb" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:44.747807 10846 raft_consensus.cc:740] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.748133 10788 tablet_service.cc:1511] Processing CreateTablet for tablet 413b57e5acb54f9b86a0741e6f235ad3 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:04:44.665015 10850 raft_consensus.cc:359] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:44.698931 10717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a48d84eceb174792919f6256ee4fbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.750080 10562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46823d8cc8df4f4bafb078e496b9a7d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.752405 10782 tablet_service.cc:1511] Processing CreateTablet for tablet be849d4cbf0b4f7a99f9935a55611658 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:04:44.750262 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413b57e5acb54f9b86a0741e6f235ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.701354 10712 tablet_service.cc:1511] Processing CreateTablet for tablet 9887fc1ac0ac404e9f590987c2f0b0be (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:04:44.701687 10710 tablet_service.cc:1511] Processing CreateTablet for tablet add2b1482b6e41b3ba139426c7da8f53 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:04:44.684501 10639 tablet_service.cc:1511] Processing CreateTablet for tablet 8a502395920c4836b0dfb455c04e108b (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:04:44.704820 10847 tablet_bootstrap.cc:492] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:44.758762 10850 raft_consensus.cc:385] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.759227 10847 ts_tablet_manager.cc:1403] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.776s	user 0.021s	sys 0.016s
I20260709 13:04:44.759512 10850 raft_consensus.cc:740] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.759996 10561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet add2b1482b6e41b3ba139426c7da8f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.753957 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb17118da73149cbb340865c5c6959c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.758337 10635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9887fc1ac0ac404e9f590987c2f0b0be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.748939 10846 consensus_queue.cc:260] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:44.764947 10846 ts_tablet_manager.cc:1434] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:04:44.718107 10848 tablet_bootstrap.cc:492] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:44.720077 10791 tablet_service.cc:1511] Processing CreateTablet for tablet 5b14050b5d4a4fb4aa74de480c7d9cdc (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:04:44.692907 10636 tablet_service.cc:1511] Processing CreateTablet for tablet be849d4cbf0b4f7a99f9935a55611658 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:04:44.801220 10846 tablet_bootstrap.cc:492] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:44.745594 10787 tablet_service.cc:1511] Processing CreateTablet for tablet add2b1482b6e41b3ba139426c7da8f53 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:04:44.759155 10709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb17118da73149cbb340865c5c6959c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.803263 10713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da58d0f3541543dc8678d61df75295f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.765709 10638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c676741447624a8080294db4e3b3cd4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.803215 10847 raft_consensus.cc:359] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:44.804816 10710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet add2b1482b6e41b3ba139426c7da8f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.806486 10712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9887fc1ac0ac404e9f590987c2f0b0be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.769970 10559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b14050b5d4a4fb4aa74de480c7d9cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.702630 10642 tablet_service.cc:1511] Processing CreateTablet for tablet 5b14050b5d4a4fb4aa74de480c7d9cdc (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:04:44.774384 10850 consensus_queue.cc:260] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:44.775880 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bb134b566b6443f887986585ebc296c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.809814 10639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a502395920c4836b0dfb455c04e108b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.809346 10789 tablet_service.cc:1511] Processing CreateTablet for tablet 69cb859af9704c2596574e50f238ce52 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:04:44.777204 10851 consensus_queue.cc:237] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:44.827652 10850 ts_tablet_manager.cc:1434] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.230s	user 0.010s	sys 0.002s
I20260709 13:04:44.779657 10863 raft_consensus.cc:515] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:04:44.781045 10544 leader_election.cc:343] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.830619 10544 leader_election.cc:304] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
W20260709 13:04:44.831763 10544 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 50773.7us
I20260709 13:04:44.833082 10870 raft_consensus.cc:2749] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:44.833302 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e1dc0bb8f54bd2ba02310d2d9d3bdc" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:44.834481 10577 raft_consensus.cc:2468] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:44.835103 10653 raft_consensus.cc:2393] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bef3584ebb2c4a2ea619936c71900230 in current term 1: Already voted for candidate 73db933c4a6848d1b2aeb56813f7bb2c in this term.
I20260709 13:04:44.837654 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e1dc0bb8f54bd2ba02310d2d9d3bdc" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:44.838336 10850 tablet_bootstrap.cc:492] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:44.838372 10801 raft_consensus.cc:3060] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:44.804214 10847 raft_consensus.cc:385] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.841867 10847 raft_consensus.cc:740] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.843325 10847 consensus_queue.cc:260] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:44.847029 10863 leader_election.cc:290] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:44.847482 10783 tablet_service.cc:1511] Processing CreateTablet for tablet 9887fc1ac0ac404e9f590987c2f0b0be (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:04:44.852581 10544 leader_election.cc:304] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:44.853695 10870 raft_consensus.cc:3060] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:44.857138 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e1dc0bb8f54bd2ba02310d2d9d3bdc" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:44.859745 10847 ts_tablet_manager.cc:1434] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.100s	user 0.007s	sys 0.002s
I20260709 13:04:44.799250 10637 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8082b4eacd414faa49143ba833d6ed (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:04:44.811070 10714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c676741447624a8080294db4e3b3cd4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.863451 10711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46823d8cc8df4f4bafb078e496b9a7d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.811337 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bad0f5b06cbf47c5a6ae75b972ee23a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.811826 10782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be849d4cbf0b4f7a99f9935a55611658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.865919 10789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69cb859af9704c2596574e50f238ce52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.815305 10643 tablet_service.cc:1511] Processing CreateTablet for tablet 6e6f6d7fae4f453994e7b12e063b6b49 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:04:44.869024 10791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b14050b5d4a4fb4aa74de480c7d9cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.849969 10565 tablet_service.cc:1511] Processing CreateTablet for tablet be849d4cbf0b4f7a99f9935a55611658 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:04:44.708963 10560 tablet_service.cc:1511] Processing CreateTablet for tablet 8a502395920c4836b0dfb455c04e108b (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:04:44.869748 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet add2b1482b6e41b3ba139426c7da8f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.865048 10567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bb134b566b6443f887986585ebc296c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.711651 10708 tablet_service.cc:1511] Processing CreateTablet for tablet 413b57e5acb54f9b86a0741e6f235ad3 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:04:44.879300 10766 leader_election.cc:304] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:44.881217 10856 raft_consensus.cc:2749] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:44.885907 10642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b14050b5d4a4fb4aa74de480c7d9cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.894292 10636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be849d4cbf0b4f7a99f9935a55611658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.902577 10643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e6f6d7fae4f453994e7b12e063b6b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.905138 10783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9887fc1ac0ac404e9f590987c2f0b0be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.921801 10560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a502395920c4836b0dfb455c04e108b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.939231 10801 raft_consensus.cc:2468] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:44.940119 10874 raft_consensus.cc:493] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.944758 10870 raft_consensus.cc:2749] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:44.945976 10637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8082b4eacd414faa49143ba833d6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.961184 10564 tablet_service.cc:1511] Processing CreateTablet for tablet 69cb859af9704c2596574e50f238ce52 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:04:44.968169 10564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69cb859af9704c2596574e50f238ce52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.968256 10619 leader_election.cc:304] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:04:44.969836 10863 raft_consensus.cc:2804] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:44.970543 10476 catalog_manager.cc:5697] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:44.901351 10848 tablet_bootstrap.cc:654] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.977417 10874 raft_consensus.cc:515] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:44.983175 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c64556f12543bfbe8833b32eaf8366" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:44.985888 10565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be849d4cbf0b4f7a99f9935a55611658. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:44.986534 10618 leader_election.cc:343] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:44.866832 10716 tablet_service.cc:1511] Processing CreateTablet for tablet 8a502395920c4836b0dfb455c04e108b (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:04:44.953362 10641 tablet_service.cc:1511] Processing CreateTablet for tablet 6d33bf08ed78416ea29aaf59733e35a2 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:04:45.009672 10850 tablet_bootstrap.cc:654] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.011300 10715 tablet_service.cc:1511] Processing CreateTablet for tablet 69cb859af9704c2596574e50f238ce52 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:04:45.000295 10846 tablet_bootstrap.cc:654] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.026965 10713 tablet_service.cc:1511] Processing CreateTablet for tablet 6e6f6d7fae4f453994e7b12e063b6b49 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:04:45.027783 10847 tablet_bootstrap.cc:492] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:45.007670 10577 raft_consensus.cc:2468] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:45.002235 10863 raft_consensus.cc:697] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:45.042323 10710 tablet_service.cc:1511] Processing CreateTablet for tablet 6d33bf08ed78416ea29aaf59733e35a2 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:04:45.040066 10863 consensus_queue.cc:237] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:45.055899 10717 tablet_service.cc:1511] Processing CreateTablet for tablet da7141649b494890b1b693fbfc917db0 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:04:45.039028 10566 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8082b4eacd414faa49143ba833d6ed (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:04:45.060415 10784 tablet_service.cc:1511] Processing CreateTablet for tablet 97eca41f7f434c02976a77643991e556 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:04:45.061324 10566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8082b4eacd414faa49143ba833d6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.066849 10874 leader_election.cc:290] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:45.040084 10713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e6f6d7fae4f453994e7b12e063b6b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.052937 10635 tablet_service.cc:1511] Processing CreateTablet for tablet 656245aa555f45a8a6ef044f0da06e46 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:04:45.072521 10641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d33bf08ed78416ea29aaf59733e35a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.997329 10782 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8082b4eacd414faa49143ba833d6ed (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:04:45.087312 10785 tablet_service.cc:1511] Processing CreateTablet for tablet a5d1771d12424af18f74abf7650fab3a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:04:45.089654 10848 tablet_bootstrap.cc:492] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:45.090315 10848 ts_tablet_manager.cc:1403] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.372s	user 0.019s	sys 0.008s
I20260709 13:04:45.092950 10568 tablet_service.cc:1511] Processing CreateTablet for tablet 6e6f6d7fae4f453994e7b12e063b6b49 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:04:45.094580 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e6f6d7fae4f453994e7b12e063b6b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.084482 10784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97eca41f7f434c02976a77643991e556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.049242 10634 tablet_service.cc:1511] Processing CreateTablet for tablet 97eca41f7f434c02976a77643991e556 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:04:45.114406 10640 tablet_service.cc:1511] Processing CreateTablet for tablet 9b930f7fe8fb4ec88f4aa71cd7094c9e (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:04:45.117573 10708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413b57e5acb54f9b86a0741e6f235ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.118340 10716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a502395920c4836b0dfb455c04e108b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.119024 10710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d33bf08ed78416ea29aaf59733e35a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.121076 10715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69cb859af9704c2596574e50f238ce52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.121397 10635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 656245aa555f45a8a6ef044f0da06e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.120898 10711 tablet_service.cc:1511] Processing CreateTablet for tablet a5d1771d12424af18f74abf7650fab3a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:04:45.121878 10717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da7141649b494890b1b693fbfc917db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.058964 10790 tablet_service.cc:1511] Processing CreateTablet for tablet 6d33bf08ed78416ea29aaf59733e35a2 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:04:45.039028 10639 tablet_service.cc:1511] Processing CreateTablet for tablet 6c617206d4bb4c0d84e4f8f6cccdbf30 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:04:45.123660 10785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d1771d12424af18f74abf7650fab3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.123750 10639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c617206d4bb4c0d84e4f8f6cccdbf30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.124589 10782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8082b4eacd414faa49143ba833d6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.106580 10709 tablet_service.cc:1511] Processing CreateTablet for tablet 656245aa555f45a8a6ef044f0da06e46 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:04:45.126470 10634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97eca41f7f434c02976a77643991e556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.109196 10559 tablet_service.cc:1511] Processing CreateTablet for tablet 6c617206d4bb4c0d84e4f8f6cccdbf30 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:04:45.127867 10559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c617206d4bb4c0d84e4f8f6cccdbf30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.130178 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d33bf08ed78416ea29aaf59733e35a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.132242 10640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b930f7fe8fb4ec88f4aa71cd7094c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.106581 10712 tablet_service.cc:1511] Processing CreateTablet for tablet 6c617206d4bb4c0d84e4f8f6cccdbf30 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:04:45.110153 10787 tablet_service.cc:1511] Processing CreateTablet for tablet da7141649b494890b1b693fbfc917db0 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:04:45.136066 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da7141649b494890b1b693fbfc917db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.111304 10847 tablet_bootstrap.cc:654] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.112555 10562 tablet_service.cc:1511] Processing CreateTablet for tablet da7141649b494890b1b693fbfc917db0 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:04:45.139708 10567 tablet_service.cc:1511] Processing CreateTablet for tablet 97eca41f7f434c02976a77643991e556 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:04:45.144227 10848 raft_consensus.cc:359] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:45.149981 10786 tablet_service.cc:1511] Processing CreateTablet for tablet 656245aa555f45a8a6ef044f0da06e46 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:04:45.147185 10562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da7141649b494890b1b693fbfc917db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.181649 10711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d1771d12424af18f74abf7650fab3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.181162 10870 raft_consensus.cc:493] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.186169 10870 raft_consensus.cc:515] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:45.158003 10848 raft_consensus.cc:385] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.198397 10709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 656245aa555f45a8a6ef044f0da06e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.190169 10848 raft_consensus.cc:740] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.183364 10642 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3d1fe775e74c4684020e6c5732ce13 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:04:45.202677 10636 tablet_service.cc:1511] Processing CreateTablet for tablet 514efa59c7814384b4f80d29eea50908 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:04:45.230297 10876 raft_consensus.cc:493] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.230952 10876 raft_consensus.cc:515] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:45.147111 10638 tablet_service.cc:1511] Processing CreateTablet for tablet 315bc88bbbfa4be190f5d7581ace8793 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:04:45.227952 10567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97eca41f7f434c02976a77643991e556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.234622 10712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c617206d4bb4c0d84e4f8f6cccdbf30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.265967 10848 consensus_queue.cc:260] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:45.266672 10563 tablet_service.cc:1511] Processing CreateTablet for tablet a5d1771d12424af18f74abf7650fab3a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:04:45.268970 10563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d1771d12424af18f74abf7650fab3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.275734 10564 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3d1fe775e74c4684020e6c5732ce13 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:04:45.244807 10876 leader_election.cc:290] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:45.248467 10636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514efa59c7814384b4f80d29eea50908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.303527 10638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 315bc88bbbfa4be190f5d7581ace8793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.251247 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e97c60086a204747ab7fc8b714daa4ea" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:45.253280 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e97c60086a204747ab7fc8b714daa4ea" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:45.241313 10786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 656245aa555f45a8a6ef044f0da06e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.290570 10846 tablet_bootstrap.cc:492] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
W20260709 13:04:45.326018 10544 leader_election.cc:343] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.326561 10846 ts_tablet_manager.cc:1403] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.526s	user 0.008s	sys 0.025s
W20260709 13:04:45.327008 10544 leader_election.cc:343] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.327562 10544 leader_election.cc:304] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:45.328656 10851 raft_consensus.cc:2749] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.296937 10788 tablet_service.cc:1511] Processing CreateTablet for tablet e4bc0b50fa5648f2bd176e361fdf2b55 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:04:45.299125 10714 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3d1fe775e74c4684020e6c5732ce13 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:04:45.301062 10791 tablet_service.cc:1511] Processing CreateTablet for tablet 9b930f7fe8fb4ec88f4aa71cd7094c9e (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:04:45.250245 10870 leader_election.cc:290] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:45.303582 10564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3d1fe775e74c4684020e6c5732ce13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.360921 10791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b930f7fe8fb4ec88f4aa71cd7094c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.379848 10783 tablet_service.cc:1511] Processing CreateTablet for tablet 315bc88bbbfa4be190f5d7581ace8793 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:04:45.390920 10847 tablet_bootstrap.cc:492] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:45.391721 10847 ts_tablet_manager.cc:1403] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.364s	user 0.021s	sys 0.031s
I20260709 13:04:45.381749 10714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3d1fe775e74c4684020e6c5732ce13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.395021 10850 tablet_bootstrap.cc:492] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:45.395805 10850 ts_tablet_manager.cc:1403] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.558s	user 0.015s	sys 0.012s
I20260709 13:04:45.385701 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c64556f12543bfbe8833b32eaf8366" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:45.386624 10565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50572 (request call id 31) took 1006 ms. Trace:
I20260709 13:04:45.397821 10727 raft_consensus.cc:2468] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:45.397746 10565 rpcz_store.cc:276] 0709 13:04:44.379933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.848725 (+468792us) service_pool.cc:224] Handling call
0709 13:04:44.968903 (+120178us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:44.969036 (+   133us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.137527 (+168491us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.385966 (+248439us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.386548 (+   582us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1792}
I20260709 13:04:45.389345 10643 tablet_service.cc:1511] Processing CreateTablet for tablet a27c6651a83844a69107fdd6f819ebde (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:04:45.391383 10870 consensus_queue.cc:1048] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:45.398494 10560 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50572 (request call id 30) took 1019 ms. Trace:
I20260709 13:04:45.427802 10560 rpcz_store.cc:276] 0709 13:04:44.379516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.707932 (+328416us) service_pool.cc:224] Handling call
0709 13:04:44.870044 (+162112us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:44.870163 (+   119us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.053661 (+183498us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.397757 (+344096us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.398408 (+   651us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.431195 10560 tablet_service.cc:1511] Processing CreateTablet for tablet e4bc0b50fa5648f2bd176e361fdf2b55 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:04:45.400956 10642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3d1fe775e74c4684020e6c5732ce13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.432560 10856 raft_consensus.cc:493] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.432972 10560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4bc0b50fa5648f2bd176e361fdf2b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.433200 10856 raft_consensus.cc:515] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:45.435528 10475 catalog_manager.cc:5697] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:45.440912 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
W20260709 13:04:45.443576 10764 leader_election.cc:343] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.387326 10848 ts_tablet_manager.cc:1434] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.297s	user 0.012s	sys 0.001s
I20260709 13:04:45.380585 10561 tablet_service.cc:1511] Processing CreateTablet for tablet 9b930f7fe8fb4ec88f4aa71cd7094c9e (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
W20260709 13:04:45.406639 10463 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 61428.1us
W20260709 13:04:45.426651 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:45.451556 10848 tablet_bootstrap.cc:492] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:45.426357 10851 consensus_queue.cc:1048] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:45.452086 10565 tablet_service.cc:1511] Processing CreateTablet for tablet 514efa59c7814384b4f80d29eea50908 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:04:45.447580 10643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a27c6651a83844a69107fdd6f819ebde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.426927 10847 raft_consensus.cc:359] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:45.454907 10847 raft_consensus.cc:385] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.455293 10847 raft_consensus.cc:740] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.456230 10847 consensus_queue.cc:260] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:45.441604 10856 leader_election.cc:290] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:45.401249 10783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 315bc88bbbfa4be190f5d7581ace8793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.478317 10565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514efa59c7814384b4f80d29eea50908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.513676 10561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b930f7fe8fb4ec88f4aa71cd7094c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.520138 10787 tablet_service.cc:1511] Processing CreateTablet for tablet 31e856ce94cd487395746d76e17c555a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:04:45.503960 10788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4bc0b50fa5648f2bd176e361fdf2b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.527951 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c64556f12543bfbe8833b32eaf8366" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:45.533313 10708 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47962 (request call id 29) took 1066 ms. Trace:
I20260709 13:04:45.538982 10847 ts_tablet_manager.cc:1434] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.140s	user 0.006s	sys 0.002s
I20260709 13:04:45.545624 10616 leader_election.cc:304] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:45.546291 10847 tablet_bootstrap.cc:492] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:45.542752 10717 tablet_service.cc:1511] Processing CreateTablet for tablet e4bc0b50fa5648f2bd176e361fdf2b55 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:04:45.492024 10846 raft_consensus.cc:359] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:45.553284 10863 raft_consensus.cc:2804] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.570613 10863 raft_consensus.cc:493] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.571084 10863 raft_consensus.cc:3060] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.571780 10846 raft_consensus.cc:385] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:45.557888 10616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 109811us
I20260709 13:04:45.572222 10846 raft_consensus.cc:740] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.497092 10790 tablet_service.cc:1511] Processing CreateTablet for tablet a27c6651a83844a69107fdd6f819ebde (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:04:45.557055 10874 consensus_queue.cc:1048] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:45.574447 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c64556f12543bfbe8833b32eaf8366" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:45.570134 10787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e856ce94cd487395746d76e17c555a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.598140 10874 consensus_queue.cc:1048] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:45.607589 10848 tablet_bootstrap.cc:654] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.608884 10876 raft_consensus.cc:493] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.609532 10876 raft_consensus.cc:515] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:45.610937 10790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a27c6651a83844a69107fdd6f819ebde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.613602 10876 leader_election.cc:290] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:45.591038 10846 consensus_queue.cc:260] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:45.614811 10578 raft_consensus.cc:2468] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:45.615276 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:45.616102 10714 tablet_service.cc:1511] Processing CreateTablet for tablet 315bc88bbbfa4be190f5d7581ace8793 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
W20260709 13:04:45.617538 10692 leader_election.cc:343] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.618731 10713 tablet_service.cc:1511] Processing CreateTablet for tablet 514efa59c7814384b4f80d29eea50908 (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:04:45.582209 10789 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34766 (request call id 31) took 1120 ms. Trace:
I20260709 13:04:45.572043 10850 raft_consensus.cc:359] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:45.593252 10717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4bc0b50fa5648f2bd176e361fdf2b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.633499 10692 leader_election.cc:304] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:45.634593 10876 raft_consensus.cc:2804] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.635039 10876 raft_consensus.cc:493] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.635394 10876 raft_consensus.cc:3060] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.644030 10876 raft_consensus.cc:515] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:45.647440 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c64556f12543bfbe8833b32eaf8366" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:45.648216 10576 raft_consensus.cc:3060] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.628841 10850 raft_consensus.cc:385] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.649909 10850 raft_consensus.cc:740] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.650902 10850 consensus_queue.cc:260] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:45.637091 10566 tablet_service.cc:1511] Processing CreateTablet for tablet 31e856ce94cd487395746d76e17c555a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:04:45.549930 10708 rpcz_store.cc:276] 0709 13:04:44.466480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.710714 (+244234us) service_pool.cc:224] Handling call
0709 13:04:45.043788 (+333074us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.045420 (+  1632us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.384960 (+339540us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.532738 (+147778us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.533239 (+   501us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.696668 10713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514efa59c7814384b4f80d29eea50908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.697283 10874 raft_consensus.cc:493] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.624912 10789 rpcz_store.cc:276] 0709 13:04:44.462063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.808255 (+346192us) service_pool.cc:224] Handling call
0709 13:04:44.863960 (+ 55705us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:44.864064 (+   104us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.196571 (+332507us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.569726 (+373155us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.582123 (+ 12397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.736232 10846 ts_tablet_manager.cc:1434] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.409s	user 0.013s	sys 0.000s
I20260709 13:04:45.738099 10846 tablet_bootstrap.cc:492] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:45.741703 10564 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50572 (request call id 39) took 1005 ms. Trace:
I20260709 13:04:45.723843 10876 leader_election.cc:290] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:45.742616 10714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 315bc88bbbfa4be190f5d7581ace8793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.694638 10568 tablet_service.cc:1511] Processing CreateTablet for tablet a27c6651a83844a69107fdd6f819ebde (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:04:45.732112 10566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e856ce94cd487395746d76e17c555a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.729131 10711 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47962 (request call id 37) took 1008 ms. Trace:
W20260709 13:04:45.753985 10716 consensus_meta.cc:287] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.010s
I20260709 13:04:45.760200 10716 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47962 (request call id 30) took 1293 ms. Trace:
I20260709 13:04:45.753985 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c64556f12543bfbe8833b32eaf8366" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:45.761652 10562 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50572 (request call id 35) took 1063 ms. Trace:
I20260709 13:04:45.762141 10562 rpcz_store.cc:276] 0709 13:04:44.693994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.096755 (+402761us) service_pool.cc:224] Handling call
0709 13:04:45.146558 (+ 49803us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.146675 (+   117us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.474512 (+327837us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.730604 (+256092us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.757773 (+ 27169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.751364 10847 tablet_bootstrap.cc:654] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.722298 10874 raft_consensus.cc:515] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:45.770028 10785 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34766 (request call id 37) took 1046 ms. Trace:
I20260709 13:04:45.770635 10785 rpcz_store.cc:276] 0709 13:04:44.723438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.085973 (+362535us) service_pool.cc:224] Handling call
0709 13:04:45.088413 (+  2440us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.088537 (+   124us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.486886 (+398349us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.735580 (+248694us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.769936 (+ 34356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.772409 10874 leader_election.cc:290] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:45.774461 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a27c6651a83844a69107fdd6f819ebde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.775466 10715 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47962 (request call id 31) took 1308 ms. Trace:
I20260709 13:04:45.775964 10715 rpcz_store.cc:276] 0709 13:04:44.467408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.010195 (+542787us) service_pool.cc:224] Handling call
0709 13:04:45.043274 (+ 33079us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.043398 (+   124us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.538532 (+495134us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.763469 (+224937us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.775390 (+ 11921us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.760653 10716 rpcz_store.cc:276] 0709 13:04:44.466914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.863344 (+396430us) service_pool.cc:224] Handling call
0709 13:04:45.115967 (+252623us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b580081a898. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c70e831 00007fda2c6ebac3 00007fda2c69a400 00007fda25b1faf4 00007fda25b1fa82 00007fda25b1f9b1
0709 13:04:45.116153 (+   186us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.116240 (+    87us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.251444 (+135204us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.759495 (+508051us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.760123 (+   628us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42055040}
I20260709 13:04:45.774329 10559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50572 (request call id 36) took 1064 ms. Trace:
I20260709 13:04:45.742267 10564 rpcz_store.cc:276] 0709 13:04:44.736417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.273690 (+537273us) service_pool.cc:224] Handling call
0709 13:04:45.302873 (+ 29183us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.303018 (+   145us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.489098 (+186080us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.709587 (+220489us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.741609 (+ 32022us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.749563 10576 raft_consensus.cc:2468] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:45.762152 10710 tablet_service.cc:1511] Processing CreateTablet for tablet 31e856ce94cd487395746d76e17c555a (DEFAULT_TABLE table=test-workload [id=0c9e1309f69b46298ca3e5a3051a4e44]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:04:45.797120 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:45.798102 10710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e856ce94cd487395746d76e17c555a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.799731 10642 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48274 (request call id 39) took 1018 ms. Trace:
I20260709 13:04:45.800184 10642 rpcz_store.cc:276] 0709 13:04:44.781245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.117881 (+336636us) service_pool.cc:224] Handling call
0709 13:04:45.246547 (+128666us) spinlock_profiling.cc:243] Waited 7.44 ms on lock 0x7b5800003998. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b61a2 00007fda2d4f7bb9 00007fda2c628d88 00007fda2c6ebb80 00007fda2c69a400 00007fda25b1faf4 00007fda25b1fa82 00007fda25b1f9b1 00007fda25b1f92c
0709 13:04:45.252351 (+  5804us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.252492 (+   141us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.544815 (+292323us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.799119 (+254304us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.799656 (+   537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17123072}
I20260709 13:04:45.755769 10711 rpcz_store.cc:276] 0709 13:04:44.721025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.119781 (+398756us) service_pool.cc:224] Handling call
0709 13:04:45.140529 (+ 20748us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.152285 (+ 11756us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.429076 (+276791us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.682201 (+253125us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.726552 (+ 44351us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:45.802039 10767 leader_election.cc:343] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.785260 10559 rpcz_store.cc:276] 0709 13:04:44.710365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.108104 (+397739us) service_pool.cc:224] Handling call
0709 13:04:45.127358 (+ 19254us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.127469 (+   111us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.558446 (+430977us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.763603 (+205157us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.774255 (+ 10652us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.802769 10767 leader_election.cc:304] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:45.781831 10709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47962 (request call id 38) took 1036 ms. Trace:
I20260709 13:04:45.803840 10892 raft_consensus.cc:2749] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.804131 10709 rpcz_store.cc:276] 0709 13:04:44.730738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.100386 (+369648us) service_pool.cc:224] Handling call
0709 13:04:45.161792 (+ 61406us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.174116 (+ 12324us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.539551 (+365435us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.754336 (+214785us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.766991 (+ 12655us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.805866 10638 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48274 (request call id 41) took 1023 ms. Trace:
I20260709 13:04:45.806362 10638 rpcz_store.cc:276] 0709 13:04:44.782000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.145917 (+363917us) service_pool.cc:224] Handling call
0709 13:04:45.253010 (+107093us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.253830 (+   820us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.579577 (+325747us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.805224 (+225647us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.805801 (+   577us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.812994 10783 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34766 (request call id 41) took 1048 ms. Trace:
I20260709 13:04:45.814306 10863 raft_consensus.cc:515] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:45.817385 10653 raft_consensus.cc:2393] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7ecd836bb7a144dc9aa07a046239d6ce in current term 1: Already voted for candidate 73db933c4a6848d1b2aeb56813f7bb2c in this term.
I20260709 13:04:45.819530 10692 leader_election.cc:304] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:45.822417 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c64556f12543bfbe8833b32eaf8366" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:45.822867 10712 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47962 (request call id 36) took 1107 ms. Trace:
I20260709 13:04:45.823416 10726 raft_consensus.cc:2393] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 7ecd836bb7a144dc9aa07a046239d6ce in this term.
I20260709 13:04:45.824236 10863 leader_election.cc:290] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:45.824580 10876 raft_consensus.cc:2804] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.829754 10784 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34766 (request call id 36) took 1112 ms. Trace:
I20260709 13:04:45.830366 10784 rpcz_store.cc:276] 0709 13:04:44.716885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.059412 (+342527us) service_pool.cc:224] Handling call
0709 13:04:45.061356 (+  1944us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.061459 (+   103us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.675850 (+614391us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.829137 (+153287us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.829684 (+   547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.843665 10643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48274 (request call id 43) took 1057 ms. Trace:
I20260709 13:04:45.848327 10643 rpcz_store.cc:276] 0709 13:04:44.786201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.387974 (+601773us) service_pool.cc:224] Handling call
0709 13:04:45.426186 (+ 38212us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.426306 (+   120us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.612767 (+186461us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.796215 (+183448us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.843574 (+ 47359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.823426 10712 rpcz_store.cc:276] 0709 13:04:44.715524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.069379 (+353855us) service_pool.cc:224] Handling call
0709 13:04:45.159505 (+ 90126us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b580081a898. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5e9c 00007fda245b5a5b 00007fda2d4cec09 00007fda2c628ee0 00007fda2c70e831 00007fda2c6ebac3 00007fda2c69a400 00007fda25b1faf4 00007fda25b1fa82 00007fda25b1f9b1
0709 13:04:45.171523 (+ 12018us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.189838 (+ 18315us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.576477 (+386639us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.822075 (+245598us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.822785 (+   710us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8657664}
I20260709 13:04:45.902165 10568 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50572 (request call id 43) took 1125 ms. Trace:
I20260709 13:04:45.902268 10787 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34766 (request call id 43) took 1117 ms. Trace:
I20260709 13:04:45.813920 10783 rpcz_store.cc:276] 0709 13:04:44.764961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.378623 (+613662us) service_pool.cc:224] Handling call
0709 13:04:45.390924 (+ 12301us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.391054 (+   130us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.743333 (+352279us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.812409 (+ 69076us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.812920 (+   511us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:45.889066 10618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 126179us
I20260709 13:04:45.903056 10787 rpcz_store.cc:276] 0709 13:04:44.784585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.519075 (+734490us) service_pool.cc:224] Handling call
0709 13:04:45.552493 (+ 33418us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.569640 (+ 17147us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.838329 (+268689us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.900676 (+ 62347us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.902190 (+  1514us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.902920 10568 rpcz_store.cc:276] 0709 13:04:44.777185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.575637 (+798452us) service_pool.cc:224] Handling call
0709 13:04:45.751583 (+175946us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.751703 (+   120us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.849046 (+ 97343us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.895604 (+ 46558us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.902098 (+  6494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.948834 10847 tablet_bootstrap.cc:492] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:45.949704 10847 ts_tablet_manager.cc:1403] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.404s	user 0.014s	sys 0.009s
I20260709 13:04:45.904006 10876 raft_consensus.cc:697] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:45.955345 10848 tablet_bootstrap.cc:492] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:45.956122 10848 ts_tablet_manager.cc:1403] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.505s	user 0.022s	sys 0.022s
I20260709 13:04:45.956398 10846 tablet_bootstrap.cc:654] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.928865 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e97c60086a204747ab7fc8b714daa4ea" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:45.968873 10876 consensus_queue.cc:237] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:45.970991 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e97c60086a204747ab7fc8b714daa4ea" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:45.972060 10576 raft_consensus.cc:2468] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:45.973670 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c64556f12543bfbe8833b32eaf8366" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:45.975018 10577 raft_consensus.cc:2393] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 7ecd836bb7a144dc9aa07a046239d6ce in this term.
I20260709 13:04:45.981808 10560 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50572 (request call id 41) took 1225 ms. Trace:
I20260709 13:04:45.982543 10560 rpcz_store.cc:276] 0709 13:04:44.756142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.430132 (+673990us) service_pool.cc:224] Handling call
0709 13:04:45.432098 (+  1966us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.432504 (+   406us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.777582 (+345078us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.980927 (+203345us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.981723 (+   796us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.985440 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:45.990829 10618 leader_election.cc:304] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:45.992059 10563 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50572 (request call id 38) took 1266 ms. Trace:
I20260709 13:04:45.992321 10618 leader_election.cc:304] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:45.993808 10874 raft_consensus.cc:2804] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.994223 10874 raft_consensus.cc:493] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.994556 10874 raft_consensus.cc:3060] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.996555 10790 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34766 (request call id 42) took 1222 ms. Trace:
W20260709 13:04:45.989306 10693 leader_election.cc:343] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.998016 10693 leader_election.cc:304] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:45.999588 10905 raft_consensus.cc:2749] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.999989 10565 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50572 (request call id 42) took 1232 ms. Trace:
I20260709 13:04:46.000416 10561 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50572 (request call id 40) took 1255 ms. Trace:
I20260709 13:04:46.000495 10565 rpcz_store.cc:276] 0709 13:04:44.767439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.450968 (+683529us) service_pool.cc:224] Handling call
0709 13:04:45.453285 (+  2317us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.453398 (+   113us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.834587 (+381189us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.999365 (+164778us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.999916 (+   551us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:45.953497 10847 raft_consensus.cc:359] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.004117 10850 ts_tablet_manager.cc:1434] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.608s	user 0.020s	sys 0.006s
I20260709 13:04:46.011668 10847 raft_consensus.cc:385] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:46.011972 10850 ts_tablet_manager.cc:1467] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c: Tablet startup took 615ms
I20260709 13:04:46.012015 10847 raft_consensus.cc:740] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.002728 10863 raft_consensus.cc:2749] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:46.013049 10847 consensus_queue.cc:260] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:45.992609 10563 rpcz_store.cc:276] 0709 13:04:44.725869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.173975 (+448106us) service_pool.cc:224] Handling call
0709 13:04:45.268238 (+ 94263us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.268387 (+   149us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.727134 (+458747us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.991356 (+264222us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.991985 (+   629us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.006770 10848 raft_consensus.cc:359] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:04:46.008162 10619 leader_election.cc:343] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.015930 10848 raft_consensus.cc:385] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.001070 10561 rpcz_store.cc:276] 0709 13:04:44.745601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.379623 (+634022us) service_pool.cc:224] Handling call
0709 13:04:45.470436 (+ 90813us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.491525 (+ 21089us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.807197 (+315672us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.996935 (+189738us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.997459 (+   524us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.010754 10874 raft_consensus.cc:515] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:45.997038 10790 rpcz_store.cc:276] 0709 13:04:44.774610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.476310 (+701700us) service_pool.cc:224] Handling call
0709 13:04:45.586459 (+110149us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.586767 (+   308us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.813245 (+226478us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:45.995964 (+182719us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:45.996487 (+   523us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.019670 10788 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34766 (request call id 40) took 1265 ms. Trace:
I20260709 13:04:46.016321 10848 raft_consensus.cc:740] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.020159 10788 rpcz_store.cc:276] 0709 13:04:44.753817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.292692 (+538875us) service_pool.cc:224] Handling call
0709 13:04:45.402348 (+109656us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b5800604e98. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda24475db1 00007fda245b5b71 00007fda245b5a22 00007fda2d4cec09 00007fda2c628ee0 00007fda2c70e831 00007fda2c6ebac3 00007fda2c69a400 00007fda25b1faf4 00007fda25b1fa82 00007fda25b1f9b1
0709 13:04:45.402570 (+   222us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.402722 (+   152us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.839700 (+436978us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:46.018993 (+179293us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:46.019590 (+   597us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90264448}
I20260709 13:04:46.022130 10847 ts_tablet_manager.cc:1434] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.072s	user 0.009s	sys 0.000s
I20260709 13:04:46.021323 10848 consensus_queue.cc:260] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.023639 10847 tablet_bootstrap.cc:492] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:46.027304 10848 ts_tablet_manager.cc:1434] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.071s	user 0.009s	sys 0.000s
I20260709 13:04:46.029269 10874 leader_election.cc:290] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:46.030211 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e97c60086a204747ab7fc8b714daa4ea" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:46.030663 10848 tablet_bootstrap.cc:492] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:46.031077 10800 raft_consensus.cc:3060] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.031490 10847 tablet_bootstrap.cc:654] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:46.012485 10850 ts_tablet_manager.cc:1469] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c: Trace:
0709 13:04:43.507260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.507500 (+   240us) service_pool.cc:224] Handling call
0709 13:04:43.511091 (+  3591us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:43.511247 (+   156us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:43.650813 (+139566us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:43.703969 (+ 53156us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:43.704574 (+   605us) inbound_call.cc:177] Queueing success response
0709 13:04:44.837918 (+1133344us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:04:45.396171 (+558253us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:04:45.396779 (+   608us) tablet_replica.cc:261] Starting instrumentation
0709 13:04:45.449351 (+ 52572us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":3158,"raft.queue_time_us":93504,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"tablet-open.queue_time_us":1133472,"thread_start_us":29455,"threads_started":1}
I20260709 13:04:46.039431 10850 tablet_bootstrap.cc:492] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:46.043490 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e97c60086a204747ab7fc8b714daa4ea" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:46.044353 10576 raft_consensus.cc:3060] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.058363 10713 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47962 (request call id 42) took 1286 ms. Trace:
I20260709 13:04:46.059520 10576 raft_consensus.cc:2468] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:46.061120 10618 leader_election.cc:304] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:46.059095 10713 rpcz_store.cc:276] 0709 13:04:44.772030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.617442 (+845412us) service_pool.cc:224] Handling call
0709 13:04:45.619687 (+  2245us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.619811 (+   124us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.838300 (+218489us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:46.057752 (+219452us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:46.058287 (+   535us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.062063 10863 raft_consensus.cc:2804] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.074142 10863 raft_consensus.cc:697] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:46.075522 10863 consensus_queue.cc:237] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.079550 10714 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47962 (request call id 41) took 1317 ms. Trace:
I20260709 13:04:46.080261 10714 rpcz_store.cc:276] 0709 13:04:44.761840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.615021 (+853181us) service_pool.cc:224] Handling call
0709 13:04:45.617044 (+  2023us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.617159 (+   115us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.990653 (+373494us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:46.072099 (+ 81446us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:46.079454 (+  7355us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.091761 10846 tablet_bootstrap.cc:492] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:46.114892 10850 tablet_bootstrap.cc:654] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.108912 10846 ts_tablet_manager.cc:1403] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.371s	user 0.005s	sys 0.023s
I20260709 13:04:46.109078 10710 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47962 (request call id 43) took 1326 ms. Trace:
I20260709 13:04:46.119369 10566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50572 (request call id 44) took 1332 ms. Trace:
I20260709 13:04:46.120083 10566 rpcz_store.cc:276] 0709 13:04:44.787353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.634911 (+847558us) service_pool.cc:224] Handling call
0709 13:04:45.698369 (+ 63458us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.706635 (+  8266us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:46.078216 (+371581us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:46.118653 (+ 40437us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:46.119283 (+   630us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.122720 10717 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47962 (request call id 40) took 1372 ms. Trace:
I20260709 13:04:46.124423 10717 rpcz_store.cc:276] 0709 13:04:44.750143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.541223 (+791080us) service_pool.cc:224] Handling call
0709 13:04:45.549506 (+  8283us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.549625 (+   119us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:46.021279 (+471654us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:46.089358 (+ 68079us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:46.117882 (+ 28524us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.129314 10800 raft_consensus.cc:2468] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:46.131764 10846 raft_consensus.cc:359] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.132718 10846 raft_consensus.cc:385] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.129963 10473 catalog_manager.cc:5697] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:46.133039 10846 raft_consensus.cc:740] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.133437 10847 tablet_bootstrap.cc:492] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:46.130669 10848 tablet_bootstrap.cc:654] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.134135 10847 ts_tablet_manager.cc:1403] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.111s	user 0.022s	sys 0.062s
I20260709 13:04:46.119481 10710 rpcz_store.cc:276] 0709 13:04:44.781868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.708117 (+926249us) service_pool.cc:224] Handling call
0709 13:04:45.797508 (+ 89391us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:04:45.797643 (+   135us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:04:45.958201 (+160558us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:46.094923 (+136722us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:46.108593 (+ 13670us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.133975 10846 consensus_queue.cc:260] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.138512 10846 ts_tablet_manager.cc:1434] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:04:46.141988 10847 raft_consensus.cc:359] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.143158 10847 raft_consensus.cc:385] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.143517 10847 raft_consensus.cc:740] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.144577 10847 consensus_queue.cc:260] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.146178 10846 tablet_bootstrap.cc:492] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:46.148140 10847 ts_tablet_manager.cc:1434] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:04:46.149567 10847 tablet_bootstrap.cc:492] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:46.156421 10850 tablet_bootstrap.cc:492] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:46.144236 10475 catalog_manager.cc:5697] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:46.165565 10850 ts_tablet_manager.cc:1403] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.126s	user 0.021s	sys 0.007s
I20260709 13:04:46.160214 10847 tablet_bootstrap.cc:654] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.161441 10846 tablet_bootstrap.cc:654] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.173431 10850 raft_consensus.cc:359] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.174880 10850 raft_consensus.cc:385] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.175211 10850 raft_consensus.cc:740] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.176230 10850 consensus_queue.cc:260] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.178113 10846 tablet_bootstrap.cc:492] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:46.179445 10846 ts_tablet_manager.cc:1403] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.003s
I20260709 13:04:46.179876 10850 ts_tablet_manager.cc:1434] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.005s
I20260709 13:04:46.181186 10850 tablet_bootstrap.cc:492] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:46.181641 10848 tablet_bootstrap.cc:492] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:46.182461 10848 ts_tablet_manager.cc:1403] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.152s	user 0.024s	sys 0.028s
I20260709 13:04:46.185036 10846 raft_consensus.cc:359] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.186266 10846 raft_consensus.cc:385] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.187773 10846 raft_consensus.cc:740] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.188407 10848 raft_consensus.cc:359] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.189319 10848 raft_consensus.cc:385] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.189710 10848 raft_consensus.cc:740] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.189755 10846 consensus_queue.cc:260] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.192026 10876 raft_consensus.cc:493] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.192461 10863 raft_consensus.cc:493] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.192775 10876 raft_consensus.cc:515] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.193148 10863 raft_consensus.cc:515] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.194491 10846 ts_tablet_manager.cc:1434] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:04:46.196013 10846 tablet_bootstrap.cc:492] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:46.197192 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a09eebcf668422aa3c84e1950676dcb" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:46.198549 10576 raft_consensus.cc:2468] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:46.202023 10863 leader_election.cc:290] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:46.202271 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a09eebcf668422aa3c84e1950676dcb" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:46.202801 10692 leader_election.cc:304] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:46.197960 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a09eebcf668422aa3c84e1950676dcb" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:46.204632 10850 tablet_bootstrap.cc:654] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.204545 10577 raft_consensus.cc:2468] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:46.203161 10727 raft_consensus.cc:2468] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:46.207427 10618 leader_election.cc:304] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:46.209229 10905 raft_consensus.cc:2804] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.209722 10863 raft_consensus.cc:2804] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.213292 10905 raft_consensus.cc:493] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.214448 10905 raft_consensus.cc:3060] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.215963 10847 tablet_bootstrap.cc:492] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:46.216599 10847 ts_tablet_manager.cc:1403] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.067s	user 0.022s	sys 0.001s
I20260709 13:04:46.218372 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a09eebcf668422aa3c84e1950676dcb" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:46.211393 10863 raft_consensus.cc:493] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.219650 10863 raft_consensus.cc:3060] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.209434 10876 leader_election.cc:290] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:46.224942 10848 consensus_queue.cc:260] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.225998 10905 raft_consensus.cc:515] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.230852 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a09eebcf668422aa3c84e1950676dcb" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:46.231604 10577 raft_consensus.cc:3060] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.232028 10847 raft_consensus.cc:359] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.233168 10847 raft_consensus.cc:385] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.233466 10847 raft_consensus.cc:740] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.250589 10905 leader_election.cc:290] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:46.250864 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a09eebcf668422aa3c84e1950676dcb" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:46.242959 10577 raft_consensus.cc:2468] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:46.252619 10692 leader_election.cc:304] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:46.254686 10905 raft_consensus.cc:2804] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.255311 10846 tablet_bootstrap.cc:654] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.256637 10905 raft_consensus.cc:697] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:46.258147 10905 consensus_queue.cc:237] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.245512 10847 consensus_queue.cc:260] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.262557 10847 ts_tablet_manager.cc:1434] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.045s	user 0.013s	sys 0.000s
I20260709 13:04:46.263835 10847 tablet_bootstrap.cc:492] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:46.266261 10863 raft_consensus.cc:515] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.267838 10653 raft_consensus.cc:2393] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7ecd836bb7a144dc9aa07a046239d6ce in current term 1: Already voted for candidate 73db933c4a6848d1b2aeb56813f7bb2c in this term.
I20260709 13:04:46.268440 10850 tablet_bootstrap.cc:492] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:46.269191 10850 ts_tablet_manager.cc:1403] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.088s	user 0.014s	sys 0.005s
I20260709 13:04:46.272672 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a09eebcf668422aa3c84e1950676dcb" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:46.274631 10848 ts_tablet_manager.cc:1434] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.092s	user 0.012s	sys 0.000s
I20260709 13:04:46.274973 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a09eebcf668422aa3c84e1950676dcb" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:46.275892 10577 raft_consensus.cc:2393] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 7ecd836bb7a144dc9aa07a046239d6ce in this term.
I20260709 13:04:46.276432 10850 raft_consensus.cc:359] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.277235 10850 raft_consensus.cc:385] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.277248 10618 leader_election.cc:304] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:46.281260 10850 raft_consensus.cc:740] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.282060 10874 raft_consensus.cc:2749] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:46.279193 10846 tablet_bootstrap.cc:492] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:46.283533 10846 ts_tablet_manager.cc:1403] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.088s	user 0.016s	sys 0.005s
I20260709 13:04:46.273281 10863 leader_election.cc:290] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:46.282307 10850 consensus_queue.cc:260] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.287650 10850 ts_tablet_manager.cc:1434] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:04:46.289168 10850 tablet_bootstrap.cc:492] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:46.291142 10847 tablet_bootstrap.cc:654] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.297096 10846 raft_consensus.cc:359] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.298338 10846 raft_consensus.cc:385] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.298954 10846 raft_consensus.cc:740] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.300283 10846 consensus_queue.cc:260] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.309053 10846 ts_tablet_manager.cc:1434] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.003s
I20260709 13:04:46.312569 10850 tablet_bootstrap.cc:654] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.313454 10846 tablet_bootstrap.cc:492] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:46.317204 10476 catalog_manager.cc:5697] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.320801 10847 tablet_bootstrap.cc:492] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:46.324681 10846 tablet_bootstrap.cc:654] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.326406 10847 ts_tablet_manager.cc:1403] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.005s
I20260709 13:04:46.328892 10850 tablet_bootstrap.cc:492] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:46.329902 10850 ts_tablet_manager.cc:1403] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.008s
I20260709 13:04:46.330730 10847 raft_consensus.cc:359] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.331882 10847 raft_consensus.cc:385] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.332355 10847 raft_consensus.cc:740] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.333945 10850 raft_consensus.cc:359] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.334811 10850 raft_consensus.cc:385] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.335127 10850 raft_consensus.cc:740] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.275935 10848 tablet_bootstrap.cc:492] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:46.335969 10850 consensus_queue.cc:260] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.345674 10847 consensus_queue.cc:260] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.346432 10905 raft_consensus.cc:493] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.346927 10850 ts_tablet_manager.cc:1434] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.005s
I20260709 13:04:46.346975 10905 raft_consensus.cc:515] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.348973 10847 ts_tablet_manager.cc:1434] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.003s
I20260709 13:04:46.352530 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec45eaa322240d2af46cc177f87d32f" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:46.353411 10577 raft_consensus.cc:2468] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:46.353921 10847 tablet_bootstrap.cc:492] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:46.355487 10692 leader_election.cc:304] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:46.352909 10848 tablet_bootstrap.cc:654] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.359408 10850 tablet_bootstrap.cc:492] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:46.371053 10846 tablet_bootstrap.cc:492] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:46.356469 10876 raft_consensus.cc:2804] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.371961 10846 ts_tablet_manager.cc:1403] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.059s	user 0.018s	sys 0.006s
I20260709 13:04:46.371801 10876 raft_consensus.cc:493] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.365065 10906 raft_consensus.cc:493] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.358331 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec45eaa322240d2af46cc177f87d32f" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:46.373330 10906 raft_consensus.cc:515] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.357139 10905 leader_election.cc:290] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:46.375110 10905 raft_consensus.cc:493] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.375617 10905 raft_consensus.cc:515] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.376562 10906 leader_election.cc:290] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:46.379027 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:46.379572 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:46.380175 10800 raft_consensus.cc:2468] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:46.380720 10727 raft_consensus.cc:2468] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:46.382781 10544 leader_election.cc:304] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:46.378963 10905 leader_election.cc:290] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:46.384706 10906 raft_consensus.cc:2804] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.385660 10876 raft_consensus.cc:3060] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.386646 10850 tablet_bootstrap.cc:654] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.390362 10906 raft_consensus.cc:493] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.395164 10906 raft_consensus.cc:3060] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.398963 10851 raft_consensus.cc:493] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.399703 10851 raft_consensus.cc:515] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.402904 10851 leader_election.cc:290] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:46.389097 10846 raft_consensus.cc:359] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.405527 10846 raft_consensus.cc:385] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.405860 10846 raft_consensus.cc:740] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.407014 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf062bca1cd416181f4c017fecb396c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:46.411486 10544 leader_election.cc:343] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.411504 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:46.413213 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:46.413645 10653 raft_consensus.cc:2468] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
W20260709 13:04:46.413260 10693 leader_election.cc:343] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.409797 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf062bca1cd416181f4c017fecb396c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:46.414570 10801 raft_consensus.cc:2468] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
W20260709 13:04:46.415858 10541 leader_election.cc:343] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.416306 10541 leader_election.cc:304] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:46.416627 10693 leader_election.cc:304] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:46.419085 10851 raft_consensus.cc:2749] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:46.421530 10905 raft_consensus.cc:2804] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.422024 10905 raft_consensus.cc:493] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.422842 10905 raft_consensus.cc:3060] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.407055 10846 consensus_queue.cc:260] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.435858 10925 consensus_queue.cc:1048] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.447412 10846 ts_tablet_manager.cc:1434] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.075s	user 0.010s	sys 0.000s
I20260709 13:04:46.450089 10928 consensus_queue.cc:1048] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:04:46.457405 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:46.432554 10906 raft_consensus.cc:515] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.461897 10874 raft_consensus.cc:493] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.462688 10874 raft_consensus.cc:515] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.467006 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:46.468084 10727 raft_consensus.cc:3060] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.468923 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6224db6d69b24c2ba435311454b28dc9" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:46.470072 10726 raft_consensus.cc:2468] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:46.473374 10616 leader_election.cc:304] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:46.474633 10863 raft_consensus.cc:2804] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.475340 10863 raft_consensus.cc:493] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.475481 10874 leader_election.cc:290] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:46.475764 10863 raft_consensus.cc:3060] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.476255 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6224db6d69b24c2ba435311454b28dc9" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:46.480250 10619 leader_election.cc:343] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.493268 10874 consensus_queue.cc:1048] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.502359 10876 raft_consensus.cc:515] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.503510 10846 tablet_bootstrap.cc:492] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:46.515671 10906 leader_election.cc:290] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:46.498368 10905 raft_consensus.cc:515] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.521142 10846 tablet_bootstrap.cc:654] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.521142 10847 tablet_bootstrap.cc:654] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.526947 10850 tablet_bootstrap.cc:492] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:46.539306 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:46.544736 10850 ts_tablet_manager.cc:1403] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.186s	user 0.051s	sys 0.076s
I20260709 13:04:46.553907 10800 raft_consensus.cc:3060] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.555233 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec45eaa322240d2af46cc177f87d32f" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:46.541033 10863 raft_consensus.cc:515] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.553929 10905 leader_election.cc:290] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:46.548794 10876 leader_election.cc:290] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:46.558604 10850 raft_consensus.cc:359] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.559482 10850 raft_consensus.cc:385] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:46.559703 10693 leader_election.cc:343] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.559864 10850 raft_consensus.cc:740] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.561086 10850 consensus_queue.cc:260] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.561915 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6224db6d69b24c2ba435311454b28dc9" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:46.563568 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:46.564363 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec45eaa322240d2af46cc177f87d32f" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:46.565043 10576 raft_consensus.cc:3060] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.560109 10863 leader_election.cc:290] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:46.568077 10848 tablet_bootstrap.cc:492] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:46.564428 10798 raft_consensus.cc:3060] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.568840 10848 ts_tablet_manager.cc:1403] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.293s	user 0.018s	sys 0.002s
W20260709 13:04:46.567994 10619 leader_election.cc:343] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.572435 10727 raft_consensus.cc:2468] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
I20260709 13:04:46.573544 10846 tablet_bootstrap.cc:492] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:46.574172 10846 ts_tablet_manager.cc:1403] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.100s	user 0.015s	sys 0.011s
I20260709 13:04:46.588297 10846 raft_consensus.cc:359] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.589138 10846 raft_consensus.cc:385] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.589443 10846 raft_consensus.cc:740] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.589810 10541 leader_election.cc:304] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:46.590248 10846 consensus_queue.cc:260] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.590943 10906 raft_consensus.cc:2804] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.591082 10848 raft_consensus.cc:359] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.591882 10848 raft_consensus.cc:385] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.592175 10848 raft_consensus.cc:740] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.593139 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:46.593631 10850 ts_tablet_manager.cc:1434] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.002s
I20260709 13:04:46.594069 10652 raft_consensus.cc:3060] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.595080 10850 tablet_bootstrap.cc:492] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:46.606113 10846 ts_tablet_manager.cc:1434] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:04:46.613902 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6224db6d69b24c2ba435311454b28dc9" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:46.614754 10727 raft_consensus.cc:3060] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.618577 10847 tablet_bootstrap.cc:492] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:46.620223 10847 ts_tablet_manager.cc:1403] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.267s	user 0.017s	sys 0.008s
I20260709 13:04:46.620604 10846 tablet_bootstrap.cc:492] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:46.622617 10850 tablet_bootstrap.cc:654] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.628571 10906 raft_consensus.cc:697] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:04:46.626055 10848 consensus_queue.cc:260] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.651440 10576 raft_consensus.cc:2468] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:46.659894 10800 raft_consensus.cc:2468] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
I20260709 13:04:46.666730 10692 leader_election.cc:304] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:46.670050 10928 raft_consensus.cc:2804] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.670531 10928 raft_consensus.cc:697] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:46.668829 10846 tablet_bootstrap.cc:654] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.667964 10727 raft_consensus.cc:2468] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:46.653945 10863 consensus_queue.cc:1048] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.668285 10848 ts_tablet_manager.cc:1434] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.099s	user 0.011s	sys 0.001s
I20260709 13:04:46.662894 10847 raft_consensus.cc:359] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.673902 10847 raft_consensus.cc:385] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.674310 10847 raft_consensus.cc:740] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.669350 10798 raft_consensus.cc:2468] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:46.670154 10906 consensus_queue.cc:237] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.683188 10616 leader_election.cc:304] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:46.684777 10874 raft_consensus.cc:2804] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.685320 10874 raft_consensus.cc:697] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:46.686169 10874 consensus_queue.cc:237] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.665855 10652 raft_consensus.cc:2468] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:46.677857 10693 leader_election.cc:304] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:46.676160 10847 consensus_queue.cc:260] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.677963 10928 consensus_queue.cc:237] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.700284 10847 ts_tablet_manager.cc:1434] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.078s	user 0.009s	sys 0.001s
I20260709 13:04:46.701411 10847 tablet_bootstrap.cc:492] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:46.697649 10905 raft_consensus.cc:2804] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.702922 10905 raft_consensus.cc:697] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
W20260709 13:04:46.690009 10618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 45912.4us
I20260709 13:04:46.703892 10905 consensus_queue.cc:237] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.704382 10848 tablet_bootstrap.cc:492] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:46.710031 10876 raft_consensus.cc:493] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.717657 10876 raft_consensus.cc:515] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.721729 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf062bca1cd416181f4c017fecb396c" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:46.723896 10693 leader_election.cc:343] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.726768 10876 leader_election.cc:290] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:46.729261 10937 raft_consensus.cc:493] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.730402 10851 raft_consensus.cc:493] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.730253 10937 raft_consensus.cc:515] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.731114 10851 raft_consensus.cc:515] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.733101 10937 leader_election.cc:290] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:46.734216 10851 leader_election.cc:290] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:46.729451 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf062bca1cd416181f4c017fecb396c" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:46.743575 10577 raft_consensus.cc:2468] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:46.744097 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b02d8d78f8e0419186c6c09d9cc307d7" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:46.744546 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b02d8d78f8e0419186c6c09d9cc307d7" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:46.746104 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:46.746791 10652 raft_consensus.cc:2468] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
W20260709 13:04:46.747958 10544 leader_election.cc:343] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.749222 10544 leader_election.cc:304] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:46.750154 10692 leader_election.cc:304] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:46.750286 10851 raft_consensus.cc:2804] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.750943 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:46.751149 10851 raft_consensus.cc:493] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.751602 10851 raft_consensus.cc:3060] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.752285 10846 tablet_bootstrap.cc:492] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
W20260709 13:04:46.752736 10544 leader_election.cc:343] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.753144 10846 ts_tablet_manager.cc:1403] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.133s	user 0.015s	sys 0.007s
I20260709 13:04:46.753016 10476 catalog_manager.cc:5697] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:46.756644 10541 leader_election.cc:343] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.756876 10928 raft_consensus.cc:2804] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.760027 10928 raft_consensus.cc:493] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.760532 10928 raft_consensus.cc:3060] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.763316 10475 catalog_manager.cc:5697] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:46.767629 10850 tablet_bootstrap.cc:492] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:46.768568 10850 ts_tablet_manager.cc:1403] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.174s	user 0.021s	sys 0.004s
I20260709 13:04:46.758315 10541 leader_election.cc:304] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:46.774922 10846 raft_consensus.cc:359] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.776499 10846 raft_consensus.cc:385] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.777098 10846 raft_consensus.cc:740] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.776068 10906 raft_consensus.cc:2749] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:46.787499 10847 tablet_bootstrap.cc:654] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.790971 10848 tablet_bootstrap.cc:654] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.788829 10850 raft_consensus.cc:359] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.793220 10850 raft_consensus.cc:385] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.796164 10850 raft_consensus.cc:740] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.796662 10476 catalog_manager.cc:5697] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.794835 10846 consensus_queue.cc:260] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.805454 10846 ts_tablet_manager.cc:1434] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 13:04:46.805166 10851 raft_consensus.cc:515] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.802825 10850 consensus_queue.cc:260] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.808880 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
W20260709 13:04:46.810665 10544 leader_election.cc:343] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.813211 10846 tablet_bootstrap.cc:492] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:46.815672 10850 ts_tablet_manager.cc:1434] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.000s
I20260709 13:04:46.820870 10850 tablet_bootstrap.cc:492] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:46.818502 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:46.823386 10652 raft_consensus.cc:3060] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.818229 10851 leader_election.cc:290] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:46.826730 10846 tablet_bootstrap.cc:654] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.848100 10928 raft_consensus.cc:515] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.849291 10876 consensus_queue.cc:1048] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.851227 10850 tablet_bootstrap.cc:654] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.851446 10928 leader_election.cc:290] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:46.855026 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf062bca1cd416181f4c017fecb396c" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
W20260709 13:04:46.856638 10693 leader_election.cc:343] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.857051 10652 raft_consensus.cc:2468] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
I20260709 13:04:46.858845 10847 tablet_bootstrap.cc:492] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:46.859841 10847 ts_tablet_manager.cc:1403] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.159s	user 0.011s	sys 0.032s
I20260709 13:04:46.858907 10544 leader_election.cc:304] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:46.862433 10851 raft_consensus.cc:2804] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.863909 10851 raft_consensus.cc:697] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:04:46.869261 10928 consensus_queue.cc:1048] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.869944 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf062bca1cd416181f4c017fecb396c" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:46.870707 10578 raft_consensus.cc:3060] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.871734 10848 tablet_bootstrap.cc:492] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:46.873030 10848 ts_tablet_manager.cc:1403] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.175s	user 0.013s	sys 0.014s
I20260709 13:04:46.876540 10851 consensus_queue.cc:237] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.881067 10476 catalog_manager.cc:5697] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.896461 10847 raft_consensus.cc:359] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.897239 10847 raft_consensus.cc:385] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.897578 10847 raft_consensus.cc:740] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.901371 10847 consensus_queue.cc:260] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.905405 10847 ts_tablet_manager.cc:1434] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.045s	user 0.011s	sys 0.000s
I20260709 13:04:46.905512 10846 tablet_bootstrap.cc:492] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:46.907691 10846 ts_tablet_manager.cc:1403] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.095s	user 0.017s	sys 0.008s
I20260709 13:04:46.905632 10848 raft_consensus.cc:359] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.909628 10892 raft_consensus.cc:493] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.910187 10850 tablet_bootstrap.cc:492] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:46.914988 10892 raft_consensus.cc:515] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.923151 10892 leader_election.cc:290] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:46.919286 10847 tablet_bootstrap.cc:492] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:46.910898 10850 ts_tablet_manager.cc:1403] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.091s	user 0.010s	sys 0.010s
I20260709 13:04:46.910295 10848 raft_consensus.cc:385] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.918992 10846 raft_consensus.cc:359] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.926210 10848 raft_consensus.cc:740] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.926535 10846 raft_consensus.cc:385] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.926959 10846 raft_consensus.cc:740] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.933267 10846 consensus_queue.cc:260] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.934727 10578 raft_consensus.cc:2468] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:46.932307 10847 tablet_bootstrap.cc:654] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.927165 10848 consensus_queue.cc:260] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.940735 10848 ts_tablet_manager.cc:1434] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.067s	user 0.005s	sys 0.005s
I20260709 13:04:46.935786 10846 ts_tablet_manager.cc:1434] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:04:46.929432 10850 raft_consensus.cc:359] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.941572 10850 raft_consensus.cc:385] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.941923 10850 raft_consensus.cc:740] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.942745 10850 consensus_queue.cc:260] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:46.944109 10692 leader_election.cc:304] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:46.946034 10850 ts_tablet_manager.cc:1434] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:04:46.947732 10848 tablet_bootstrap.cc:492] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:46.927454 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6224db6d69b24c2ba435311454b28dc9" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:46.954543 10846 tablet_bootstrap.cc:492] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:46.955269 10727 raft_consensus.cc:2393] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 1: Already voted for candidate 73db933c4a6848d1b2aeb56813f7bb2c in this term.
I20260709 13:04:46.957351 10850 tablet_bootstrap.cc:492] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:46.962380 10848 tablet_bootstrap.cc:654] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.966050 10850 tablet_bootstrap.cc:654] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.967928 10940 raft_consensus.cc:493] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.968559 10846 tablet_bootstrap.cc:654] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.968544 10940 raft_consensus.cc:515] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:46.973779 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6224db6d69b24c2ba435311454b28dc9" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:46.976050 10767 leader_election.cc:304] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:46.977021 10847 tablet_bootstrap.cc:492] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:46.977674 10892 raft_consensus.cc:3060] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.977906 10847 ts_tablet_manager.cc:1403] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.002s
I20260709 13:04:46.982116 10847 raft_consensus.cc:359] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.983134 10847 raft_consensus.cc:385] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.983448 10847 raft_consensus.cc:740] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.983604 10846 tablet_bootstrap.cc:492] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:46.984253 10846 ts_tablet_manager.cc:1403] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.004s
I20260709 13:04:46.984474 10847 consensus_queue.cc:260] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:46.987663 10846 raft_consensus.cc:359] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:46.988511 10846 raft_consensus.cc:385] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.988556 10847 ts_tablet_manager.cc:1434] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:04:46.988963 10846 raft_consensus.cc:740] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.989955 10847 tablet_bootstrap.cc:492] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:46.991451 10940 leader_election.cc:290] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:46.991918 10848 tablet_bootstrap.cc:492] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:46.992503 10848 ts_tablet_manager.cc:1403] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.000s
I20260709 13:04:46.992983 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec45eaa322240d2af46cc177f87d32f" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:46.993402 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ec45eaa322240d2af46cc177f87d32f" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:46.994812 10578 raft_consensus.cc:2393] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 7ecd836bb7a144dc9aa07a046239d6ce in this term.
I20260709 13:04:46.992985 10928 raft_consensus.cc:2804] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.995697 10928 raft_consensus.cc:697] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
W20260709 13:04:46.993136 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 49122.2us
I20260709 13:04:47.002094 10846 consensus_queue.cc:260] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.005750 10846 ts_tablet_manager.cc:1434] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 13:04:47.007040 10846 tablet_bootstrap.cc:492] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:47.006958 10847 tablet_bootstrap.cc:654] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.012388 10618 leader_election.cc:304] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:46.996744 10928 consensus_queue.cc:237] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.023478 10940 raft_consensus.cc:3060] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.024803 10848 raft_consensus.cc:359] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.026027 10848 raft_consensus.cc:385] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.026326 10848 raft_consensus.cc:740] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.031518 10892 raft_consensus.cc:2749] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.027843 10848 consensus_queue.cc:260] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.052388 10850 tablet_bootstrap.cc:492] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:47.053076 10850 ts_tablet_manager.cc:1403] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.096s	user 0.013s	sys 0.027s
I20260709 13:04:47.055285 10851 raft_consensus.cc:493] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.056107 10851 raft_consensus.cc:515] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.059883 10876 raft_consensus.cc:493] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.060424 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ecb24ddae546da9915163c05742a7e" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:47.060606 10876 raft_consensus.cc:515] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.061201 10727 raft_consensus.cc:2468] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:47.062220 10848 ts_tablet_manager.cc:1434] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.069s	user 0.014s	sys 0.000s
I20260709 13:04:47.069878 10541 leader_election.cc:304] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:47.071003 10906 raft_consensus.cc:2804] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.071481 10906 raft_consensus.cc:493] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.071863 10906 raft_consensus.cc:3060] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.072053 10848 tablet_bootstrap.cc:492] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:47.064992 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b02d8d78f8e0419186c6c09d9cc307d7" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:47.074970 10693 leader_election.cc:343] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.075090 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b02d8d78f8e0419186c6c09d9cc307d7" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:47.065639 10876 leader_election.cc:290] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:47.077924 10874 raft_consensus.cc:493] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.078701 10874 raft_consensus.cc:515] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.082321 10874 leader_election.cc:290] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:47.083359 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f26ffe11b8143728804f33b884ed36a" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:47.080621 10847 tablet_bootstrap.cc:492] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:47.084620 10847 ts_tablet_manager.cc:1403] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.095s	user 0.027s	sys 0.027s
I20260709 13:04:47.076033 10578 raft_consensus.cc:2468] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:47.082414 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f26ffe11b8143728804f33b884ed36a" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
W20260709 13:04:47.085876 10619 leader_election.cc:343] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.095448 10692 leader_election.cc:304] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:47.097087 10851 leader_election.cc:290] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:47.097976 10848 tablet_bootstrap.cc:654] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.097801 10905 raft_consensus.cc:2804] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.098460 10876 consensus_queue.cc:1048] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.099046 10905 raft_consensus.cc:493] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.099480 10905 raft_consensus.cc:3060] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.067088 10846 tablet_bootstrap.cc:654] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.102018 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ecb24ddae546da9915163c05742a7e" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:47.104449 10544 leader_election.cc:343] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:47.116966 10616 leader_election.cc:343] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.117784 10616 leader_election.cc:304] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:47.119275 10874 raft_consensus.cc:2749] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.118974 10850 raft_consensus.cc:359] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.125283 10850 raft_consensus.cc:385] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.125762 10850 raft_consensus.cc:740] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.126716 10850 consensus_queue.cc:260] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.108832 10906 raft_consensus.cc:515] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.133010 10949 consensus_queue.cc:1048] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.125591 10851 raft_consensus.cc:493] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.138937 10851 raft_consensus.cc:515] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.142629 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ecb24ddae546da9915163c05742a7e" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:47.143781 10727 raft_consensus.cc:3060] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.144702 10940 raft_consensus.cc:2749] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.148434 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:47.147377 10851 leader_election.cc:290] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:47.070714 10473 catalog_manager.cc:5697] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:47.150300 10544 leader_election.cc:343] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.157115 10850 ts_tablet_manager.cc:1434] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.104s	user 0.013s	sys 0.000s
I20260709 13:04:47.158562 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:47.160326 10544 leader_election.cc:343] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.160840 10544 leader_election.cc:304] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:47.161536 10876 consensus_queue.cc:1048] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.158788 10850 tablet_bootstrap.cc:492] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:47.162322 10851 raft_consensus.cc:2749] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.174947 10846 tablet_bootstrap.cc:492] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:47.175787 10846 ts_tablet_manager.cc:1403] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.169s	user 0.020s	sys 0.008s
I20260709 13:04:47.142788 10905 raft_consensus.cc:515] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.178541 10848 tablet_bootstrap.cc:492] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:47.177318 10847 raft_consensus.cc:359] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.179219 10847 raft_consensus.cc:385] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.179608 10847 raft_consensus.cc:740] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.180136 10848 ts_tablet_manager.cc:1403] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.110s	user 0.019s	sys 0.010s
I20260709 13:04:47.180529 10847 consensus_queue.cc:260] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.172019 10851 consensus_queue.cc:1048] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:04:47.181071 10906 leader_election.cc:290] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:47.187187 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ecb24ddae546da9915163c05742a7e" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:47.192253 10846 raft_consensus.cc:359] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.193090 10846 raft_consensus.cc:385] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.193367 10846 raft_consensus.cc:740] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.194067 10846 consensus_queue.cc:260] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.204697 10847 ts_tablet_manager.cc:1434] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.116s	user 0.009s	sys 0.001s
I20260709 13:04:47.205883 10847 tablet_bootstrap.cc:492] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:47.207463 10846 ts_tablet_manager.cc:1434] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 13:04:47.209266 10846 tablet_bootstrap.cc:492] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:47.199741 10940 consensus_queue.cc:1048] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:04:47.216120 10850 tablet_bootstrap.cc:654] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:47.214037 10544 leader_election.cc:343] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.220028 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b02d8d78f8e0419186c6c09d9cc307d7" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:47.220772 10578 raft_consensus.cc:3060] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.228572 10846 tablet_bootstrap.cc:654] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.231401 10905 leader_election.cc:290] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
W20260709 13:04:47.246290 10962 log.cc:927] Time spent T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:04:47.244642 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b02d8d78f8e0419186c6c09d9cc307d7" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:47.250567 10848 raft_consensus.cc:359] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.252317 10848 raft_consensus.cc:385] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.252650 10848 raft_consensus.cc:740] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.251731 10653 raft_consensus.cc:3060] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.255155 10848 consensus_queue.cc:260] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.274470 10848 ts_tablet_manager.cc:1434] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.094s	user 0.015s	sys 0.003s
I20260709 13:04:47.275830 10848 tablet_bootstrap.cc:492] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:47.277010 10892 raft_consensus.cc:493] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.277673 10892 raft_consensus.cc:515] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.278069 10940 consensus_queue.cc:1048] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.280164 10892 leader_election.cc:290] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:47.277220 10967 consensus_queue.cc:1048] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.277760 10847 tablet_bootstrap.cc:654] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.284752 10848 tablet_bootstrap.cc:654] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.298228 10727 raft_consensus.cc:2468] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
I20260709 13:04:47.315116 10541 leader_election.cc:304] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:47.316648 10937 raft_consensus.cc:2804] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.317256 10937 raft_consensus.cc:697] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:04:47.318230 10851 raft_consensus.cc:493] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.318305 10937 consensus_queue.cc:237] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.318974 10851 raft_consensus.cc:515] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.315208 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:47.365918 10652 raft_consensus.cc:2393] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 1: Already voted for candidate bef3584ebb2c4a2ea619936c71900230 in this term.
I20260709 13:04:47.369580 10799 raft_consensus.cc:3060] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.376806 10846 tablet_bootstrap.cc:492] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:47.377693 10846 ts_tablet_manager.cc:1403] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.169s	user 0.027s	sys 0.020s
W20260709 13:04:47.381827 10541 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 66884.1us
I20260709 13:04:47.385555 10846 raft_consensus.cc:359] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.393761 10846 raft_consensus.cc:385] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.394706 10846 raft_consensus.cc:740] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.395819 10578 raft_consensus.cc:2468] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:47.390734 10653 raft_consensus.cc:2468] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:47.401252 10693 leader_election.cc:304] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:47.402644 10848 tablet_bootstrap.cc:492] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:47.402850 10925 raft_consensus.cc:2804] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.395924 10906 consensus_queue.cc:1048] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.406823 10847 tablet_bootstrap.cc:492] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:47.407850 10847 ts_tablet_manager.cc:1403] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.202s	user 0.015s	sys 0.014s
I20260709 13:04:47.398492 10928 consensus_queue.cc:1048] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.395922 10846 consensus_queue.cc:260] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.411736 10906 consensus_queue.cc:1048] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.413403 10846 ts_tablet_manager.cc:1434] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.001s
I20260709 13:04:47.415000 10925 raft_consensus.cc:697] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:47.415812 10847 raft_consensus.cc:359] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.416705 10847 raft_consensus.cc:385] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.417176 10847 raft_consensus.cc:740] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.416419 10925 consensus_queue.cc:237] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.422099 10973 raft_consensus.cc:493] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.422696 10973 raft_consensus.cc:515] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.428308 10973 leader_election.cc:290] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:47.429204 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f26ffe11b8143728804f33b884ed36a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:47.430056 10727 raft_consensus.cc:2468] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:47.429189 10846 tablet_bootstrap.cc:492] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:47.432073 10764 leader_election.cc:304] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:47.433941 10973 raft_consensus.cc:2804] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.434407 10973 raft_consensus.cc:493] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.434927 10973 raft_consensus.cc:3060] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.403259 10848 ts_tablet_manager.cc:1403] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.128s	user 0.022s	sys 0.008s
I20260709 13:04:47.387006 10474 catalog_manager.cc:5697] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.440833 10846 tablet_bootstrap.cc:654] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.442005 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f26ffe11b8143728804f33b884ed36a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:47.442934 10652 raft_consensus.cc:2468] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:47.448144 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:47.450263 10544 leader_election.cc:343] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.418066 10847 consensus_queue.cc:260] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.455569 10846 tablet_bootstrap.cc:492] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:47.456274 10846 ts_tablet_manager.cc:1403] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.004s
I20260709 13:04:47.452677 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:47.457585 10973 raft_consensus.cc:515] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.466171 10766 leader_election.cc:304] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:47.468511 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f26ffe11b8143728804f33b884ed36a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:47.469522 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:47.469590 10653 raft_consensus.cc:3060] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:47.471633 10544 leader_election.cc:343] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.472164 10544 leader_election.cc:304] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:47.473711 10937 raft_consensus.cc:2749] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.478101 10972 raft_consensus.cc:2749] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.481184 10848 raft_consensus.cc:359] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.482095 10848 raft_consensus.cc:385] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.482366 10848 raft_consensus.cc:740] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.460101 10851 leader_election.cc:290] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:47.487280 10847 ts_tablet_manager.cc:1434] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.000s
I20260709 13:04:47.488346 10848 consensus_queue.cc:260] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.489562 10847 tablet_bootstrap.cc:492] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:47.490522 10851 raft_consensus.cc:493] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.491032 10851 raft_consensus.cc:515] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.492964 10928 consensus_queue.cc:1048] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.494266 10851 leader_election.cc:290] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:47.496554 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95de1505ade4f5389a46d7aca7d7136" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:47.498363 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95de1505ade4f5389a46d7aca7d7136" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:47.499074 10727 raft_consensus.cc:2468] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:47.503181 10846 raft_consensus.cc:359] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.503898 10846 raft_consensus.cc:385] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.504145 10846 raft_consensus.cc:740] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.504326 10541 leader_election.cc:304] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:47.504890 10846 consensus_queue.cc:260] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.510937 10972 raft_consensus.cc:493] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.511289 10850 tablet_bootstrap.cc:492] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:47.511966 10850 ts_tablet_manager.cc:1403] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.354s	user 0.030s	sys 0.068s
I20260709 13:04:47.511590 10972 raft_consensus.cc:515] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
W20260709 13:04:47.512593 10544 leader_election.cc:343] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.517704 10973 leader_election.cc:290] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:47.520231 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf062bca1cd416181f4c017fecb396c" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:47.520943 10972 leader_election.cc:290] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:47.520100 10848 ts_tablet_manager.cc:1434] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 13:04:47.521945 10846 ts_tablet_manager.cc:1434] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.000s
I20260709 13:04:47.523110 10846 tablet_bootstrap.cc:492] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:47.523144 10848 tablet_bootstrap.cc:492] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:47.523921 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f26ffe11b8143728804f33b884ed36a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:47.524626 10727 raft_consensus.cc:3060] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.526024 10851 raft_consensus.cc:2804] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.526446 10851 raft_consensus.cc:493] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.526782 10851 raft_consensus.cc:3060] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.527431 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf062bca1cd416181f4c017fecb396c" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:47.529148 10766 leader_election.cc:304] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:47.530134 10972 raft_consensus.cc:2749] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.530471 10847 tablet_bootstrap.cc:654] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.537561 10851 raft_consensus.cc:515] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.541682 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95de1505ade4f5389a46d7aca7d7136" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:47.542483 10725 raft_consensus.cc:3060] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:47.543277 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:47.546862 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95de1505ade4f5389a46d7aca7d7136" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:47.549145 10850 raft_consensus.cc:359] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:04:47.549903 10544 leader_election.cc:343] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.550145 10850 raft_consensus.cc:385] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.551155 10850 raft_consensus.cc:740] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.552006 10850 consensus_queue.cc:260] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.555991 10850 ts_tablet_manager.cc:1434] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.001s
I20260709 13:04:47.556958 10725 raft_consensus.cc:2468] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
I20260709 13:04:47.557240 10850 tablet_bootstrap.cc:492] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:47.556404 10474 catalog_manager.cc:5697] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:47.558794 10846 tablet_bootstrap.cc:654] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.562757 10851 leader_election.cc:290] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:47.567633 10846 tablet_bootstrap.cc:492] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:47.568475 10846 ts_tablet_manager.cc:1403] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.001s
I20260709 13:04:47.571738 10846 raft_consensus.cc:359] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.573307 10846 raft_consensus.cc:385] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.573696 10846 raft_consensus.cc:740] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.574924 10847 tablet_bootstrap.cc:492] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:47.574620 10846 consensus_queue.cc:260] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.575671 10847 ts_tablet_manager.cc:1403] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.088s	user 0.015s	sys 0.001s
I20260709 13:04:47.577610 10799 raft_consensus.cc:3060] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.565150 10541 leader_election.cc:304] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:47.579264 10847 raft_consensus.cc:359] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.579783 10851 raft_consensus.cc:2804] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.580097 10847 raft_consensus.cc:385] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.580358 10851 raft_consensus.cc:697] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:04:47.581153 10850 tablet_bootstrap.cc:654] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.581442 10851 consensus_queue.cc:237] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.583712 10653 raft_consensus.cc:2468] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:47.580464 10847 raft_consensus.cc:740] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.585049 10949 raft_consensus.cc:493] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.585146 10847 consensus_queue.cc:260] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.585758 10949 raft_consensus.cc:515] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.588479 10949 leader_election.cc:290] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:47.588496 10847 ts_tablet_manager.cc:1434] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:04:47.579532 10846 ts_tablet_manager.cc:1434] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:04:47.591253 10767 leader_election.cc:304] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:04:47.592300 10972 raft_consensus.cc:2804] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.592840 10972 raft_consensus.cc:697] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:47.593993 10972 consensus_queue.cc:237] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.595090 10476 catalog_manager.cc:5697] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:47.590039 10847 tablet_bootstrap.cc:492] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:47.603612 10848 tablet_bootstrap.cc:654] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.608431 10846 tablet_bootstrap.cc:492] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:47.614629 10847 tablet_bootstrap.cc:654] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.586026 10727 raft_consensus.cc:2468] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:47.633652 10846 tablet_bootstrap.cc:654] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.637060 10848 tablet_bootstrap.cc:492] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:47.637884 10848 ts_tablet_manager.cc:1403] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.115s	user 0.014s	sys 0.006s
I20260709 13:04:47.637173 10473 catalog_manager.cc:5697] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:47.644210 10847 tablet_bootstrap.cc:492] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
W20260709 13:04:47.644671 10541 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 79665.5us
I20260709 13:04:47.644898 10847 ts_tablet_manager.cc:1403] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.002s
I20260709 13:04:47.648038 10949 raft_consensus.cc:493] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.648674 10949 raft_consensus.cc:515] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.601194 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2bb2563ed2a407c9c219a632057e54a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:47.654369 10847 raft_consensus.cc:359] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.655069 10949 leader_election.cc:290] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:47.655396 10847 raft_consensus.cc:385] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.655726 10847 raft_consensus.cc:740] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.656641 10848 raft_consensus.cc:359] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.658505 10848 raft_consensus.cc:385] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.658808 10848 raft_consensus.cc:740] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.660712 10848 consensus_queue.cc:260] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.664575 10979 raft_consensus.cc:493] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.662389 10846 tablet_bootstrap.cc:492] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:47.662194 10847 consensus_queue.cc:260] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.666215 10846 ts_tablet_manager.cc:1403] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.006s
I20260709 13:04:47.656665 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e728368868473181686e35396f3ae5" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:47.665659 10979 raft_consensus.cc:515] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.675935 10979 leader_election.cc:290] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:47.668946 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2bb2563ed2a407c9c219a632057e54a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:47.677155 10846 raft_consensus.cc:359] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.670444 10848 ts_tablet_manager.cc:1434] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:04:47.670279 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e728368868473181686e35396f3ae5" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
W20260709 13:04:47.670006 10544 leader_election.cc:343] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.680017 10848 tablet_bootstrap.cc:492] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:47.681573 10847 ts_tablet_manager.cc:1434] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.000s
I20260709 13:04:47.682873 10846 raft_consensus.cc:385] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.685169 10928 consensus_queue.cc:1048] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.685868 10846 raft_consensus.cc:740] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.686805 10846 consensus_queue.cc:260] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.682941 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2988fb83d740c6a6b692c5fc8d8105" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:47.688618 10848 tablet_bootstrap.cc:654] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.689211 10846 ts_tablet_manager.cc:1434] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.022s	user 0.002s	sys 0.003s
I20260709 13:04:47.689951 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2988fb83d740c6a6b692c5fc8d8105" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:47.695303 10846 tablet_bootstrap.cc:492] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
W20260709 13:04:47.692754 10544 leader_election.cc:343] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:47.701395 10544 leader_election.cc:343] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:47.702729 10544 leader_election.cc:343] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:47.690712 10541 leader_election.cc:343] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.704972 10541 leader_election.cc:304] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:47.707636 10949 raft_consensus.cc:2749] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:47.711383 10541 leader_election.cc:343] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.712242 10541 leader_election.cc:304] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:47.713519 10928 raft_consensus.cc:493] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.714802 10928 raft_consensus.cc:515] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.713693 10846 tablet_bootstrap.cc:654] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.718466 10937 raft_consensus.cc:2749] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.735394 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:47.736042 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:47.738126 10692 leader_election.cc:343] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:47.738927 10693 leader_election.cc:343] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.735461 10928 leader_election.cc:290] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:47.739761 10693 leader_election.cc:304] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:47.742067 10544 leader_election.cc:304] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:47.745625 10979 raft_consensus.cc:2749] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.746990 10928 raft_consensus.cc:2749] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.684819 10847 tablet_bootstrap.cc:492] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:47.780251 10847 tablet_bootstrap.cc:654] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.780310 10848 tablet_bootstrap.cc:492] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:47.781890 10848 ts_tablet_manager.cc:1403] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.102s	user 0.008s	sys 0.008s
I20260709 13:04:47.782963 10850 tablet_bootstrap.cc:492] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:47.783569 10850 ts_tablet_manager.cc:1403] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.227s	user 0.027s	sys 0.050s
I20260709 13:04:47.786753 10848 raft_consensus.cc:359] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.788069 10848 raft_consensus.cc:385] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.788480 10848 raft_consensus.cc:740] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.788671 10850 raft_consensus.cc:359] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.791620 10850 raft_consensus.cc:385] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.792085 10850 raft_consensus.cc:740] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.793007 10847 tablet_bootstrap.cc:492] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:47.793716 10847 ts_tablet_manager.cc:1403] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.109s	user 0.015s	sys 0.007s
I20260709 13:04:47.795794 10846 tablet_bootstrap.cc:492] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:47.796422 10846 ts_tablet_manager.cc:1403] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.101s	user 0.028s	sys 0.004s
I20260709 13:04:47.792968 10850 consensus_queue.cc:260] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.870373 10925 raft_consensus.cc:493] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.879101 10925 raft_consensus.cc:515] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.869462 10848 consensus_queue.cc:260] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.884476 10972 raft_consensus.cc:493] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.886036 10972 raft_consensus.cc:515] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.888625 10925 leader_election.cc:290] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:47.871575 10973 raft_consensus.cc:493] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.798158 10847 raft_consensus.cc:359] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.890642 10847 raft_consensus.cc:385] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.870312 10874 raft_consensus.cc:493] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.891248 10847 raft_consensus.cc:740] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.891913 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b99c17dd874ae2bd745e4ed1ba0330" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:47.892114 10874 raft_consensus.cc:515] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.893334 10798 raft_consensus.cc:2468] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:47.890376 10973 raft_consensus.cc:515] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:47.895486 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:47.896576 10576 raft_consensus.cc:2468] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:47.898844 10766 leader_election.cc:304] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:47.899827 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ecb24ddae546da9915163c05742a7e" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:47.892774 10847 consensus_queue.cc:260] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:47.901125 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ecb24ddae546da9915163c05742a7e" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:47.901661 10693 leader_election.cc:304] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:47.902729 10725 raft_consensus.cc:2393] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 1: Already voted for candidate bef3584ebb2c4a2ea619936c71900230 in this term.
I20260709 13:04:47.798527 10850 ts_tablet_manager.cc:1434] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.004s
I20260709 13:04:47.908587 10850 tablet_bootstrap.cc:492] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:47.909839 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:47.910246 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b99c17dd874ae2bd745e4ed1ba0330" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:47.910919 10798 raft_consensus.cc:2468] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:47.913053 10619 leader_election.cc:304] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:04:47.913446 10928 consensus_queue.cc:1048] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.914489 10938 raft_consensus.cc:2804] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.914938 10938 raft_consensus.cc:493] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.915267 10938 raft_consensus.cc:3060] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.904435 10925 raft_consensus.cc:2804] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.917100 10925 raft_consensus.cc:493] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.917606 10925 raft_consensus.cc:3060] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.903991 10973 leader_election.cc:290] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:47.922194 10973 raft_consensus.cc:2804] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.926178 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:47.926831 10874 leader_election.cc:290] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:47.929945 10973 raft_consensus.cc:493] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.930737 10577 raft_consensus.cc:2468] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:47.927832 10972 leader_election.cc:290] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:47.902745 10847 ts_tablet_manager.cc:1434] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.109s	user 0.001s	sys 0.008s
I20260709 13:04:47.927548 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:47.935075 10693 leader_election.cc:343] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.936321 10979 consensus_queue.cc:1048] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.937172 10973 raft_consensus.cc:3060] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.939061 10847 tablet_bootstrap.cc:492] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:47.965425 10928 consensus_queue.cc:1048] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.973273 10846 raft_consensus.cc:359] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.974829 10846 raft_consensus.cc:385] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.975322 10846 raft_consensus.cc:740] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.976473 10847 tablet_bootstrap.cc:654] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:47.985599 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 87211.9us
I20260709 13:04:47.987352 10850 tablet_bootstrap.cc:654] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.987681 10766 leader_election.cc:304] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:47.987977 10798 raft_consensus.cc:3060] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.976840 10846 consensus_queue.cc:260] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:47.994179 10846 ts_tablet_manager.cc:1434] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.197s	user 0.012s	sys 0.002s
I20260709 13:04:47.996371 10846 tablet_bootstrap.cc:492] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:47.990278 10973 raft_consensus.cc:515] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:47.998464 10473 catalog_manager.cc:5697] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.007877 10973 leader_election.cc:290] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:48.011046 10846 tablet_bootstrap.cc:654] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.012207 10848 ts_tablet_manager.cc:1434] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.230s	user 0.010s	sys 0.039s
I20260709 13:04:48.013921 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:48.014676 10848 tablet_bootstrap.cc:492] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:48.014952 10577 raft_consensus.cc:3060] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.012409 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
W20260709 13:04:48.040298 10544 consensus_peers.cc:597] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:48.009014 10938 raft_consensus.cc:515] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.053875 10651 raft_consensus.cc:2393] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 1: Already voted for candidate 73db933c4a6848d1b2aeb56813f7bb2c in this term.
I20260709 13:04:48.059517 10937 consensus_queue.cc:1048] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:48.063625 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:48.065060 10799 raft_consensus.cc:2393] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 5a12f1c20c674e7f8242df41a9acf5dc in this term.
I20260709 13:04:48.079667 10577 raft_consensus.cc:2468] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:48.083364 11001 consensus_queue.cc:1048] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:48.084774 10925 raft_consensus.cc:515] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.089951 10847 tablet_bootstrap.cc:492] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:48.090164 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:48.090759 10847 ts_tablet_manager.cc:1403] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.152s	user 0.025s	sys 0.029s
I20260709 13:04:48.091100 10577 raft_consensus.cc:2393] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 5a12f1c20c674e7f8242df41a9acf5dc in this term.
I20260709 13:04:48.093040 10618 leader_election.cc:304] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:48.094157 10938 leader_election.cc:290] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:48.094249 10874 raft_consensus.cc:2749] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.094697 10847 raft_consensus.cc:359] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.095467 10847 raft_consensus.cc:385] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.095880 10847 raft_consensus.cc:740] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.096891 10847 consensus_queue.cc:260] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.098856 10925 leader_election.cc:290] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:48.099821 10847 ts_tablet_manager.cc:1434] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:04:48.101293 10847 tablet_bootstrap.cc:492] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:48.109498 10766 leader_election.cc:304] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:48.116050 10850 tablet_bootstrap.cc:492] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:48.116771 10850 ts_tablet_manager.cc:1403] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.209s	user 0.020s	sys 0.008s
I20260709 13:04:48.113025 11001 raft_consensus.cc:2804] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:48.118463 11001 raft_consensus.cc:697] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:48.119809 11001 consensus_queue.cc:237] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.123473 10972 raft_consensus.cc:2749] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.128341 10937 consensus_queue.cc:1048] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:48.133181 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b99c17dd874ae2bd745e4ed1ba0330" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:48.137115 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b99c17dd874ae2bd745e4ed1ba0330" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:48.141775 10799 raft_consensus.cc:3060] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.139137 10850 raft_consensus.cc:359] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.147224 10847 tablet_bootstrap.cc:654] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.147020 10850 raft_consensus.cc:385] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.147679 10850 raft_consensus.cc:740] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
W20260709 13:04:48.151502 10693 leader_election.cc:343] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.154729 10848 tablet_bootstrap.cc:654] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.164810 10846 tablet_bootstrap.cc:492] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:48.170734 10972 consensus_queue.cc:1048] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:48.170773 10846 ts_tablet_manager.cc:1403] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.175s	user 0.040s	sys 0.014s
I20260709 13:04:48.174577 10850 consensus_queue.cc:260] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.185542 10850 ts_tablet_manager.cc:1434] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.068s	user 0.010s	sys 0.004s
I20260709 13:04:48.188264 10850 tablet_bootstrap.cc:492] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:48.191926 10949 raft_consensus.cc:493] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.192600 10949 raft_consensus.cc:515] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.189283 10846 raft_consensus.cc:359] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.194288 10846 raft_consensus.cc:385] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.194600 10846 raft_consensus.cc:740] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.195385 10846 consensus_queue.cc:260] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.196234 10949 leader_election.cc:290] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:48.199085 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a99af7813c7b40da8a055799f327ac72" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
W20260709 13:04:48.201081 10541 leader_election.cc:343] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.201536 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a99af7813c7b40da8a055799f327ac72" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:48.203143 10544 leader_election.cc:343] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.203667 10544 leader_election.cc:304] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:48.204591 10937 raft_consensus.cc:2749] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.205240 10846 ts_tablet_manager.cc:1434] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:04:48.206727 10846 tablet_bootstrap.cc:492] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:48.212715 10967 raft_consensus.cc:493] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.208546 10850 tablet_bootstrap.cc:654] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.210165 10847 tablet_bootstrap.cc:492] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:48.227406 10847 ts_tablet_manager.cc:1403] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.127s	user 0.021s	sys 0.012s
I20260709 13:04:48.227644 10938 raft_consensus.cc:493] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.227833 10967 raft_consensus.cc:515] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.231520 10847 raft_consensus.cc:359] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.236740 10938 raft_consensus.cc:515] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.238286 10799 raft_consensus.cc:2468] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:48.239331 10938 leader_election.cc:290] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:48.240489 10693 leader_election.cc:304] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:48.241092 10967 leader_election.cc:290] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:48.240507 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4c2675dde04ec982395518aa83bfb0" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:48.243221 10925 raft_consensus.cc:2804] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:48.243544 10726 raft_consensus.cc:2468] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:48.244081 10925 raft_consensus.cc:697] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:48.246354 10616 leader_election.cc:304] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:48.238420 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec71cde09fe64d089f95382f32269db5" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:48.253667 10938 raft_consensus.cc:2804] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.254735 10938 raft_consensus.cc:493] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.253495 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4c2675dde04ec982395518aa83bfb0" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:48.255199 10938 raft_consensus.cc:3060] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.255115 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec71cde09fe64d089f95382f32269db5" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:48.259373 10619 leader_election.cc:343] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:48.260941 10693 leader_election.cc:343] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:48.262346 10693 leader_election.cc:343] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.263042 10693 leader_election.cc:304] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:48.245812 10925 consensus_queue.cc:237] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.235127 10847 raft_consensus.cc:385] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.265786 10847 raft_consensus.cc:740] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.267022 10847 consensus_queue.cc:260] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.269280 10847 ts_tablet_manager.cc:1434] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 13:04:48.270685 10847 tablet_bootstrap.cc:492] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:48.265781 10967 raft_consensus.cc:2749] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.288501 10938 raft_consensus.cc:515] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.306535 10937 raft_consensus.cc:493] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.307219 10937 raft_consensus.cc:515] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.313169 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4d2b26b86c4f22976db2fc47a3ec1a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:48.314465 10937 leader_election.cc:290] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
W20260709 13:04:48.315686 10544 leader_election.cc:343] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.317113 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4d2b26b86c4f22976db2fc47a3ec1a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:48.318910 10848 tablet_bootstrap.cc:492] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
W20260709 13:04:48.318818 10544 leader_election.cc:343] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.319469 10544 leader_election.cc:304] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:48.320904 10937 raft_consensus.cc:2749] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.322559 10848 ts_tablet_manager.cc:1403] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.309s	user 0.027s	sys 0.032s
I20260709 13:04:48.323313 10846 tablet_bootstrap.cc:654] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.324664 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4c2675dde04ec982395518aa83bfb0" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:48.325646 10727 raft_consensus.cc:3060] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.325752 10475 catalog_manager.cc:5697] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.326995 10848 raft_consensus.cc:359] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.327812 10848 raft_consensus.cc:385] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.328075 10848 raft_consensus.cc:740] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.328825 10848 consensus_queue.cc:260] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.332726 10848 ts_tablet_manager.cc:1434] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:04:48.334059 10848 tablet_bootstrap.cc:492] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:48.341555 10848 tablet_bootstrap.cc:654] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.348769 10938 leader_election.cc:290] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:48.343794 10847 tablet_bootstrap.cc:654] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.353379 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4c2675dde04ec982395518aa83bfb0" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:48.354182 10799 raft_consensus.cc:3060] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.360538 10850 tablet_bootstrap.cc:492] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:48.361142 10850 ts_tablet_manager.cc:1403] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.174s	user 0.015s	sys 0.002s
I20260709 13:04:48.360508 10727 raft_consensus.cc:2468] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:48.362866 10616 leader_election.cc:304] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:48.363766 10938 raft_consensus.cc:2804] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:48.365310 10938 raft_consensus.cc:697] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:48.366438 10938 consensus_queue.cc:237] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.366019 10476 catalog_manager.cc:5697] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.367316 10847 tablet_bootstrap.cc:492] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:48.367707 10799 raft_consensus.cc:2468] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:48.368011 10847 ts_tablet_manager.cc:1403] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.098s	user 0.005s	sys 0.011s
I20260709 13:04:48.370404 10848 tablet_bootstrap.cc:492] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:48.370986 10848 ts_tablet_manager.cc:1403] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.006s
I20260709 13:04:48.371471 10847 raft_consensus.cc:359] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.372241 10847 raft_consensus.cc:385] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.372519 10847 raft_consensus.cc:740] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.373329 10847 consensus_queue.cc:260] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.375854 10847 ts_tablet_manager.cc:1434] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:48.376999 10847 tablet_bootstrap.cc:492] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:48.378911 10848 raft_consensus.cc:359] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.379726 10848 raft_consensus.cc:385] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.380023 10848 raft_consensus.cc:740] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.380805 10848 consensus_queue.cc:260] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.385726 10848 ts_tablet_manager.cc:1434] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 13:04:48.382411 10850 raft_consensus.cc:359] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.386592 10847 tablet_bootstrap.cc:654] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.386654 10850 raft_consensus.cc:385] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.387138 10850 raft_consensus.cc:740] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.388036 10850 consensus_queue.cc:260] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.392069 10850 ts_tablet_manager.cc:1434] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:04:48.398855 10850 tablet_bootstrap.cc:492] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:48.402802 10847 tablet_bootstrap.cc:492] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:48.402938 10848 tablet_bootstrap.cc:492] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:48.403519 10847 ts_tablet_manager.cc:1403] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.005s
I20260709 13:04:48.406932 10847 raft_consensus.cc:359] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.407502 10847 raft_consensus.cc:385] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.407796 10847 raft_consensus.cc:740] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.408028 10937 raft_consensus.cc:493] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.408577 10937 raft_consensus.cc:515] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.408502 10847 consensus_queue.cc:260] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.409916 10846 tablet_bootstrap.cc:492] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:48.410483 10846 ts_tablet_manager.cc:1403] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.204s	user 0.015s	sys 0.014s
I20260709 13:04:48.411185 10937 leader_election.cc:290] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:48.412034 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca28e86e81447cb899fbb41f02a7f03" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:48.413394 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca28e86e81447cb899fbb41f02a7f03" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
W20260709 13:04:48.413944 10544 leader_election.cc:343] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.413841 10846 raft_consensus.cc:359] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.414662 10846 raft_consensus.cc:385] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.414950 10846 raft_consensus.cc:740] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.415848 10846 consensus_queue.cc:260] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.416612 10848 tablet_bootstrap.cc:654] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:48.418471 10541 leader_election.cc:343] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.419092 10847 ts_tablet_manager.cc:1434] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.003s
I20260709 13:04:48.419023 10541 leader_election.cc:304] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:48.420912 10846 ts_tablet_manager.cc:1434] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:04:48.422277 10846 tablet_bootstrap.cc:492] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:48.424322 10850 tablet_bootstrap.cc:654] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.425033 10847 tablet_bootstrap.cc:492] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:48.429723 10937 raft_consensus.cc:2749] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.431401 10476 catalog_manager.cc:5697] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.444002 10847 tablet_bootstrap.cc:654] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.447732 10846 tablet_bootstrap.cc:654] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.453624 10850 tablet_bootstrap.cc:492] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:48.454764 10848 tablet_bootstrap.cc:492] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:48.455346 10848 ts_tablet_manager.cc:1403] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.014s
I20260709 13:04:48.456077 10850 ts_tablet_manager.cc:1403] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.058s	user 0.016s	sys 0.008s
I20260709 13:04:48.455350 10846 tablet_bootstrap.cc:492] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:48.457651 10846 ts_tablet_manager.cc:1403] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.000s
I20260709 13:04:48.459753 10847 tablet_bootstrap.cc:492] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:48.459393 10850 raft_consensus.cc:359] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.460224 10850 raft_consensus.cc:385] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.460331 10847 ts_tablet_manager.cc:1403] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.005s
I20260709 13:04:48.460553 10850 raft_consensus.cc:740] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.461500 10850 consensus_queue.cc:260] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.463609 10850 ts_tablet_manager.cc:1434] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:48.463774 10847 raft_consensus.cc:359] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.464615 10847 raft_consensus.cc:385] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.464887 10850 tablet_bootstrap.cc:492] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:48.464691 10848 raft_consensus.cc:359] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.464967 10847 raft_consensus.cc:740] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.465458 10848 raft_consensus.cc:385] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.465832 10848 raft_consensus.cc:740] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.466181 10847 consensus_queue.cc:260] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.466563 10848 consensus_queue.cc:260] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.468667 10847 ts_tablet_manager.cc:1434] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:48.473034 10850 tablet_bootstrap.cc:654] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.473573 10847 tablet_bootstrap.cc:492] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:48.477628 10848 ts_tablet_manager.cc:1434] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:04:48.460625 10846 raft_consensus.cc:359] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.479038 10846 raft_consensus.cc:385] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.479490 10846 raft_consensus.cc:740] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.480579 10846 consensus_queue.cc:260] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.483332 10848 tablet_bootstrap.cc:492] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:48.487498 10847 tablet_bootstrap.cc:654] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.492818 10848 tablet_bootstrap.cc:654] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.495322 10846 ts_tablet_manager.cc:1434] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:04:48.497591 10846 tablet_bootstrap.cc:492] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:48.501603 10847 tablet_bootstrap.cc:492] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:48.502278 10847 ts_tablet_manager.cc:1403] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.009s
I20260709 13:04:48.506502 10847 raft_consensus.cc:359] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.507282 10847 raft_consensus.cc:385] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.507577 10847 raft_consensus.cc:740] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.509537 10847 consensus_queue.cc:260] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.512019 10846 tablet_bootstrap.cc:654] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.514667 10850 tablet_bootstrap.cc:492] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:48.515332 10850 ts_tablet_manager.cc:1403] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.051s	user 0.020s	sys 0.015s
I20260709 13:04:48.518226 10847 ts_tablet_manager.cc:1434] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:04:48.518652 10850 raft_consensus.cc:359] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.519512 10850 raft_consensus.cc:385] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.519995 10850 raft_consensus.cc:740] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.520910 10850 consensus_queue.cc:260] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.519589 10847 tablet_bootstrap.cc:492] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:48.524549 10848 tablet_bootstrap.cc:492] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:48.525254 10848 ts_tablet_manager.cc:1403] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.000s
I20260709 13:04:48.527467 10850 ts_tablet_manager.cc:1434] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:04:48.528911 10850 tablet_bootstrap.cc:492] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:48.529250 10848 raft_consensus.cc:359] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.530081 10848 raft_consensus.cc:385] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.537864 10973 raft_consensus.cc:493] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.538734 10847 tablet_bootstrap.cc:654] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.542927 10850 tablet_bootstrap.cc:654] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.542460 10973 raft_consensus.cc:515] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.546380 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a99af7813c7b40da8a055799f327ac72" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:48.547367 10727 raft_consensus.cc:2468] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:48.549355 10764 leader_election.cc:304] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:48.550551 11000 raft_consensus.cc:2804] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.550746 10846 tablet_bootstrap.cc:492] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:48.551000 11000 raft_consensus.cc:493] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.551551 11000 raft_consensus.cc:3060] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.551644 10846 ts_tablet_manager.cc:1403] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.054s	user 0.019s	sys 0.000s
I20260709 13:04:48.553684 10973 leader_election.cc:290] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:48.554277 10848 raft_consensus.cc:740] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.554441 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a99af7813c7b40da8a055799f327ac72" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:48.555176 10577 raft_consensus.cc:2468] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:48.556229 10846 raft_consensus.cc:359] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.557006 10846 raft_consensus.cc:385] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.557331 10846 raft_consensus.cc:740] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.558424 10846 consensus_queue.cc:260] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.557014 10848 consensus_queue.cc:260] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.565338 10846 ts_tablet_manager.cc:1434] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 13:04:48.568202 10848 ts_tablet_manager.cc:1434] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:04:48.568871 10846 tablet_bootstrap.cc:492] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:48.569584 10848 tablet_bootstrap.cc:492] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:48.568363 11000 raft_consensus.cc:515] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.572533 10925 raft_consensus.cc:493] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.573091 10925 raft_consensus.cc:515] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.574985 10847 tablet_bootstrap.cc:492] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:48.575531 10925 leader_election.cc:290] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:48.577574 11000 leader_election.cc:290] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:48.578096 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f96bbe7d674a5485410f0aa8ec40e6" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:48.579650 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a99af7813c7b40da8a055799f327ac72" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:48.583336 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a99af7813c7b40da8a055799f327ac72" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:48.583932 10727 raft_consensus.cc:3060] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.580809 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f96bbe7d674a5485410f0aa8ec40e6" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:48.584237 10576 raft_consensus.cc:3060] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.584744 10577 raft_consensus.cc:2468] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:48.585016 10973 consensus_queue.cc:1048] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:48.581158 10651 raft_consensus.cc:3060] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:48.584089 10693 leader_election.cc:343] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.594909 10727 raft_consensus.cc:2468] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:48.596184 10692 leader_election.cc:304] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:48.597258 10764 leader_election.cc:304] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:48.599437 10967 raft_consensus.cc:2804] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.599941 10967 raft_consensus.cc:493] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.600065 11000 raft_consensus.cc:2804] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:48.600387 10967 raft_consensus.cc:3060] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.601140 11000 raft_consensus.cc:697] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:48.602355 11000 consensus_queue.cc:237] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.603154 10848 tablet_bootstrap.cc:654] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.605888 10846 tablet_bootstrap.cc:654] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.611742 10973 raft_consensus.cc:493] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.612381 10973 raft_consensus.cc:515] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.615298 10973 leader_election.cc:290] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:48.575620 10847 ts_tablet_manager.cc:1403] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.007s
I20260709 13:04:48.597393 10576 raft_consensus.cc:2468] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:48.620986 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:48.621933 10578 raft_consensus.cc:2468] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
W20260709 13:04:48.622694 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:48.623044 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:48.623941 10727 raft_consensus.cc:2468] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:48.623807 10766 leader_election.cc:304] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:48.625396 10973 raft_consensus.cc:2804] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.625998 10973 raft_consensus.cc:493] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.626475 10973 raft_consensus.cc:3060] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.631148 10938 raft_consensus.cc:493] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.637410 10938 raft_consensus.cc:515] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.639992 10938 leader_election.cc:290] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:48.642145 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2988fb83d740c6a6b692c5fc8d8105" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:48.643007 10799 raft_consensus.cc:2468] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:48.644305 10476 catalog_manager.cc:5697] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.645604 10619 leader_election.cc:304] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:04:48.646840 10938 raft_consensus.cc:2804] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.649863 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2988fb83d740c6a6b692c5fc8d8105" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:48.650599 10847 raft_consensus.cc:359] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.651053 10578 raft_consensus.cc:2468] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:48.651553 10847 raft_consensus.cc:385] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.652015 10847 raft_consensus.cc:740] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.652882 10847 consensus_queue.cc:260] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.654491 10850 tablet_bootstrap.cc:492] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:48.655042 10850 ts_tablet_manager.cc:1403] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.126s	user 0.011s	sys 0.004s
I20260709 13:04:48.655210 10847 ts_tablet_manager.cc:1434] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:04:48.659047 10847 tablet_bootstrap.cc:492] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:48.647318 10938 raft_consensus.cc:493] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.662019 10938 raft_consensus.cc:3060] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.662796 10850 raft_consensus.cc:359] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.663646 10850 raft_consensus.cc:385] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.663985 10850 raft_consensus.cc:740] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.664851 10850 consensus_queue.cc:260] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.666026 10847 tablet_bootstrap.cc:654] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.667713 10850 ts_tablet_manager.cc:1434] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 13:04:48.669363 10850 tablet_bootstrap.cc:492] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:48.677460 10874 raft_consensus.cc:493] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.678196 10874 raft_consensus.cc:515] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.683370 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4d2b26b86c4f22976db2fc47a3ec1a" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:48.683724 10575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4d2b26b86c4f22976db2fc47a3ec1a" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:48.684900 10575 raft_consensus.cc:2468] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:48.687647 10618 leader_election.cc:304] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
W20260709 13:04:48.698927 10619 leader_election.cc:343] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.689989 10874 leader_election.cc:290] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:48.708001 10850 tablet_bootstrap.cc:654] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.712512 10905 raft_consensus.cc:493] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.713131 10905 raft_consensus.cc:515] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.718120 10874 raft_consensus.cc:493] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.718341 10925 raft_consensus.cc:493] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.718842 10874 raft_consensus.cc:515] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.718956 10925 raft_consensus.cc:515] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.721737 10847 tablet_bootstrap.cc:492] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:48.721856 10905 leader_election.cc:290] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:48.722389 10847 ts_tablet_manager.cc:1403] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.006s
I20260709 13:04:48.719367 11040 raft_consensus.cc:2804] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.722594 11000 consensus_queue.cc:1048] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:48.724211 11040 raft_consensus.cc:493] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.726773 11040 raft_consensus.cc:3060] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.730702 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cfb8e7fc7e448f8b02ff6992503bc35" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:48.732636 10847 raft_consensus.cc:359] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.732779 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cfb8e7fc7e448f8b02ff6992503bc35" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:48.735255 10847 raft_consensus.cc:385] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.721416 10925 leader_election.cc:290] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:48.738232 10848 tablet_bootstrap.cc:492] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:48.738816 10848 ts_tablet_manager.cc:1403] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.170s	user 0.011s	sys 0.022s
I20260709 13:04:48.735673 10847 raft_consensus.cc:740] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.700294 10967 raft_consensus.cc:515] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.740842 10847 consensus_queue.cc:260] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.745927 10650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e728368868473181686e35396f3ae5" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:48.746801 10650 raft_consensus.cc:2468] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:48.747521 10938 raft_consensus.cc:515] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.754752 10967 leader_election.cc:290] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:48.756701 10874 leader_election.cc:290] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:48.750344 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e728368868473181686e35396f3ae5" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:48.758729 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f96bbe7d674a5485410f0aa8ec40e6" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
W20260709 13:04:48.751351 10693 leader_election.cc:343] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.761157 10973 raft_consensus.cc:515] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:48.761991 10846 tablet_bootstrap.cc:492] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:48.797168 10846 ts_tablet_manager.cc:1403] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.230s	user 0.034s	sys 0.001s
I20260709 13:04:48.752054 10848 raft_consensus.cc:359] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.798054 10848 raft_consensus.cc:385] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.798652 10848 raft_consensus.cc:740] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.799499 10848 consensus_queue.cc:260] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.774261 10847 ts_tablet_manager.cc:1434] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.051s	user 0.010s	sys 0.000s
I20260709 13:04:48.803069 10847 tablet_bootstrap.cc:492] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:48.802631 10846 raft_consensus.cc:359] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.803658 10846 raft_consensus.cc:385] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.804013 10846 raft_consensus.cc:740] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.804939 10846 consensus_queue.cc:260] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.776748 10693 leader_election.cc:304] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:48.780618 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f96bbe7d674a5485410f0aa8ec40e6" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:48.774749 10578 raft_consensus.cc:2468] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:48.783962 10576 raft_consensus.cc:3060] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.784032 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2988fb83d740c6a6b692c5fc8d8105" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:48.785638 10949 raft_consensus.cc:493] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.785166 10938 leader_election.cc:290] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:48.774212 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e728368868473181686e35396f3ae5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
W20260709 13:04:48.795639 11034 log.cc:927] Time spent T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:48.795349 10575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e728368868473181686e35396f3ae5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:48.808561 10848 ts_tablet_manager.cc:1434] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.069s	user 0.009s	sys 0.000s
I20260709 13:04:48.812925 10973 leader_election.cc:290] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:48.815332 10846 ts_tablet_manager.cc:1434] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:04:48.810388 10928 raft_consensus.cc:2804] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.814157 10798 raft_consensus.cc:3060] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:48.815429 10693 leader_election.cc:343] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.829177 10575 raft_consensus.cc:2468] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:48.819515 10949 raft_consensus.cc:515] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.832587 10846 tablet_bootstrap.cc:492] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:48.828662 10928 raft_consensus.cc:493] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.857833 10928 raft_consensus.cc:3060] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.842814 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:48.862843 10726 raft_consensus.cc:3060] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.863159 10850 tablet_bootstrap.cc:492] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:48.863906 10850 ts_tablet_manager.cc:1403] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.195s	user 0.030s	sys 0.023s
I20260709 13:04:48.844167 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:48.866918 10578 raft_consensus.cc:3060] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.870275 10850 raft_consensus.cc:359] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:48.871939 10850 raft_consensus.cc:385] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:48.839727 10693 leader_election.cc:304] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:48.846386 10949 leader_election.cc:290] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:48.850111 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2988fb83d740c6a6b692c5fc8d8105" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:48.873431 10574 raft_consensus.cc:3060] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.875594 10967 raft_consensus.cc:2749] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:48.878285 10949 raft_consensus.cc:493] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.878101 11040 raft_consensus.cc:515] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.879083 10949 raft_consensus.cc:515] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:48.883507 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:48.883831 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4d2b26b86c4f22976db2fc47a3ec1a" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:48.851934 10846 tablet_bootstrap.cc:654] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:48.855999 10874 consensus_queue.cc:1048] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:48.857192 10576 raft_consensus.cc:2468] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:48.872258 10850 raft_consensus.cc:740] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:48.899155 10850 consensus_queue.cc:260] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:48.905936 10619 leader_election.cc:343] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.907509 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da58d0f3541543dc8678d61df75295f9" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:48.911501 10847 tablet_bootstrap.cc:654] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:48.907475 10544 leader_election.cc:343] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.919582 10848 tablet_bootstrap.cc:492] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:48.908483 11045 raft_consensus.cc:493] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:48.925463 10544 leader_election.cc:343] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:48.930337 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 170748us
I20260709 13:04:48.933881 10949 leader_election.cc:290] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:48.941510 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da58d0f3541543dc8678d61df75295f9" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:48.954358 10850 ts_tablet_manager.cc:1434] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.090s	user 0.004s	sys 0.009s
W20260709 13:04:48.962479 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 151767us
W20260709 13:04:48.965273 10693 consensus_peers.cc:597] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:48.970888 10798 raft_consensus.cc:2468] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:48.965361 10850 tablet_bootstrap.cc:492] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:48.952235 11040 leader_election.cc:290] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:48.976670 10619 leader_election.cc:304] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:04:48.976825 10618 leader_election.cc:304] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:48.977211 10574 raft_consensus.cc:2468] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:48.989161 11048 raft_consensus.cc:493] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.990047 11048 raft_consensus.cc:515] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.992046 11044 consensus_queue.cc:1048] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:48.994514 11043 raft_consensus.cc:2804] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:48.995030 11043 raft_consensus.cc:493] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.995126 10874 raft_consensus.cc:2804] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:48.995420 11043 raft_consensus.cc:3060] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.995718 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4d2b26b86c4f22976db2fc47a3ec1a" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:48.996819 10574 raft_consensus.cc:3060] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:48.925850 11045 raft_consensus.cc:515] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:48.996827 11048 leader_election.cc:290] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
W20260709 13:04:48.995970 10693 leader_election.cc:343] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:48.995826 10874 raft_consensus.cc:697] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:49.001151 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:49.013589 10848 tablet_bootstrap.cc:654] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.004806 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca28e86e81447cb899fbb41f02a7f03" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:49.001431 11045 leader_election.cc:290] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:49.029911 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2bb2563ed2a407c9c219a632057e54a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:49.004277 11046 consensus_queue.cc:1048] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 13:04:49.011552 10578 raft_consensus.cc:2468] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
W20260709 13:04:49.008702 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 50220.5us
I20260709 13:04:49.048748 11048 raft_consensus.cc:493] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.003489 10874 consensus_queue.cc:237] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.070328 10766 leader_election.cc:304] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:49.056463 10979 raft_consensus.cc:493] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.079299 10979 raft_consensus.cc:515] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.082453 10726 raft_consensus.cc:2468] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
W20260709 13:04:49.060662 10544 leader_election.cc:343] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.084102 11000 raft_consensus.cc:2804] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.084583 10544 leader_election.cc:304] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:49.084985 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca28e86e81447cb899fbb41f02a7f03" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:49.085773 10576 raft_consensus.cc:2468] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:49.089848 10850 tablet_bootstrap.cc:654] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.058847 10692 leader_election.cc:304] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:49.074290 10928 raft_consensus.cc:515] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
W20260709 13:04:49.099437 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 45838.6us
I20260709 13:04:49.102205 10727 raft_consensus.cc:2393] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 7ecd836bb7a144dc9aa07a046239d6ce in this term.
I20260709 13:04:49.102483 10967 raft_consensus.cc:2804] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.103616 10967 raft_consensus.cc:697] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
W20260709 13:04:49.076669 10616 leader_election.cc:343] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.102586 10851 raft_consensus.cc:2749] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.125442 10925 raft_consensus.cc:493] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.126219 10925 raft_consensus.cc:515] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
W20260709 13:04:49.129354 10693 leader_election.cc:343] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.131062 10925 leader_election.cc:290] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:49.117121 10574 raft_consensus.cc:2468] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:49.129390 10618 leader_election.cc:304] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:49.134263 10575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2bb2563ed2a407c9c219a632057e54a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:49.135087 10575 raft_consensus.cc:2468] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:49.134140 10928 leader_election.cc:290] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
W20260709 13:04:49.137332 10541 leader_election.cc:343] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.147503 10541 leader_election.cc:304] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:49.150266 10851 raft_consensus.cc:2749] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.139886 10967 consensus_queue.cc:237] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:49.141633 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:49.142961 10618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 48138.2us
I20260709 13:04:49.144156 11045 raft_consensus.cc:2804] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.071681 11048 raft_consensus.cc:515] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:49.174634 10618 leader_election.cc:304] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:49.170323 11045 raft_consensus.cc:493] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.177112 11045 raft_consensus.cc:3060] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.195339 11000 raft_consensus.cc:697] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:49.196453 11056 consensus_queue.cc:1048] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:49.197463 11000 consensus_queue.cc:237] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:49.206532 10847 tablet_bootstrap.cc:492] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:49.208163 10847 ts_tablet_manager.cc:1403] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.405s	user 0.022s	sys 0.014s
I20260709 13:04:49.170333 10973 raft_consensus.cc:493] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.212344 10846 tablet_bootstrap.cc:492] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:49.212960 10846 ts_tablet_manager.cc:1403] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.381s	user 0.035s	sys 0.015s
I20260709 13:04:49.186758 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:49.182858 10692 leader_election.cc:304] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:49.230023 10848 tablet_bootstrap.cc:492] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:49.235520 10848 ts_tablet_manager.cc:1403] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.344s	user 0.034s	sys 0.012s
I20260709 13:04:49.236073 10850 tablet_bootstrap.cc:492] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:49.236737 10850 ts_tablet_manager.cc:1403] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.277s	user 0.009s	sys 0.014s
I20260709 13:04:49.213881 11043 raft_consensus.cc:515] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:49.215256 10973 raft_consensus.cc:515] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:49.248453 11043 leader_election.cc:290] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:49.248920 10973 leader_election.cc:290] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
W20260709 13:04:49.225153 10693 leader_election.cc:343] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:49.244920 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 73786.1us
I20260709 13:04:49.253446 10905 raft_consensus.cc:2804] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.254163 10905 raft_consensus.cc:493] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.254038 10846 raft_consensus.cc:359] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.254796 10905 raft_consensus.cc:3060] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.258713 10850 raft_consensus.cc:359] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:49.249991 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec71cde09fe64d089f95382f32269db5" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:49.259572 10850 raft_consensus.cc:385] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.259922 10850 raft_consensus.cc:740] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.254873 10846 raft_consensus.cc:385] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.260664 10846 raft_consensus.cc:740] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.260382 10651 raft_consensus.cc:2468] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:49.261605 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec71cde09fe64d089f95382f32269db5" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:49.261593 10846 consensus_queue.cc:260] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.262566 10725 raft_consensus.cc:2468] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:49.262580 10767 leader_election.cc:304] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:04:49.264137 11042 raft_consensus.cc:2804] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.265003 11042 raft_consensus.cc:697] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:49.266422 11042 consensus_queue.cc:237] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.269860 10979 leader_election.cc:290] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:49.240417 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e728368868473181686e35396f3ae5" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:49.271173 10653 raft_consensus.cc:2393] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7ecd836bb7a144dc9aa07a046239d6ce in current term 1: Already voted for candidate 73db933c4a6848d1b2aeb56813f7bb2c in this term.
I20260709 13:04:49.270911 10850 consensus_queue.cc:260] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:49.266418 10847 raft_consensus.cc:359] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:49.273444 10847 raft_consensus.cc:385] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.274187 10847 raft_consensus.cc:740] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.273692 10846 ts_tablet_manager.cc:1434] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.060s	user 0.010s	sys 0.000s
I20260709 13:04:49.279577 10850 ts_tablet_manager.cc:1434] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 13:04:49.279417 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e728368868473181686e35396f3ae5" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:49.280181 10574 raft_consensus.cc:3060] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.280933 10850 tablet_bootstrap.cc:492] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:49.283835 11048 leader_election.cc:290] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:49.277148 10846 tablet_bootstrap.cc:492] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:49.275136 10847 consensus_queue.cc:260] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
W20260709 13:04:49.290094 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 67106.2us
I20260709 13:04:49.295163 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48d84eceb174792919f6256ee4fbb21" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:49.296092 10652 raft_consensus.cc:2468] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:49.301882 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48d84eceb174792919f6256ee4fbb21" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:49.304713 11056 raft_consensus.cc:2804] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.305457 11056 raft_consensus.cc:493] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:49.291561 10692 leader_election.cc:343] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.332525 10847 ts_tablet_manager.cc:1434] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.124s	user 0.003s	sys 0.008s
I20260709 13:04:49.325903 11056 raft_consensus.cc:3060] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.331357 10575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e728368868473181686e35396f3ae5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:49.335655 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:49.347996 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e728368868473181686e35396f3ae5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:49.351105 10848 raft_consensus.cc:359] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.360214 10848 raft_consensus.cc:385] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.360533 10848 raft_consensus.cc:740] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.357055 10692 leader_election.cc:304] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:49.343405 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:49.364334 10725 raft_consensus.cc:2393] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 7ecd836bb7a144dc9aa07a046239d6ce in this term.
W20260709 13:04:49.366097 10544 leader_election.cc:343] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.368861 10618 leader_election.cc:304] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:49.346469 10693 leader_election.cc:304] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:49.370177 10846 tablet_bootstrap.cc:654] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.370384 11046 raft_consensus.cc:2804] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:04:49.370878 10693 leader_election.cc:343] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.371091 11046 raft_consensus.cc:493] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.371701 11046 raft_consensus.cc:3060] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.370358 10938 raft_consensus.cc:2749] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.378573 10847 tablet_bootstrap.cc:492] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:49.378862 11000 consensus_queue.cc:1048] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:04:49.383090 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 124446us
I20260709 13:04:49.383900 10928 raft_consensus.cc:2749] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.386343 10848 consensus_queue.cc:260] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.388794 10476 catalog_manager.cc:5697] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:49.392258 10848 ts_tablet_manager.cc:1434] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.156s	user 0.006s	sys 0.001s
I20260709 13:04:49.395279 10905 raft_consensus.cc:515] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:49.399709 10799 raft_consensus.cc:2468] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:49.416184 10905 leader_election.cc:290] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:49.419473 10575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2bb2563ed2a407c9c219a632057e54a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:49.419615 10544 leader_election.cc:304] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:49.420363 10575 raft_consensus.cc:3060] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.421885 11061 raft_consensus.cc:2804] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.422494 11061 raft_consensus.cc:493] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.422924 11061 raft_consensus.cc:3060] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.399477 10848 tablet_bootstrap.cc:492] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:49.343266 11045 raft_consensus.cc:515] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:49.440832 11046 raft_consensus.cc:515] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:49.452935 10574 raft_consensus.cc:2468] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:49.447921 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2bb2563ed2a407c9c219a632057e54a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:49.471433 11045 leader_election.cc:290] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:49.457083 10937 raft_consensus.cc:493] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.472821 10937 raft_consensus.cc:515] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.472743 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca28e86e81447cb899fbb41f02a7f03" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:49.474212 10726 raft_consensus.cc:3060] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.475922 10937 leader_election.cc:290] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:49.479051 10850 tablet_bootstrap.cc:654] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.484843 10726 raft_consensus.cc:2468] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
W20260709 13:04:49.490620 10693 leader_election.cc:343] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.492694 10616 leader_election.cc:304] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:49.493248 11046 leader_election.cc:290] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:49.495193 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48d84eceb174792919f6256ee4fbb21" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:49.495663 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48d84eceb174792919f6256ee4fbb21" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:49.496309 10653 raft_consensus.cc:3060] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.501135 11045 raft_consensus.cc:2804] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.502377 11045 raft_consensus.cc:697] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:49.503458 11045 consensus_queue.cc:237] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:49.507799 10474 catalog_manager.cc:5697] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:49.512682 10693 leader_election.cc:343] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.512957 10847 tablet_bootstrap.cc:654] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.515017 10692 leader_election.cc:304] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:49.517139 11046 raft_consensus.cc:2804] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.517926 11046 raft_consensus.cc:697] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:49.520501 10848 tablet_bootstrap.cc:654] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.519150 11046 consensus_queue.cc:237] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:49.520870 11056 raft_consensus.cc:515] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:49.544353 10541 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 88581.4us
W20260709 13:04:49.543440 10544 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 73054.8us
I20260709 13:04:49.556650 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95882d5aa57c46ba86be3160648fc85a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:49.559021 11056 leader_election.cc:290] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:49.559481 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95882d5aa57c46ba86be3160648fc85a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:49.560580 10575 raft_consensus.cc:2468] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:49.561530 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec71cde09fe64d089f95382f32269db5" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:49.562216 10725 raft_consensus.cc:3060] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.562465 10653 raft_consensus.cc:2468] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:49.569537 10692 leader_election.cc:304] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:49.570945 10967 raft_consensus.cc:2804] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.571504 10967 raft_consensus.cc:697] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:49.571969 10693 leader_election.cc:304] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:49.572479 10967 consensus_queue.cc:237] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:49.573199 10925 raft_consensus.cc:2804] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.573827 10925 raft_consensus.cc:697] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:49.574723 10925 consensus_queue.cc:237] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:49.545394 11061 raft_consensus.cc:515] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.585196 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec71cde09fe64d089f95382f32269db5" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:49.586045 10652 raft_consensus.cc:3060] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.587744 10846 tablet_bootstrap.cc:492] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:49.588290 10846 ts_tablet_manager.cc:1403] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.311s	user 0.010s	sys 0.020s
I20260709 13:04:49.592767 10846 raft_consensus.cc:359] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:49.593636 10846 raft_consensus.cc:385] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.593895 10846 raft_consensus.cc:740] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.594571 10846 consensus_queue.cc:260] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:49.596654 10846 ts_tablet_manager.cc:1434] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:04:49.597927 10846 tablet_bootstrap.cc:492] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:49.600132 10874 consensus_queue.cc:1048] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:49.630635 11061 leader_election.cc:290] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:49.631953 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca28e86e81447cb899fbb41f02a7f03" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:49.637815 10576 raft_consensus.cc:3060] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:49.640918 10544 leader_election.cc:343] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.635520 10850 tablet_bootstrap.cc:492] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:49.643944 10850 ts_tablet_manager.cc:1403] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.363s	user 0.021s	sys 0.002s
I20260709 13:04:49.672441 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:49.658501 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:49.677131 10798 raft_consensus.cc:3060] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.685642 10928 consensus_queue.cc:1048] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:49.687153 10725 raft_consensus.cc:2468] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:49.688632 10764 leader_election.cc:304] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:49.689831 10973 raft_consensus.cc:2804] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.690549 10973 raft_consensus.cc:697] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:49.691461 10973 consensus_queue.cc:237] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:49.698331 10544 leader_election.cc:343] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.707281 10544 leader_election.cc:304] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:49.710163 11061 raft_consensus.cc:2749] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.714908 10850 raft_consensus.cc:359] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:49.716151 10850 raft_consensus.cc:385] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.716549 10850 raft_consensus.cc:740] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.733639 10850 consensus_queue.cc:260] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:49.743276 10850 ts_tablet_manager.cc:1434] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.099s	user 0.008s	sys 0.003s
I20260709 13:04:49.750495 10848 tablet_bootstrap.cc:492] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:49.751250 10848 ts_tablet_manager.cc:1403] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.352s	user 0.027s	sys 0.021s
I20260709 13:04:49.754801 10848 raft_consensus.cc:359] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:49.755611 10848 raft_consensus.cc:385] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.755332 11045 consensus_queue.cc:1048] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:49.755906 10848 raft_consensus.cc:740] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.762956 11045 raft_consensus.cc:493] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.763626 11045 raft_consensus.cc:515] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:49.764806 10848 consensus_queue.cc:260] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:49.768904 10850 tablet_bootstrap.cc:492] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:49.774897 10652 raft_consensus.cc:2468] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
W20260709 13:04:49.776185 10693 consensus_peers.cc:597] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Couldn't send request to peer 5a12f1c20c674e7f8242df41a9acf5dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:49.777562 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b99c17dd874ae2bd745e4ed1ba0330" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:49.777815 11045 leader_election.cc:290] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:49.778419 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b99c17dd874ae2bd745e4ed1ba0330" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:49.780138 10847 tablet_bootstrap.cc:492] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:49.780064 10619 leader_election.cc:304] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:49.780335 10798 raft_consensus.cc:2468] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
I20260709 13:04:49.780862 10847 ts_tablet_manager.cc:1403] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.420s	user 0.040s	sys 0.065s
I20260709 13:04:49.781549 11045 raft_consensus.cc:2749] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:49.781911 10848 ts_tablet_manager.cc:1434] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.001s
I20260709 13:04:49.791489 10850 tablet_bootstrap.cc:654] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.798985 10846 tablet_bootstrap.cc:654] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:49.795982 10544 leader_election.cc:343] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.802956 10938 consensus_queue.cc:1048] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:49.793632 10576 raft_consensus.cc:2468] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:49.814862 10848 tablet_bootstrap.cc:492] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:49.814396 10973 consensus_queue.cc:1048] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:49.823024 10973 consensus_queue.cc:1048] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:49.879341 10848 tablet_bootstrap.cc:654] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:49.883437 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:49.870203 10473 catalog_manager.cc:5697] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:49.885121 10473 catalog_manager.cc:5697] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:49.887993 10473 catalog_manager.cc:5697] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:49.890543 10619 consensus_peers.cc:597] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Couldn't send request to peer 5a12f1c20c674e7f8242df41a9acf5dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:49.903041 10652 raft_consensus.cc:3060] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.904124 10846 tablet_bootstrap.cc:492] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:49.904724 10846 ts_tablet_manager.cc:1403] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.307s	user 0.025s	sys 0.020s
I20260709 13:04:49.879771 10544 leader_election.cc:304] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:49.910801 10846 raft_consensus.cc:359] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.916320 10846 raft_consensus.cc:385] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.916647 10846 raft_consensus.cc:740] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.919719 10846 consensus_queue.cc:260] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.922454 11074 raft_consensus.cc:2804] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.926384 11074 raft_consensus.cc:697] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
W20260709 13:04:49.928719 10544 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 81316.1us
I20260709 13:04:49.931974 10847 raft_consensus.cc:359] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:49.932883 10847 raft_consensus.cc:385] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.933180 10847 raft_consensus.cc:740] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.934027 10847 consensus_queue.cc:260] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:49.936326 10847 ts_tablet_manager.cc:1434] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.155s	user 0.005s	sys 0.001s
I20260709 13:04:49.929235 10474 catalog_manager.cc:5697] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:49.927443 11074 consensus_queue.cc:237] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:49.941308 10847 tablet_bootstrap.cc:492] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:49.941504 10846 ts_tablet_manager.cc:1434] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.000s
I20260709 13:04:49.951457 10850 tablet_bootstrap.cc:492] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:49.952220 10850 ts_tablet_manager.cc:1403] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.186s	user 0.018s	sys 0.027s
I20260709 13:04:49.964847 10846 tablet_bootstrap.cc:492] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:49.996256 10938 consensus_queue.cc:1048] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:50.004225 10850 raft_consensus.cc:359] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.006016 10850 raft_consensus.cc:385] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.006330 10850 raft_consensus.cc:740] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.012727 10850 consensus_queue.cc:260] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.011440 10475 catalog_manager.cc:5697] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.024463 10475 catalog_manager.cc:5697] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.033432 10846 tablet_bootstrap.cc:654] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:50.019484 10693 consensus_peers.cc:597] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Couldn't send request to peer 5a12f1c20c674e7f8242df41a9acf5dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:50.052771 10850 ts_tablet_manager.cc:1434] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.100s	user 0.010s	sys 0.006s
I20260709 13:04:50.055040 10475 catalog_manager.cc:5697] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.094575 10847 tablet_bootstrap.cc:654] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.089834 10851 raft_consensus.cc:493] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.096164 10851 raft_consensus.cc:515] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
W20260709 13:04:50.091691 11098 log.cc:927] Time spent T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: Append to log took a long time: real 0.066s	user 0.003s	sys 0.004s
I20260709 13:04:50.103325 10851 leader_election.cc:290] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:50.105638 10474 catalog_manager.cc:5697] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.105942 10848 tablet_bootstrap.cc:492] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:50.118634 10848 ts_tablet_manager.cc:1403] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.318s	user 0.022s	sys 0.009s
I20260709 13:04:50.110352 10850 tablet_bootstrap.cc:492] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:50.112775 10846 tablet_bootstrap.cc:492] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:50.113754 11045 consensus_queue.cc:1048] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.122121 10846 ts_tablet_manager.cc:1403] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.177s	user 0.020s	sys 0.024s
I20260709 13:04:50.126771 11046 consensus_queue.cc:1048] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.125948 10846 raft_consensus.cc:359] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:50.129963 10846 raft_consensus.cc:385] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.130281 10846 raft_consensus.cc:740] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.136865 10967 raft_consensus.cc:493] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.139634 10967 raft_consensus.cc:515] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:50.145221 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46823d8cc8df4f4bafb078e496b9a7d2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:50.161325 10846 consensus_queue.cc:260] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:50.164952 10848 raft_consensus.cc:359] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.166961 10848 raft_consensus.cc:385] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.167415 10848 raft_consensus.cc:740] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.146111 10799 raft_consensus.cc:2468] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:50.178020 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46823d8cc8df4f4bafb078e496b9a7d2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:50.172700 10905 consensus_queue.cc:1048] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
W20260709 13:04:50.182953 10541 leader_election.cc:343] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.172765 11046 consensus_queue.cc:1048] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:04:50.152175 11000 consensus_queue.cc:1048] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
W20260709 13:04:50.186798 10693 consensus_peers.cc:597] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:50.187465 10967 leader_election.cc:290] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:50.179152 10848 consensus_queue.cc:260] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.215150 10846 ts_tablet_manager.cc:1434] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.005s
I20260709 13:04:50.216691 10846 tablet_bootstrap.cc:492] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:50.219041 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da58d0f3541543dc8678d61df75295f9" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:50.220979 10693 leader_election.cc:343] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.225178 10850 tablet_bootstrap.cc:654] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.228358 10846 tablet_bootstrap.cc:654] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.231664 10544 leader_election.cc:304] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:50.236544 10848 ts_tablet_manager.cc:1434] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.118s	user 0.008s	sys 0.002s
I20260709 13:04:50.238168 10848 tablet_bootstrap.cc:492] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:50.239253 10847 tablet_bootstrap.cc:492] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:50.241096 10847 ts_tablet_manager.cc:1403] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.304s	user 0.022s	sys 0.013s
I20260709 13:04:50.246804 11074 raft_consensus.cc:2804] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.253978 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da58d0f3541543dc8678d61df75295f9" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:50.270947 10577 raft_consensus.cc:2468] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:50.258230 11074 raft_consensus.cc:493] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.272868 11074 raft_consensus.cc:3060] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.273365 10967 consensus_queue.cc:1048] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.272461 10692 leader_election.cc:304] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:50.276154 11046 raft_consensus.cc:2804] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.277081 11046 raft_consensus.cc:493] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.277644 11046 raft_consensus.cc:3060] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.302057 10851 raft_consensus.cc:493] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.314527 10851 raft_consensus.cc:515] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:50.318569 10851 leader_election.cc:290] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:50.318961 10649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:50.324469 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
W20260709 13:04:50.329270 10544 leader_election.cc:343] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:50.333314 10541 leader_election.cc:343] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.334020 10541 leader_election.cc:304] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:50.337713 10851 raft_consensus.cc:2749] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.337792 10979 raft_consensus.cc:493] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.338467 10979 raft_consensus.cc:515] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.341696 10979 leader_election.cc:290] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:50.363624 11058 consensus_queue.cc:1048] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.366969 10848 tablet_bootstrap.cc:654] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.380194 10850 tablet_bootstrap.cc:492] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:50.381009 10850 ts_tablet_manager.cc:1403] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.280s	user 0.029s	sys 0.036s
I20260709 13:04:50.381247 10847 raft_consensus.cc:359] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.382977 10847 raft_consensus.cc:385] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.383255 10847 raft_consensus.cc:740] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.384219 10847 consensus_queue.cc:260] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.384796 10850 raft_consensus.cc:359] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:50.385783 10850 raft_consensus.cc:385] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.386126 10850 raft_consensus.cc:740] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.386829 10850 consensus_queue.cc:260] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:50.387737 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:50.389535 10798 raft_consensus.cc:3060] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:50.390272 10544 leader_election.cc:343] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:50.391322 10544 consensus_peers.cc:597] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:50.392863 11074 raft_consensus.cc:515] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.396953 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46823d8cc8df4f4bafb078e496b9a7d2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:50.401419 10649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:50.400751 10541 leader_election.cc:343] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.412110 10850 ts_tablet_manager.cc:1434] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.000s
W20260709 13:04:50.418562 10544 leader_election.cc:343] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.419121 10544 leader_election.cc:304] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:50.421940 10850 tablet_bootstrap.cc:492] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:50.387641 11046 raft_consensus.cc:515] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:50.429466 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da58d0f3541543dc8678d61df75295f9" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:50.429859 11046 leader_election.cc:290] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:50.430418 10574 raft_consensus.cc:3060] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.432904 11074 leader_election.cc:290] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:50.433656 11074 raft_consensus.cc:2749] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.435238 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46823d8cc8df4f4bafb078e496b9a7d2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:50.435734 10847 ts_tablet_manager.cc:1434] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.194s	user 0.007s	sys 0.000s
I20260709 13:04:50.435950 10799 raft_consensus.cc:3060] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.436192 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da58d0f3541543dc8678d61df75295f9" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:50.437040 10847 tablet_bootstrap.cc:492] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:50.439769 10846 tablet_bootstrap.cc:492] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:50.440346 10846 ts_tablet_manager.cc:1403] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.224s	user 0.013s	sys 0.002s
I20260709 13:04:50.443843 10846 raft_consensus.cc:359] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.444635 10846 raft_consensus.cc:385] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.444911 10846 raft_consensus.cc:740] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
W20260709 13:04:50.445574 10693 leader_election.cc:343] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.446868 10799 raft_consensus.cc:2468] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
I20260709 13:04:50.448380 10544 leader_election.cc:304] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:50.449728 10851 raft_consensus.cc:2804] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.456090 10851 raft_consensus.cc:697] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:04:50.462507 10851 consensus_queue.cc:237] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.461871 10846 consensus_queue.cc:260] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.477414 10850 tablet_bootstrap.cc:654] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.478010 10846 ts_tablet_manager.cc:1434] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:04:50.486877 10938 raft_consensus.cc:493] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.488245 10938 raft_consensus.cc:515] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.493297 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cfb8e7fc7e448f8b02ff6992503bc35" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:50.493716 10574 raft_consensus.cc:2468] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:50.494333 10727 raft_consensus.cc:2468] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:50.497537 10692 leader_election.cc:304] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:50.499190 10616 leader_election.cc:304] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:50.499598 11046 raft_consensus.cc:2804] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.500581 10874 raft_consensus.cc:2804] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.500819 11046 raft_consensus.cc:697] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:50.501119 10874 raft_consensus.cc:493] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.501791 10874 raft_consensus.cc:3060] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.502017 11046 consensus_queue.cc:237] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:50.506568 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cfb8e7fc7e448f8b02ff6992503bc35" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:50.503597 10653 raft_consensus.cc:3060] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.508029 10938 leader_election.cc:290] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:50.514232 10846 tablet_bootstrap.cc:492] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
W20260709 13:04:50.518083 10619 leader_election.cc:343] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.524286 10848 tablet_bootstrap.cc:492] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:50.525111 10848 ts_tablet_manager.cc:1403] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.287s	user 0.016s	sys 0.007s
I20260709 13:04:50.525883 10847 tablet_bootstrap.cc:654] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.535135 10979 consensus_queue.cc:1048] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.548419 10850 tablet_bootstrap.cc:492] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:50.549063 10850 ts_tablet_manager.cc:1403] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.128s	user 0.021s	sys 0.001s
I20260709 13:04:50.554705 10850 raft_consensus.cc:359] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.555804 10846 tablet_bootstrap.cc:654] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.555994 10850 raft_consensus.cc:385] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.556432 10850 raft_consensus.cc:740] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.548506 10874 raft_consensus.cc:515] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.557435 10850 consensus_queue.cc:260] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.559864 10475 catalog_manager.cc:5697] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.566635 10474 catalog_manager.cc:5697] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.582103 10848 raft_consensus.cc:359] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.583141 10848 raft_consensus.cc:385] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.589489 10848 raft_consensus.cc:740] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.589039 10850 ts_tablet_manager.cc:1434] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 13:04:50.592299 10848 consensus_queue.cc:260] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.594277 10848 ts_tablet_manager.cc:1434] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.069s	user 0.010s	sys 0.000s
I20260709 13:04:50.595481 10848 tablet_bootstrap.cc:492] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:50.608491 10850 tablet_bootstrap.cc:492] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:50.611380 10874 leader_election.cc:290] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:50.613927 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cfb8e7fc7e448f8b02ff6992503bc35" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:50.614660 10798 raft_consensus.cc:3060] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.612485 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cfb8e7fc7e448f8b02ff6992503bc35" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:50.622572 10727 raft_consensus.cc:3060] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.631570 10848 tablet_bootstrap.cc:654] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.635550 10798 raft_consensus.cc:2468] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:50.644135 10846 tablet_bootstrap.cc:492] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:50.644817 10846 ts_tablet_manager.cc:1403] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.157s	user 0.019s	sys 0.011s
I20260709 13:04:50.645196 10847 tablet_bootstrap.cc:492] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:50.646251 10847 ts_tablet_manager.cc:1403] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.210s	user 0.011s	sys 0.017s
I20260709 13:04:50.648241 10846 raft_consensus.cc:359] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.649096 10846 raft_consensus.cc:385] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.649367 10846 raft_consensus.cc:740] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.650267 10846 consensus_queue.cc:260] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.652042 10619 leader_election.cc:304] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:04:50.652988 10846 ts_tablet_manager.cc:1434] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:04:50.653586 10874 raft_consensus.cc:2804] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.654289 10846 tablet_bootstrap.cc:492] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:50.654408 10874 raft_consensus.cc:697] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:50.655383 10874 consensus_queue.cc:237] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.655406 10850 tablet_bootstrap.cc:654] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.681293 10727 raft_consensus.cc:2468] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:50.683151 10848 tablet_bootstrap.cc:492] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:50.684422 10848 ts_tablet_manager.cc:1403] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.089s	user 0.012s	sys 0.010s
I20260709 13:04:50.686465 10847 raft_consensus.cc:359] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.687355 10847 raft_consensus.cc:385] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.687675 10847 raft_consensus.cc:740] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.688499 10847 consensus_queue.cc:260] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.688606 10848 raft_consensus.cc:359] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.689779 10848 raft_consensus.cc:385] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.690104 10848 raft_consensus.cc:740] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.695686 10846 tablet_bootstrap.cc:654] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.696897 11061 raft_consensus.cc:493] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.697590 11061 raft_consensus.cc:515] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.700244 11061 leader_election.cc:290] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:50.702062 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:50.702353 10650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:50.703831 10544 leader_election.cc:343] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.708622 10847 ts_tablet_manager.cc:1434] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.062s	user 0.001s	sys 0.008s
W20260709 13:04:50.711274 10544 leader_election.cc:343] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.711843 10544 leader_election.cc:304] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:50.712241 10850 tablet_bootstrap.cc:492] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:50.712798 10850 ts_tablet_manager.cc:1403] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.105s	user 0.012s	sys 0.009s
I20260709 13:04:50.713080 11061 raft_consensus.cc:2749] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.716653 10850 raft_consensus.cc:359] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:50.717511 10850 raft_consensus.cc:385] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.718016 10850 raft_consensus.cc:740] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.721153 10850 consensus_queue.cc:260] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:50.716456 10848 consensus_queue.cc:260] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.726794 10850 ts_tablet_manager.cc:1434] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.003s
I20260709 13:04:50.730330 10847 tablet_bootstrap.cc:492] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:50.737847 10850 tablet_bootstrap.cc:492] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:50.745765 10850 tablet_bootstrap.cc:654] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.746558 10848 ts_tablet_manager.cc:1434] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.004s
I20260709 13:04:50.751494 10847 tablet_bootstrap.cc:654] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.755201 10850 tablet_bootstrap.cc:492] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:50.762048 10850 ts_tablet_manager.cc:1403] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.004s
I20260709 13:04:50.768168 10850 raft_consensus.cc:359] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.762864 10979 raft_consensus.cc:493] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.757341 10474 catalog_manager.cc:5697] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.770414 10848 tablet_bootstrap.cc:492] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:50.770275 10979 raft_consensus.cc:515] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.771742 10847 tablet_bootstrap.cc:492] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:50.772902 10847 ts_tablet_manager.cc:1403] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.002s
I20260709 13:04:50.776273 10847 raft_consensus.cc:359] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.776535 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69cb859af9704c2596574e50f238ce52" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:50.777083 10847 raft_consensus.cc:385] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.777495 10847 raft_consensus.cc:740] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.769780 10850 raft_consensus.cc:385] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.778482 10850 raft_consensus.cc:740] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.780252 10848 tablet_bootstrap.cc:654] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.780148 10850 consensus_queue.cc:260] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.781164 10846 tablet_bootstrap.cc:492] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:50.781338 11058 raft_consensus.cc:493] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.781724 10846 ts_tablet_manager.cc:1403] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.128s	user 0.025s	sys 0.036s
I20260709 13:04:50.781891 11058 raft_consensus.cc:515] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:50.783211 10544 leader_election.cc:343] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.783377 10850 ts_tablet_manager.cc:1434] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.001s
I20260709 13:04:50.784904 10846 raft_consensus.cc:359] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.785301 11058 leader_election.cc:290] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:50.785682 10846 raft_consensus.cc:385] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.786008 10846 raft_consensus.cc:740] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.786162 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48d84eceb174792919f6256ee4fbb21" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:50.787074 10846 consensus_queue.cc:260] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.788333 10650 raft_consensus.cc:3060] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.778431 10847 consensus_queue.cc:260] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.790788 10847 ts_tablet_manager.cc:1434] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:04:50.791222 10846 ts_tablet_manager.cc:1434] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:04:50.792361 10847 tablet_bootstrap.cc:492] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:50.792889 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48d84eceb174792919f6256ee4fbb21" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:50.794875 10764 leader_election.cc:304] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:50.796337 11058 raft_consensus.cc:2749] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.798523 10967 raft_consensus.cc:493] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.799086 10967 raft_consensus.cc:515] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.802583 10967 leader_election.cc:290] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:50.804040 10846 tablet_bootstrap.cc:492] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:50.806089 10848 tablet_bootstrap.cc:492] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:50.808574 10848 ts_tablet_manager.cc:1403] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.000s
I20260709 13:04:50.808547 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add2b1482b6e41b3ba139426c7da8f53" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:50.809641 10577 raft_consensus.cc:2468] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:50.809857 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69cb859af9704c2596574e50f238ce52" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:50.811367 10846 tablet_bootstrap.cc:654] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.812180 10848 raft_consensus.cc:359] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.812492 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add2b1482b6e41b3ba139426c7da8f53" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:50.812937 10848 raft_consensus.cc:385] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.813302 10848 raft_consensus.cc:740] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.813794 10692 leader_election.cc:304] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:50.814015 10848 consensus_queue.cc:260] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.814730 10847 tablet_bootstrap.cc:654] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.815891 10967 raft_consensus.cc:2804] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.816273 10967 raft_consensus.cc:493] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.816601 10967 raft_consensus.cc:3060] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:50.816895 10693 leader_election.cc:343] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:50.823534 10541 leader_election.cc:343] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.824199 10541 leader_election.cc:304] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:50.826223 10847 tablet_bootstrap.cc:492] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:50.826776 10979 leader_election.cc:290] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:50.827229 10847 ts_tablet_manager.cc:1403] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.004s
I20260709 13:04:50.826150 11061 raft_consensus.cc:2749] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.830868 10847 raft_consensus.cc:359] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.831744 10847 raft_consensus.cc:385] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.832042 10847 raft_consensus.cc:740] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.835307 10846 tablet_bootstrap.cc:492] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:50.836192 10846 ts_tablet_manager.cc:1403] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.000s
I20260709 13:04:50.841384 10847 consensus_queue.cc:260] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.840617 10846 raft_consensus.cc:359] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.843353 10848 ts_tablet_manager.cc:1434] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.003s
I20260709 13:04:50.843667 10846 raft_consensus.cc:385] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.844014 10846 raft_consensus.cc:740] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.846174 10847 ts_tablet_manager.cc:1434] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.004s
I20260709 13:04:50.847436 10847 tablet_bootstrap.cc:492] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:50.854427 10847 tablet_bootstrap.cc:654] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.857352 10846 consensus_queue.cc:260] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.859532 10846 ts_tablet_manager.cc:1434] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 13:04:50.860742 10846 tablet_bootstrap.cc:492] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:50.863698 10848 tablet_bootstrap.cc:492] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:50.863843 10874 raft_consensus.cc:493] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.864532 10874 raft_consensus.cc:515] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:50.866905 10874 leader_election.cc:290] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:50.867583 10846 tablet_bootstrap.cc:654] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.868363 10967 raft_consensus.cc:515] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.869010 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da58d0f3541543dc8678d61df75295f9" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:50.869885 10574 raft_consensus.cc:2393] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 7ecd836bb7a144dc9aa07a046239d6ce in this term.
I20260709 13:04:50.871026 10967 leader_election.cc:290] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:50.871577 10848 tablet_bootstrap.cc:654] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.878085 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add2b1482b6e41b3ba139426c7da8f53" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:50.882701 10850 tablet_bootstrap.cc:492] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:50.893246 11061 raft_consensus.cc:493] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.894093 11061 raft_consensus.cc:515] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.900615 11061 leader_election.cc:290] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:50.902580 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:50.907292 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:50.908152 10727 raft_consensus.cc:2468] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:50.909404 10541 leader_election.cc:304] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:50.910530 11061 raft_consensus.cc:2804] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.910948 11061 raft_consensus.cc:493] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.911264 11061 raft_consensus.cc:3060] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.914678 10928 raft_consensus.cc:493] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.915987 11046 raft_consensus.cc:493] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.915685 10928 raft_consensus.cc:515] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.916543 11046 raft_consensus.cc:515] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:50.891705 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add2b1482b6e41b3ba139426c7da8f53" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:50.918020 10577 raft_consensus.cc:3060] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:50.914775 10544 leader_election.cc:343] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.923588 10649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:50.925982 10693 leader_election.cc:343] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:50.927168 10693 leader_election.cc:343] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.927915 11046 leader_election.cc:290] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:50.932969 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:50.923934 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da58d0f3541543dc8678d61df75295f9" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:50.948218 10850 tablet_bootstrap.cc:654] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.948956 10616 leader_election.cc:304] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:50.951917 10874 raft_consensus.cc:3060] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:50.953670 10693 leader_election.cc:343] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.955109 10693 leader_election.cc:304] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:50.957691 10967 raft_consensus.cc:2749] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.959444 10928 leader_election.cc:290] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:50.963718 11058 raft_consensus.cc:493] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.964330 11058 raft_consensus.cc:515] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.969149 11058 leader_election.cc:290] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:50.969204 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95882d5aa57c46ba86be3160648fc85a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:50.970104 10652 raft_consensus.cc:2468] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:50.971563 10767 leader_election.cc:304] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:04:50.972508 11058 raft_consensus.cc:2804] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.972906 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95882d5aa57c46ba86be3160648fc85a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:50.973758 10574 raft_consensus.cc:2468] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:50.972903 11058 raft_consensus.cc:493] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.974627 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46823d8cc8df4f4bafb078e496b9a7d2" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
W20260709 13:04:50.974879 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:50.975056 11058 raft_consensus.cc:3060] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.980458 11061 raft_consensus.cc:515] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:50.983954 10874 raft_consensus.cc:2749] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.985880 11058 raft_consensus.cc:515] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:50.987620 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:50.988336 10727 raft_consensus.cc:3060] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.988488 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:50.989449 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95882d5aa57c46ba86be3160648fc85a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:50.990098 10573 raft_consensus.cc:3060] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:50.990862 10544 leader_election.cc:343] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.987159 11061 leader_election.cc:290] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:51.003430 10846 tablet_bootstrap.cc:492] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:51.004122 10846 ts_tablet_manager.cc:1403] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.144s	user 0.027s	sys 0.015s
I20260709 13:04:51.005072 10847 tablet_bootstrap.cc:492] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:51.006404 10847 ts_tablet_manager.cc:1403] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.159s	user 0.041s	sys 0.030s
I20260709 13:04:51.007488 10846 raft_consensus.cc:359] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.008255 10846 raft_consensus.cc:385] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.008536 10846 raft_consensus.cc:740] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.009232 10846 consensus_queue.cc:260] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.011819 10846 ts_tablet_manager.cc:1434] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:04:51.012352 10727 raft_consensus.cc:2468] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
I20260709 13:04:51.013212 10846 tablet_bootstrap.cc:492] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:51.013859 10541 leader_election.cc:304] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:51.014926 11074 raft_consensus.cc:2804] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.015466 11074 raft_consensus.cc:697] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:04:51.016323 11074 consensus_queue.cc:237] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:51.019162 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46823d8cc8df4f4bafb078e496b9a7d2" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:51.020119 10798 raft_consensus.cc:2393] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7ecd836bb7a144dc9aa07a046239d6ce in current term 1: Already voted for candidate bef3584ebb2c4a2ea619936c71900230 in this term.
I20260709 13:04:51.020946 10846 tablet_bootstrap.cc:654] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.033699 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95882d5aa57c46ba86be3160648fc85a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:51.034137 10693 leader_election.cc:304] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:51.034864 10651 raft_consensus.cc:3060] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.035192 11133 raft_consensus.cc:3060] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.009413 10847 raft_consensus.cc:359] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.042618 10847 raft_consensus.cc:385] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.043078 10847 raft_consensus.cc:740] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.044299 10847 consensus_queue.cc:260] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.048276 10847 ts_tablet_manager.cc:1434] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 13:04:51.048743 10850 tablet_bootstrap.cc:492] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:51.049340 10850 ts_tablet_manager.cc:1403] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.265s	user 0.012s	sys 0.007s
I20260709 13:04:51.049571 10847 tablet_bootstrap.cc:492] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:51.054423 10573 raft_consensus.cc:2468] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:51.056849 10766 leader_election.cc:304] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:51.058440 10847 tablet_bootstrap.cc:654] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.058533 11065 raft_consensus.cc:2804] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.059186 11065 raft_consensus.cc:697] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:51.060238 11065 consensus_queue.cc:237] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:51.079013 10798 raft_consensus.cc:3060] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.087139 10928 consensus_queue.cc:1048] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.089771 11074 consensus_queue.cc:1048] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.091742 10475 catalog_manager.cc:5697] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.104534 10577 raft_consensus.cc:2468] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:51.114024 11133 raft_consensus.cc:2749] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.120880 10851 raft_consensus.cc:493] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.121393 10692 leader_election.cc:304] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:51.121899 10851 raft_consensus.cc:515] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.124723 11046 raft_consensus.cc:2804] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.125360 11046 raft_consensus.cc:697] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:51.125743 10851 leader_election.cc:290] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:51.126417 10979 raft_consensus.cc:493] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.126529 11046 consensus_queue.cc:237] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:51.127106 10979 raft_consensus.cc:515] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:51.133244 10928 consensus_queue.cc:1048] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.134850 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:51.138290 10544 leader_election.cc:343] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.140725 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:51.144132 10979 leader_election.cc:290] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:51.146395 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be849d4cbf0b4f7a99f9935a55611658" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:51.147825 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be849d4cbf0b4f7a99f9935a55611658" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:51.151861 10544 leader_election.cc:343] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:51.152022 10541 leader_election.cc:343] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.152742 10541 leader_election.cc:304] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
W20260709 13:04:51.153008 10544 leader_election.cc:343] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.153667 10544 leader_election.cc:304] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:51.154125 10979 raft_consensus.cc:2749] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.154511 11074 raft_consensus.cc:2749] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.132786 10851 consensus_queue.cc:1048] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:04:51.158854 10850 raft_consensus.cc:359] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:51.160344 10850 raft_consensus.cc:385] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.161887 11058 leader_election.cc:290] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:51.160730 10850 raft_consensus.cc:740] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.188325 10850 consensus_queue.cc:260] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:51.188516 10474 catalog_manager.cc:5697] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:51.194594 10848 tablet_bootstrap.cc:492] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:51.195268 10848 ts_tablet_manager.cc:1403] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.332s	user 0.012s	sys 0.001s
I20260709 13:04:51.198534 10848 raft_consensus.cc:359] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:51.199062 10846 tablet_bootstrap.cc:492] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:51.199316 10848 raft_consensus.cc:385] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.199928 10848 raft_consensus.cc:740] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.200248 10846 ts_tablet_manager.cc:1403] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.187s	user 0.047s	sys 0.074s
I20260709 13:04:51.200730 10848 consensus_queue.cc:260] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:51.203506 10848 ts_tablet_manager.cc:1434] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:51.204717 10848 tablet_bootstrap.cc:492] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:51.210173 10850 ts_tablet_manager.cc:1434] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.160s	user 0.010s	sys 0.000s
I20260709 13:04:51.212697 10846 raft_consensus.cc:359] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.213637 10846 raft_consensus.cc:385] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.214075 10846 raft_consensus.cc:740] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.215188 10846 consensus_queue.cc:260] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.217988 10850 tablet_bootstrap.cc:492] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:51.218873 10847 tablet_bootstrap.cc:492] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:51.219415 10847 ts_tablet_manager.cc:1403] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.170s	user 0.019s	sys 0.028s
I20260709 13:04:51.227355 11058 raft_consensus.cc:493] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.228292 11058 raft_consensus.cc:515] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:51.219046 11106 raft_consensus.cc:493] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.229218 11106 raft_consensus.cc:515] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:51.231504 11106 leader_election.cc:290] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:51.233007 10651 raft_consensus.cc:2468] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:51.233664 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add2b1482b6e41b3ba139426c7da8f53" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:51.235219 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413b57e5acb54f9b86a0741e6f235ad3" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:51.240090 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413b57e5acb54f9b86a0741e6f235ad3" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:51.240762 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add2b1482b6e41b3ba139426c7da8f53" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:51.242147 10577 raft_consensus.cc:2393] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 1: Already voted for candidate 7ecd836bb7a144dc9aa07a046239d6ce in this term.
I20260709 13:04:51.243381 10766 leader_election.cc:304] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:51.245683 11058 leader_election.cc:290] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:51.246649 11058 raft_consensus.cc:3060] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.251449 10905 raft_consensus.cc:493] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.252131 10905 raft_consensus.cc:515] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:51.255738 10619 leader_election.cc:343] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.258177 10473 catalog_manager.cc:5697] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.260185 10905 leader_election.cc:290] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:51.268664 10846 ts_tablet_manager.cc:1434] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.001s
I20260709 13:04:51.294024 10846 tablet_bootstrap.cc:492] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:51.344242 10979 raft_consensus.cc:493] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.332046 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:51.345304 10979 raft_consensus.cc:515] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.389132 10847 raft_consensus.cc:359] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:51.390447 10847 raft_consensus.cc:385] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.390805 10847 raft_consensus.cc:740] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.391710 10847 consensus_queue.cc:260] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:51.405197 11154 consensus_queue.cc:1048] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.417445 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:51.420833 10850 tablet_bootstrap.cc:654] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.422479 10846 tablet_bootstrap.cc:654] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.424535 10848 tablet_bootstrap.cc:654] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.424786 10726 raft_consensus.cc:2468] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:51.420667 11058 raft_consensus.cc:2749] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.437819 10847 ts_tablet_manager.cc:1434] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.218s	user 0.008s	sys 0.003s
I20260709 13:04:51.450695 10847 tablet_bootstrap.cc:492] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:51.452345 10979 leader_election.cc:290] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
W20260709 13:04:51.431103 10693 leader_election.cc:343] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:51.493673 10693 leader_election.cc:343] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.498796 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6f6d7fae4f453994e7b12e063b6b49" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:51.501870 11154 consensus_queue.cc:1048] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:04:51.510187 10541 leader_election.cc:343] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.498311 10693 leader_election.cc:304] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:51.514854 11133 raft_consensus.cc:2749] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.520820 10847 tablet_bootstrap.cc:654] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.523190 11065 consensus_queue.cc:1048] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.530365 10848 tablet_bootstrap.cc:492] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:51.531189 10848 ts_tablet_manager.cc:1403] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.327s	user 0.017s	sys 0.004s
I20260709 13:04:51.532732 10850 tablet_bootstrap.cc:492] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:51.533450 10850 ts_tablet_manager.cc:1403] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.316s	user 0.018s	sys 0.003s
W20260709 13:04:51.542289 10764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 112650us
I20260709 13:04:51.544422 10847 tablet_bootstrap.cc:492] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:51.547343 10847 ts_tablet_manager.cc:1403] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.097s	user 0.016s	sys 0.002s
I20260709 13:04:51.553316 10847 raft_consensus.cc:359] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:51.554859 10847 raft_consensus.cc:385] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:51.539366 10544 consensus_peers.cc:597] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 -> Peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Couldn't send request to peer 5a12f1c20c674e7f8242df41a9acf5dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:51.555205 10847 raft_consensus.cc:740] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
W20260709 13:04:51.554185 10616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 60626.5us
I20260709 13:04:51.568516 10616 leader_election.cc:304] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:51.568640 10847 consensus_queue.cc:260] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:51.570188 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6f6d7fae4f453994e7b12e063b6b49" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:51.570526 11163 raft_consensus.cc:2804] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.570814 10847 ts_tablet_manager.cc:1434] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 13:04:51.570971 11163 raft_consensus.cc:493] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.571555 11163 raft_consensus.cc:3060] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.572157 10847 tablet_bootstrap.cc:492] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:51.581705 10848 raft_consensus.cc:359] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:51.582918 10848 raft_consensus.cc:385] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.583504 10848 raft_consensus.cc:740] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.583659 11163 raft_consensus.cc:515] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:51.584558 10848 consensus_queue.cc:260] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:51.587412 10848 ts_tablet_manager.cc:1434] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.000s
I20260709 13:04:51.569077 10850 raft_consensus.cc:359] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.591027 10850 raft_consensus.cc:385] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.591441 10850 raft_consensus.cc:740] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.592528 10850 consensus_queue.cc:260] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.595674 10850 ts_tablet_manager.cc:1434] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.002s
W20260709 13:04:51.590075 10544 leader_election.cc:343] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.598100 10544 leader_election.cc:304] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:51.599234 11156 raft_consensus.cc:2749] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:51.603034 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 89304us
I20260709 13:04:51.597386 10848 tablet_bootstrap.cc:492] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:51.607034 11133 raft_consensus.cc:493] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.607880 11133 raft_consensus.cc:515] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.612188 11133 leader_election.cc:290] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:51.615152 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:51.634806 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413b57e5acb54f9b86a0741e6f235ad3" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:51.635293 10574 raft_consensus.cc:2468] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:51.637188 10692 leader_election.cc:304] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:51.638260 11133 raft_consensus.cc:2804] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.638798 11133 raft_consensus.cc:493] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.637192 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413b57e5acb54f9b86a0741e6f235ad3" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:51.624136 11156 consensus_queue.cc:1048] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.624627 10928 consensus_queue.cc:1048] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.639873 10724 raft_consensus.cc:3060] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.644065 10850 tablet_bootstrap.cc:492] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:51.646828 11161 consensus_queue.cc:1048] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.648795 10928 raft_consensus.cc:493] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.649394 10928 raft_consensus.cc:515] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:51.657092 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:51.661860 10928 leader_election.cc:290] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
W20260709 13:04:51.662460 10619 leader_election.cc:343] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.664386 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
W20260709 13:04:51.666210 10692 leader_election.cc:343] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.669269 11154 raft_consensus.cc:493] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.669947 11154 raft_consensus.cc:515] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.672914 11154 leader_election.cc:290] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:51.632881 11163 leader_election.cc:290] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:51.639264 11133 raft_consensus.cc:3060] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.674703 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:51.617180 10846 tablet_bootstrap.cc:492] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:51.677947 10846 ts_tablet_manager.cc:1403] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.404s	user 0.012s	sys 0.027s
W20260709 13:04:51.677248 10616 leader_election.cc:343] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.691936 10850 tablet_bootstrap.cc:654] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.691975 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:51.697962 10573 raft_consensus.cc:2468] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:51.700654 10618 leader_election.cc:304] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:51.702237 10846 raft_consensus.cc:359] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:51.704964 10846 raft_consensus.cc:385] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.705298 10846 raft_consensus.cc:740] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.705782 11163 raft_consensus.cc:2804] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.699697 10724 raft_consensus.cc:2468] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:51.706195 10846 consensus_queue.cc:260] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:51.706836 11163 raft_consensus.cc:493] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.715747 10847 tablet_bootstrap.cc:654] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.717338 10848 tablet_bootstrap.cc:654] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.719123 10846 ts_tablet_manager.cc:1434] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 13:04:51.714923 11163 raft_consensus.cc:3060] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.736616 10616 leader_election.cc:304] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:51.737926 11106 raft_consensus.cc:2804] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.738511 11106 raft_consensus.cc:697] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:51.739408 11106 consensus_queue.cc:237] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:51.716588 10693 leader_election.cc:343] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:51.764581 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 49545.1us
I20260709 13:04:51.768834 11152 consensus_queue.cc:1048] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.776499 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:51.803582 10846 tablet_bootstrap.cc:492] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:51.868403 11133 raft_consensus.cc:515] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.871338 11133 leader_election.cc:290] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:51.875351 10848 tablet_bootstrap.cc:492] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:51.876415 10848 ts_tablet_manager.cc:1403] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.279s	user 0.052s	sys 0.022s
I20260709 13:04:51.875254 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:51.878791 10576 raft_consensus.cc:3060] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.897727 10848 raft_consensus.cc:359] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:51.898926 10848 raft_consensus.cc:385] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.899358 10848 raft_consensus.cc:740] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.900308 10848 consensus_queue.cc:260] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:51.910504 10848 ts_tablet_manager.cc:1434] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.001s
I20260709 13:04:51.911201 10847 tablet_bootstrap.cc:492] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:51.911846 10847 ts_tablet_manager.cc:1403] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.340s	user 0.017s	sys 0.009s
W20260709 13:04:51.890702 10693 leader_election.cc:343] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.923837 10846 tablet_bootstrap.cc:654] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.928963 10847 raft_consensus.cc:359] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.931317 10847 raft_consensus.cc:385] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.930698 10474 catalog_manager.cc:5697] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.938966 10850 tablet_bootstrap.cc:492] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:51.931632 10847 raft_consensus.cc:740] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.940661 10847 consensus_queue.cc:260] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.943089 10847 ts_tablet_manager.cc:1434] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 13:04:51.944600 10847 tablet_bootstrap.cc:492] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:51.939749 10850 ts_tablet_manager.cc:1403] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.296s	user 0.046s	sys 0.055s
I20260709 13:04:51.948458 10848 tablet_bootstrap.cc:492] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:51.924510 10693 leader_election.cc:304] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:51.954084 11161 raft_consensus.cc:493] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.933849 11163 raft_consensus.cc:515] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:51.952618 10576 raft_consensus.cc:2468] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:51.965373 11133 raft_consensus.cc:2749] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.968678 11074 raft_consensus.cc:493] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.982024 11074 raft_consensus.cc:515] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:51.985666 11074 leader_election.cc:290] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:51.975663 10692 leader_election.cc:304] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:51.984265 11163 leader_election.cc:290] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:51.987069 11152 raft_consensus.cc:2804] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.989728 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:51.989754 11152 raft_consensus.cc:697] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
W20260709 13:04:51.991815 10544 leader_election.cc:343] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.967705 11161 raft_consensus.cc:515] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:51.996558 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:51.997308 10574 raft_consensus.cc:3060] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.997933 11161 leader_election.cc:290] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:51.992771 11156 raft_consensus.cc:493] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.998750 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be849d4cbf0b4f7a99f9935a55611658" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:51.998941 11156 raft_consensus.cc:515] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:04:52.000430 10767 leader_election.cc:343] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.986661 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:52.004285 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be849d4cbf0b4f7a99f9935a55611658" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:51.991418 11152 consensus_queue.cc:237] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:52.006424 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:51.971099 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 81763.8us
W20260709 13:04:52.009100 10544 leader_election.cc:343] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.012089 10544 leader_election.cc:304] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:52.011655 10847 tablet_bootstrap.cc:654] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.020205 10850 raft_consensus.cc:359] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.022346 10850 raft_consensus.cc:385] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.023573 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:52.016604 11156 leader_election.cc:290] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:52.025857 10649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:52.027225 11175 log.cc:927] Time spent T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:04:52.014655 11074 raft_consensus.cc:2749] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:52.010831 10578 raft_consensus.cc:2468] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:52.022612 10850 raft_consensus.cc:740] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
W20260709 13:04:52.029464 10616 leader_election.cc:343] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:52.031180 10544 leader_election.cc:343] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.031280 10850 consensus_queue.cc:260] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:52.032251 10544 leader_election.cc:343] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.032610 10544 leader_election.cc:304] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:52.033785 11074 raft_consensus.cc:2749] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:52.036113 10766 leader_election.cc:304] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:52.037668 11161 raft_consensus.cc:2804] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.038151 11161 raft_consensus.cc:493] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.038564 11161 raft_consensus.cc:3060] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.042351 10850 ts_tablet_manager.cc:1434] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.096s	user 0.006s	sys 0.003s
I20260709 13:04:52.046470 10574 raft_consensus.cc:2468] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:52.074048 10850 tablet_bootstrap.cc:492] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:52.058655 10846 tablet_bootstrap.cc:492] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:52.053567 10618 leader_election.cc:304] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:52.055312 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:52.078363 10846 ts_tablet_manager.cc:1403] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.290s	user 0.021s	sys 0.019s
I20260709 13:04:52.079866 11163 raft_consensus.cc:2804] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:52.080402 11163 raft_consensus.cc:697] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:52.081321 11163 consensus_queue.cc:237] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:52.085410 10476 catalog_manager.cc:5697] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:52.096256 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:52.103549 10846 raft_consensus.cc:359] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.106597 10846 raft_consensus.cc:385] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.107470 10846 raft_consensus.cc:740] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.053395 11161 raft_consensus.cc:515] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.108362 10846 consensus_queue.cc:260] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.112131 10846 ts_tablet_manager.cc:1434] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.005s
I20260709 13:04:52.113550 10848 tablet_bootstrap.cc:654] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.114204 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be849d4cbf0b4f7a99f9935a55611658" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:52.121577 11161 leader_election.cc:290] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:52.122996 10578 raft_consensus.cc:3060] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.126076 10850 tablet_bootstrap.cc:654] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.129456 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be849d4cbf0b4f7a99f9935a55611658" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
W20260709 13:04:52.133726 10767 leader_election.cc:343] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.139262 10799 raft_consensus.cc:3060] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.155890 10847 tablet_bootstrap.cc:492] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:52.156564 10847 ts_tablet_manager.cc:1403] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.212s	user 0.021s	sys 0.003s
I20260709 13:04:52.158236 10846 tablet_bootstrap.cc:492] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:52.160279 10847 raft_consensus.cc:359] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:52.161245 10847 raft_consensus.cc:385] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.163512 10847 raft_consensus.cc:740] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.164966 10848 tablet_bootstrap.cc:492] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:52.165652 10848 ts_tablet_manager.cc:1403] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.227s	user 0.021s	sys 0.009s
W20260709 13:04:52.167847 10693 leader_election.cc:343] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.170622 10848 raft_consensus.cc:359] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.171489 10848 raft_consensus.cc:385] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.171777 10848 raft_consensus.cc:740] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.174546 10848 consensus_queue.cc:260] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.191944 10847 consensus_queue.cc:260] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:52.194942 10848 ts_tablet_manager.cc:1434] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.001s
W20260709 13:04:52.197804 10888 log.cc:927] Time spent T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292: Append to log took a long time: real 0.066s	user 0.001s	sys 0.001s
I20260709 13:04:52.212435 10848 tablet_bootstrap.cc:492] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:52.215884 10474 catalog_manager.cc:5697] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.220901 10578 raft_consensus.cc:2468] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:52.228420 11074 raft_consensus.cc:493] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.226310 10848 tablet_bootstrap.cc:654] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.229286 11074 raft_consensus.cc:515] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.239044 10847 ts_tablet_manager.cc:1434] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.082s	user 0.010s	sys 0.000s
I20260709 13:04:52.237287 10766 leader_election.cc:304] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:52.243391 11065 raft_consensus.cc:2804] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:52.244242 11065 raft_consensus.cc:697] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:52.244411 10847 tablet_bootstrap.cc:492] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:52.251796 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97eca41f7f434c02976a77643991e556" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:52.254639 11161 raft_consensus.cc:493] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:04:52.255411 11161 raft_consensus.cc:515] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:52.257334 10850 tablet_bootstrap.cc:492] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:52.258104 10850 ts_tablet_manager.cc:1403] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.185s	user 0.015s	sys 0.022s
I20260709 13:04:52.261222 11161 leader_election.cc:290] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:52.262534 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:52.263465 11074 leader_election.cc:290] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:52.265893 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97eca41f7f434c02976a77643991e556" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:52.266589 11163 consensus_queue.cc:1048] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:04:52.267724 10544 leader_election.cc:343] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:52.279211 10544 leader_election.cc:343] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.279709 10544 leader_election.cc:304] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:52.268067 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:52.281416 11074 raft_consensus.cc:2749] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:52.268337 11065 consensus_queue.cc:237] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.251816 10846 tablet_bootstrap.cc:654] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.287282 10766 leader_election.cc:304] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
W20260709 13:04:52.288545 10619 consensus_peers.cc:597] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Couldn't send request to peer 5a12f1c20c674e7f8242df41a9acf5dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:52.289312 11161 raft_consensus.cc:2749] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:52.352779 10850 raft_consensus.cc:359] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.353987 10850 raft_consensus.cc:385] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.354334 10850 raft_consensus.cc:740] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.394940 10850 consensus_queue.cc:260] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.402864 10847 tablet_bootstrap.cc:654] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.404425 10850 ts_tablet_manager.cc:1434] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.124s	user 0.010s	sys 0.000s
I20260709 13:04:52.429234 10850 tablet_bootstrap.cc:492] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:52.417927 11161 raft_consensus.cc:493] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.434955 10848 tablet_bootstrap.cc:492] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:52.435628 10848 ts_tablet_manager.cc:1403] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.224s	user 0.052s	sys 0.054s
I20260709 13:04:52.437431 11163 raft_consensus.cc:493] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.438130 11163 raft_consensus.cc:515] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.438409 10848 raft_consensus.cc:359] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.439332 10848 raft_consensus.cc:385] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.439663 10848 raft_consensus.cc:740] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.440603 10848 consensus_queue.cc:260] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.442206 11161 raft_consensus.cc:515] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.445112 11161 leader_election.cc:290] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:52.455080 10799 raft_consensus.cc:3060] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.457712 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4d2b26b86c4f22976db2fc47a3ec1a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:52.461359 10848 ts_tablet_manager.cc:1434] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:04:52.450502 11152 raft_consensus.cc:493] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.468232 10846 tablet_bootstrap.cc:492] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:52.468878 10846 ts_tablet_manager.cc:1403] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.311s	user 0.014s	sys 0.015s
I20260709 13:04:52.462407 10474 catalog_manager.cc:5697] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.471038 10848 tablet_bootstrap.cc:492] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
W20260709 13:04:52.482398 10693 consensus_peers.cc:597] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:52.467823 11152 raft_consensus.cc:515] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:52.487614 10848 tablet_bootstrap.cc:654] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.488415 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3d1fe775e74c4684020e6c5732ce13" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:52.490145 11152 leader_election.cc:290] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:52.495469 11163 leader_election.cc:290] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:52.498036 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:52.501264 10619 leader_election.cc:343] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.503131 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3d1fe775e74c4684020e6c5732ce13" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:52.505405 10692 leader_election.cc:343] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:52.506565 10693 leader_election.cc:343] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.507431 10693 leader_election.cc:304] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:52.506343 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:52.509668 10727 raft_consensus.cc:2468] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:52.511552 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4d2b26b86c4f22976db2fc47a3ec1a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:52.513782 11103 consensus_queue.cc:1048] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:04:52.510928 10846 raft_consensus.cc:359] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:52.514816 10846 raft_consensus.cc:385] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.515177 10846 raft_consensus.cc:740] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.515810 10616 leader_election.cc:304] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:52.515965 10846 consensus_queue.cc:260] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:52.516708 11152 raft_consensus.cc:2749] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:52.520117 10850 tablet_bootstrap.cc:654] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.520745 11163 raft_consensus.cc:2804] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.521334 11163 raft_consensus.cc:493] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.521839 11163 raft_consensus.cc:3060] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.523535 10846 ts_tablet_manager.cc:1434] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.001s
I20260709 13:04:52.524837 10846 tablet_bootstrap.cc:492] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
W20260709 13:04:52.528649 11191 log.cc:927] Time spent T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:04:52.528898 10848 tablet_bootstrap.cc:492] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:52.530138 10848 ts_tablet_manager.cc:1403] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.059s	user 0.020s	sys 0.003s
I20260709 13:04:52.535667 10848 raft_consensus.cc:359] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.536422 10848 raft_consensus.cc:385] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.536736 10848 raft_consensus.cc:740] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.537019 10766 leader_election.cc:304] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:52.537463 10848 consensus_queue.cc:260] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.544238 10874 raft_consensus.cc:493] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.544840 10874 raft_consensus.cc:515] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.547356 10874 leader_election.cc:290] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:52.548586 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:52.550904 10848 ts_tablet_manager.cc:1434] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.004s
W20260709 13:04:52.551893 10619 leader_election.cc:343] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.552176 10848 tablet_bootstrap.cc:492] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:52.554773 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:52.555584 10726 raft_consensus.cc:2468] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:52.558274 10846 tablet_bootstrap.cc:654] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.600522 10850 tablet_bootstrap.cc:492] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:52.601390 10850 ts_tablet_manager.cc:1403] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.182s	user 0.022s	sys 0.003s
I20260709 13:04:52.608870 10967 raft_consensus.cc:493] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.609844 10967 raft_consensus.cc:515] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:52.599614 11058 raft_consensus.cc:493] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.613874 11058 raft_consensus.cc:515] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.614830 10967 leader_election.cc:290] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:52.618198 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:52.618947 10726 raft_consensus.cc:2468] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:52.620044 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:52.622527 10850 raft_consensus.cc:359] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.623310 10850 raft_consensus.cc:385] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.623607 10850 raft_consensus.cc:740] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.624464 10850 consensus_queue.cc:260] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.616233 11058 leader_election.cc:290] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:52.615994 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6f6d7fae4f453994e7b12e063b6b49" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:52.627898 10574 raft_consensus.cc:2468] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:52.630421 10692 leader_election.cc:304] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:52.631685 11152 raft_consensus.cc:2804] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.632097 11152 raft_consensus.cc:493] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.632449 11152 raft_consensus.cc:3060] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.634948 11065 raft_consensus.cc:2749] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:52.638370 11156 raft_consensus.cc:493] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.639333 10764 leader_election.cc:304] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:52.639155 11156 raft_consensus.cc:515] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:52.643821 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69cb859af9704c2596574e50f238ce52" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:52.645759 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69cb859af9704c2596574e50f238ce52" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:52.651145 10544 leader_election.cc:343] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:52.651964 10541 leader_election.cc:343] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.652418 10541 leader_election.cc:304] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:52.652709 11000 raft_consensus.cc:2804] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.653146 11000 raft_consensus.cc:493] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.653514 11000 raft_consensus.cc:3060] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.653703 11155 raft_consensus.cc:2749] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:52.650602 11156 leader_election.cc:290] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:52.662479 10850 ts_tablet_manager.cc:1434] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 13:04:52.664357 10850 tablet_bootstrap.cc:492] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:52.674993 10848 tablet_bootstrap.cc:654] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.689080 10847 tablet_bootstrap.cc:492] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:52.689889 10847 ts_tablet_manager.cc:1403] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.449s	user 0.018s	sys 0.011s
I20260709 13:04:52.698578 10846 tablet_bootstrap.cc:492] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:52.699482 10846 ts_tablet_manager.cc:1403] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.175s	user 0.013s	sys 0.009s
I20260709 13:04:52.689363 11000 raft_consensus.cc:515] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.656978 11163 raft_consensus.cc:515] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:52.679162 10616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 106364us
I20260709 13:04:52.705224 11000 leader_election.cc:290] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:52.707130 11163 leader_election.cc:290] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:52.734965 10651 raft_consensus.cc:2393] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 1: Already voted for candidate 73db933c4a6848d1b2aeb56813f7bb2c in this term.
I20260709 13:04:52.735553 10846 raft_consensus.cc:359] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:52.739447 10846 raft_consensus.cc:385] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.739775 10846 raft_consensus.cc:740] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.743043 10847 raft_consensus.cc:359] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.745100 10847 raft_consensus.cc:385] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.745427 10847 raft_consensus.cc:740] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.745909 10846 consensus_queue.cc:260] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:52.746205 10847 consensus_queue.cc:260] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.744083 10649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:52.756937 10846 ts_tablet_manager.cc:1434] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.003s
I20260709 13:04:52.757725 10847 ts_tablet_manager.cc:1434] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.001s
I20260709 13:04:52.732587 10616 leader_election.cc:304] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:52.762537 11201 raft_consensus.cc:2804] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.762974 11201 raft_consensus.cc:493] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.763317 11201 raft_consensus.cc:3060] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.766921 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6f6d7fae4f453994e7b12e063b6b49" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:52.783179 10846 tablet_bootstrap.cc:492] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:52.783403 10850 tablet_bootstrap.cc:654] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.785414 10848 tablet_bootstrap.cc:492] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:52.785899 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:52.786309 10848 ts_tablet_manager.cc:1403] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.234s	user 0.011s	sys 0.016s
I20260709 13:04:52.786648 10726 raft_consensus.cc:3060] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:52.790257 10693 leader_election.cc:343] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.791926 10848 raft_consensus.cc:359] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.793708 10848 raft_consensus.cc:385] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.794094 10848 raft_consensus.cc:740] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.799202 10847 tablet_bootstrap.cc:492] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:52.801294 10848 consensus_queue.cc:260] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.806387 10848 ts_tablet_manager.cc:1434] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 13:04:52.810494 10848 tablet_bootstrap.cc:492] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:52.795111 11161 raft_consensus.cc:493] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.817459 10846 tablet_bootstrap.cc:654] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:52.819411 10544 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 71353.5us
I20260709 13:04:52.813841 11161 raft_consensus.cc:515] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.824839 11204 raft_consensus.cc:493] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.821395 11152 raft_consensus.cc:515] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:52.830433 11106 consensus_queue.cc:1048] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:52.833407 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:52.852768 10799 raft_consensus.cc:2393] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 5a12f1c20c674e7f8242df41a9acf5dc in this term.
I20260709 13:04:52.847362 11000 consensus_queue.cc:1048] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:52.853951 11152 leader_election.cc:290] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:52.855450 11161 leader_election.cc:290] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:52.855584 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6f6d7fae4f453994e7b12e063b6b49" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
W20260709 13:04:52.855815 10767 consensus_peers.cc:597] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:52.851315 11204 raft_consensus.cc:515] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:04:52.857372 10693 leader_election.cc:343] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:52.860219 10649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:52.860898 11204 leader_election.cc:290] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:52.861181 10649 raft_consensus.cc:2468] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:52.862794 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6f6d7fae4f453994e7b12e063b6b49" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:52.863181 10767 leader_election.cc:304] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:04:52.863497 10576 raft_consensus.cc:3060] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.864284 11065 raft_consensus.cc:2804] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:52.864677 11065 raft_consensus.cc:493] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.865003 11065 raft_consensus.cc:3060] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.874507 10848 tablet_bootstrap.cc:654] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.885272 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:52.886322 10800 raft_consensus.cc:3060] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:52.889991 10726 raft_consensus.cc:2468] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:52.892632 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:52.894385 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:52.896425 10724 raft_consensus.cc:2393] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 5a12f1c20c674e7f8242df41a9acf5dc in this term.
I20260709 13:04:52.878558 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:52.894445 10573 raft_consensus.cc:2468] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:52.914075 10850 tablet_bootstrap.cc:492] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:52.914680 10850 ts_tablet_manager.cc:1403] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.251s	user 0.016s	sys 0.009s
I20260709 13:04:52.915099 10847 tablet_bootstrap.cc:654] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:52.922502 10850 raft_consensus.cc:359] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.923405 10850 raft_consensus.cc:385] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:52.923698 10850 raft_consensus.cc:740] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:52.924360 10850 consensus_queue.cc:260] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.927942 10850 ts_tablet_manager.cc:1434] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:04:52.934134 10764 leader_election.cc:304] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:52.936388 11161 raft_consensus.cc:2804] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:52.936949 11161 raft_consensus.cc:697] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:52.944597 10846 tablet_bootstrap.cc:492] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:52.937855 11161 consensus_queue.cc:237] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.952322 10616 leader_election.cc:304] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:52.953516 11106 raft_consensus.cc:2749] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:04:52.954066 10616 consensus_peers.cc:597] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Couldn't send request to peer 7ecd836bb7a144dc9aa07a046239d6ce. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:52.934953 11201 raft_consensus.cc:515] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:52.963913 10846 ts_tablet_manager.cc:1403] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.183s	user 0.017s	sys 0.012s
I20260709 13:04:52.982278 10850 tablet_bootstrap.cc:492] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:52.984212 10848 tablet_bootstrap.cc:492] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:52.985000 10848 ts_tablet_manager.cc:1403] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.175s	user 0.011s	sys 0.028s
I20260709 13:04:52.990899 11065 raft_consensus.cc:515] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:52.992421 10801 raft_consensus.cc:2393] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 5a12f1c20c674e7f8242df41a9acf5dc in this term.
I20260709 13:04:52.993414 11065 leader_election.cc:290] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:52.995637 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:52.996414 10652 raft_consensus.cc:3060] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.019295 10571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:53.020277 10571 raft_consensus.cc:3060] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.007915 10619 leader_election.cc:304] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:53.023774 10847 tablet_bootstrap.cc:492] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:53.024382 10847 ts_tablet_manager.cc:1403] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.226s	user 0.010s	sys 0.013s
I20260709 13:04:53.024478 10576 raft_consensus.cc:2468] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:53.013293 11156 raft_consensus.cc:493] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.026422 10692 leader_election.cc:304] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:53.026043 11156 raft_consensus.cc:515] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.029148 10967 raft_consensus.cc:2804] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:53.030629 10846 raft_consensus.cc:359] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.031435 10967 raft_consensus.cc:697] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:53.024410 10848 raft_consensus.cc:359] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.033193 10848 raft_consensus.cc:385] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.033528 10848 raft_consensus.cc:740] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.034153 10848 consensus_queue.cc:260] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.031401 10846 raft_consensus.cc:385] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.034801 10846 raft_consensus.cc:740] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.032423 10967 consensus_queue.cc:237] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.037539 11201 leader_election.cc:290] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:53.035559 10846 consensus_queue.cc:260] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.039700 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:53.041532 10723 raft_consensus.cc:3060] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.032671 10649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3d1fe775e74c4684020e6c5732ce13" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:53.043838 10544 leader_election.cc:343] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.045862 10846 ts_tablet_manager.cc:1434] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.001s
I20260709 13:04:53.047801 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:53.048983 11156 leader_election.cc:290] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:53.050509 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3d1fe775e74c4684020e6c5732ce13" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:53.051399 10727 raft_consensus.cc:2468] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:53.053306 10541 leader_election.cc:304] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
W20260709 13:04:53.054941 10619 leader_election.cc:343] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.055406 10571 raft_consensus.cc:2468] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:53.058354 11061 raft_consensus.cc:2804] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:53.058904 11061 raft_consensus.cc:493] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.059249 11061 raft_consensus.cc:3060] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.063779 10766 leader_election.cc:304] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:53.065544 11065 raft_consensus.cc:2804] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:53.066149 11065 raft_consensus.cc:697] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:53.067291 11065 consensus_queue.cc:237] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.050915 10847 raft_consensus.cc:359] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:53.069878 10847 raft_consensus.cc:385] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.070226 10847 raft_consensus.cc:740] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.087370 10847 consensus_queue.cc:260] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:53.089267 10848 ts_tablet_manager.cc:1434] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.104s	user 0.009s	sys 0.000s
I20260709 13:04:53.093102 10846 tablet_bootstrap.cc:492] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:53.107656 10848 tablet_bootstrap.cc:492] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:53.113953 10847 ts_tablet_manager.cc:1434] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.089s	user 0.010s	sys 0.000s
I20260709 13:04:53.115798 11203 raft_consensus.cc:493] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.116580 11203 raft_consensus.cc:515] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:53.137814 10850 tablet_bootstrap.cc:654] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.151247 10476 catalog_manager.cc:5697] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:53.157980 11216 log.cc:927] Time spent T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:04:53.158805 11214 log.cc:927] Time spent T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:04:53.162206 11212 log.cc:927] Time spent T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:04:53.173035 11203 leader_election.cc:290] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:53.201406 10848 tablet_bootstrap.cc:654] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.205752 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d33bf08ed78416ea29aaf59733e35a2" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:53.208691 10847 tablet_bootstrap.cc:492] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
W20260709 13:04:53.210073 10693 leader_election.cc:343] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.214951 10649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d33bf08ed78416ea29aaf59733e35a2" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:53.216986 11061 raft_consensus.cc:515] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.218142 11074 raft_consensus.cc:493] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.218684 11074 raft_consensus.cc:515] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:53.220971 11074 leader_election.cc:290] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:53.211752 10652 raft_consensus.cc:2468] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:53.222126 11106 raft_consensus.cc:2749] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:53.224433 10693 leader_election.cc:343] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.224970 10693 leader_election.cc:304] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:53.227145 10967 raft_consensus.cc:2749] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:53.230751 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:53.233914 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:04:53.235116 10544 leader_election.cc:343] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.239424 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:53.240309 10725 raft_consensus.cc:2468] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:53.243355 10541 leader_election.cc:304] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:53.248639 11061 leader_election.cc:290] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:53.238412 10723 raft_consensus.cc:2468] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:53.254129 11074 raft_consensus.cc:2804] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:53.256045 10847 tablet_bootstrap.cc:654] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.266654 10846 tablet_bootstrap.cc:654] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.267832 10616 leader_election.cc:304] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:53.276674 11201 raft_consensus.cc:2804] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:53.269099 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3d1fe775e74c4684020e6c5732ce13" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:53.277905 10723 raft_consensus.cc:3060] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.277249 11201 raft_consensus.cc:697] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:53.259656 11074 raft_consensus.cc:493] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.298363 11074 raft_consensus.cc:3060] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.307412 11201 consensus_queue.cc:237] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:53.313933 10723 raft_consensus.cc:2468] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
I20260709 13:04:53.290184 10649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3d1fe775e74c4684020e6c5732ce13" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:53.319309 10541 leader_election.cc:304] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:53.320276 11155 raft_consensus.cc:2804] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:53.320835 11155 raft_consensus.cc:697] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:04:53.321772 11155 consensus_queue.cc:237] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.350260 10848 tablet_bootstrap.cc:492] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:53.352337 10848 ts_tablet_manager.cc:1403] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.245s	user 0.014s	sys 0.018s
W20260709 13:04:53.353102 10544 leader_election.cc:343] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.353642 10474 catalog_manager.cc:5697] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:53.351774 11058 raft_consensus.cc:493] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.372933 11058 raft_consensus.cc:515] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.374922 11065 consensus_queue.cc:1048] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:53.379558 11200 raft_consensus.cc:493] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.380277 11200 raft_consensus.cc:515] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.382997 11200 leader_election.cc:290] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:53.385241 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:53.386152 10578 raft_consensus.cc:2468] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:53.387611 10618 leader_election.cc:304] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:53.390525 11207 raft_consensus.cc:2804] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:53.390962 11207 raft_consensus.cc:493] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.391319 11207 raft_consensus.cc:3060] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.396073 10848 raft_consensus.cc:359] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:53.396940 10848 raft_consensus.cc:385] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.397217 10848 raft_consensus.cc:740] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.401858 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:53.402725 11074 raft_consensus.cc:515] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:53.375552 11058 leader_election.cc:290] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:53.406296 10850 tablet_bootstrap.cc:492] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:53.406973 10850 ts_tablet_manager.cc:1403] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.436s	user 0.035s	sys 0.045s
I20260709 13:04:53.414471 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:53.415380 10723 raft_consensus.cc:3060] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.410729 10848 consensus_queue.cc:260] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:53.417649 10475 catalog_manager.cc:5697] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:53.419556 10850 raft_consensus.cc:359] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.420475 10850 raft_consensus.cc:385] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.420857 10850 raft_consensus.cc:740] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.422971 10850 consensus_queue.cc:260] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.426358 11074 leader_election.cc:290] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:53.414460 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:53.435813 10850 ts_tablet_manager.cc:1434] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.001s
I20260709 13:04:53.449895 10578 raft_consensus.cc:2468] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:53.450953 10850 tablet_bootstrap.cc:492] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:53.451267 10649 raft_consensus.cc:3060] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.436821 11207 raft_consensus.cc:515] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.455289 10653 raft_consensus.cc:2393] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 1: Already voted for candidate 73db933c4a6848d1b2aeb56813f7bb2c in this term.
I20260709 13:04:53.448343 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:53.469384 10847 tablet_bootstrap.cc:492] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:53.470114 10847 ts_tablet_manager.cc:1403] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.269s	user 0.020s	sys 0.011s
I20260709 13:04:53.479678 10848 ts_tablet_manager.cc:1434] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.126s	user 0.008s	sys 0.000s
I20260709 13:04:53.477276 10723 raft_consensus.cc:2468] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
I20260709 13:04:53.483640 10848 tablet_bootstrap.cc:492] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:53.483645 10541 leader_election.cc:304] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:53.486102 10766 leader_election.cc:304] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:53.488219 11207 leader_election.cc:290] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
W20260709 13:04:53.489526 10544 leader_election.cc:343] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.496569 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:53.496831 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:53.501447 11203 raft_consensus.cc:493] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.502398 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:53.503122 10573 raft_consensus.cc:3060] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.503636 11061 raft_consensus.cc:493] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.504204 11061 raft_consensus.cc:515] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.492647 11000 raft_consensus.cc:2804] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:53.506080 11000 raft_consensus.cc:493] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.506476 11000 raft_consensus.cc:3060] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.506835 11061 leader_election.cc:290] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:53.516687 10847 raft_consensus.cc:359] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:53.519037 10847 raft_consensus.cc:385] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.519343 10847 raft_consensus.cc:740] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.492961 11155 raft_consensus.cc:2804] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:53.520154 10846 tablet_bootstrap.cc:492] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:53.520838 10846 ts_tablet_manager.cc:1403] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.434s	user 0.010s	sys 0.012s
I20260709 13:04:53.520661 10847 consensus_queue.cc:260] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:53.523088 10850 tablet_bootstrap.cc:654] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.525291 10847 ts_tablet_manager.cc:1434] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.004s
I20260709 13:04:53.525422 11058 consensus_queue.cc:1048] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:53.526433 10847 tablet_bootstrap.cc:492] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:53.528308 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c617206d4bb4c0d84e4f8f6cccdbf30" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:53.526511 10846 raft_consensus.cc:359] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.529080 10846 raft_consensus.cc:385] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:53.530018 10544 leader_election.cc:343] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.529356 10846 raft_consensus.cc:740] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.520191 11155 raft_consensus.cc:697] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:04:53.502422 11203 raft_consensus.cc:515] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:53.533593 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c617206d4bb4c0d84e4f8f6cccdbf30" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:53.534109 10847 tablet_bootstrap.cc:654] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.523231 10848 tablet_bootstrap.cc:654] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.534349 10846 consensus_queue.cc:260] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.532982 11155 consensus_queue.cc:237] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:53.548897 10846 ts_tablet_manager.cc:1434] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:04:53.549598 10928 consensus_queue.cc:1048] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:04:53.550949 10541 leader_election.cc:343] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.551592 10541 leader_election.cc:304] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:53.563428 10476 catalog_manager.cc:5697] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:53.554438 11061 raft_consensus.cc:2749] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:53.577507 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d1771d12424af18f74abf7650fab3a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:53.579499 10693 consensus_peers.cc:597] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.580170 10693 leader_election.cc:343] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.586467 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d1771d12424af18f74abf7650fab3a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:53.586285 11203 leader_election.cc:290] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
W20260709 13:04:53.588582 10692 leader_election.cc:343] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.589087 10692 leader_election.cc:304] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:53.588814 11000 raft_consensus.cc:515] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.591060 10797 raft_consensus.cc:2393] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 5a12f1c20c674e7f8242df41a9acf5dc in this term.
I20260709 13:04:53.591087 11133 raft_consensus.cc:2749] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:53.595039 10846 tablet_bootstrap.cc:492] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:53.597179 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:53.598232 10651 raft_consensus.cc:2393] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 1: Already voted for candidate 73db933c4a6848d1b2aeb56813f7bb2c in this term.
I20260709 13:04:53.599843 11000 leader_election.cc:290] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:53.590898 10573 raft_consensus.cc:2468] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:53.601423 10618 leader_election.cc:304] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:53.602324 11207 raft_consensus.cc:2804] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:53.603812 11207 raft_consensus.cc:697] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:53.610918 11207 consensus_queue.cc:237] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.615566 11161 consensus_queue.cc:1048] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:53.616392 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:53.617256 10573 raft_consensus.cc:2393] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 1: Already voted for candidate 73db933c4a6848d1b2aeb56813f7bb2c in this term.
I20260709 13:04:53.659119 10848 tablet_bootstrap.cc:492] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:53.659651 10848 ts_tablet_manager.cc:1403] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.176s	user 0.011s	sys 0.013s
I20260709 13:04:53.666332 10850 tablet_bootstrap.cc:492] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:53.666961 10850 ts_tablet_manager.cc:1403] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.216s	user 0.018s	sys 0.007s
I20260709 13:04:53.665318 10848 raft_consensus.cc:359] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:53.667943 10848 raft_consensus.cc:385] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.668177 10848 raft_consensus.cc:740] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.673768 10766 leader_election.cc:304] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:53.676157 11161 raft_consensus.cc:2749] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:53.676887 11000 consensus_queue.cc:1048] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:53.668658 10848 consensus_queue.cc:260] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:53.681986 10847 tablet_bootstrap.cc:492] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:53.682628 10847 ts_tablet_manager.cc:1403] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.156s	user 0.014s	sys 0.003s
I20260709 13:04:53.686062 10847 raft_consensus.cc:359] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.687474 10847 raft_consensus.cc:385] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.687814 10847 raft_consensus.cc:740] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.689708 10847 consensus_queue.cc:260] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.694263 10847 ts_tablet_manager.cc:1434] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:04:53.699380 10848 ts_tablet_manager.cc:1434] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:04:53.700057 10846 tablet_bootstrap.cc:654] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.699043 10474 catalog_manager.cc:5697] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:53.714846 11061 raft_consensus.cc:493] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.718221 11061 raft_consensus.cc:515] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.723333 10848 tablet_bootstrap.cc:492] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:53.730136 10850 raft_consensus.cc:359] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.734773 11156 consensus_queue.cc:1048] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:53.731223 10850 raft_consensus.cc:385] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.705142 10476 catalog_manager.cc:5697] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:53.741973 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a27c6651a83844a69107fdd6f819ebde" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:53.737978 10850 raft_consensus.cc:740] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
W20260709 13:04:53.746769 10544 leader_election.cc:343] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.747668 10850 consensus_queue.cc:260] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.745106 11061 leader_election.cc:290] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:53.751854 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a27c6651a83844a69107fdd6f819ebde" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:53.756628 10544 leader_election.cc:343] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.757210 10544 leader_election.cc:304] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:53.761925 10847 tablet_bootstrap.cc:492] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:53.772656 11156 raft_consensus.cc:2749] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:53.773216 10847 tablet_bootstrap.cc:654] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.773579 10848 tablet_bootstrap.cc:654] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:53.788909 10850 ts_tablet_manager.cc:1434] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.122s	user 0.009s	sys 0.002s
W20260709 13:04:53.807768 11235 log.cc:927] Time spent T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:04:53.827548 11074 raft_consensus.cc:493] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.828163 11074 raft_consensus.cc:515] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.830703 11074 leader_election.cc:290] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:53.835254 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:53.836918 10544 leader_election.cc:343] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.839847 11103 raft_consensus.cc:493] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.840442 11103 raft_consensus.cc:515] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:53.841113 10928 raft_consensus.cc:493] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.841766 10928 raft_consensus.cc:515] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:53.848511 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:53.856907 10928 leader_election.cc:290] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:53.858179 10850 tablet_bootstrap.cc:492] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:53.863605 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "656245aa555f45a8a6ef044f0da06e46" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:53.866333 11103 leader_election.cc:290] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
W20260709 13:04:53.868330 10544 consensus_peers.cc:597] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:53.871970 10874 raft_consensus.cc:493] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.872570 10874 raft_consensus.cc:515] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:53.875386 10874 leader_election.cc:290] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:53.883317 10848 tablet_bootstrap.cc:492] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:53.883968 10848 ts_tablet_manager.cc:1403] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.161s	user 0.017s	sys 0.018s
I20260709 13:04:53.876811 10649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "656245aa555f45a8a6ef044f0da06e46" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:53.889730 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be849d4cbf0b4f7a99f9935a55611658" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:53.870302 10723 raft_consensus.cc:3060] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.893347 10650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:53.883080 11227 raft_consensus.cc:493] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.896791 11227 raft_consensus.cc:515] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:53.902842 10846 tablet_bootstrap.cc:492] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:53.903836 10846 ts_tablet_manager.cc:1403] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.335s	user 0.025s	sys 0.003s
W20260709 13:04:53.917505 10693 leader_election.cc:343] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.918793 11207 consensus_queue.cc:1048] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:53.918885 10693 leader_election.cc:304] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:53.925938 10848 raft_consensus.cc:359] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:53.926822 10848 raft_consensus.cc:385] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.927090 10848 raft_consensus.cc:740] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.928345 10848 consensus_queue.cc:260] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:53.930517 10848 ts_tablet_manager.cc:1434] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.001s
I20260709 13:04:53.931836 10848 tablet_bootstrap.cc:492] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:53.930218 10846 raft_consensus.cc:359] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:53.941745 10846 raft_consensus.cc:385] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.942441 10846 raft_consensus.cc:740] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.943822 10846 consensus_queue.cc:260] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:53.901290 11227 leader_election.cc:290] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:53.954977 10847 tablet_bootstrap.cc:492] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:53.955827 10847 ts_tablet_manager.cc:1403] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.210s	user 0.025s	sys 0.008s
W20260709 13:04:53.957970 10764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 95822us
W20260709 13:04:53.959012 10693 leader_election.cc:343] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:53.959498 10693 leader_election.cc:304] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:53.962925 10846 ts_tablet_manager.cc:1434] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.059s	user 0.010s	sys 0.000s
I20260709 13:04:53.970238 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be849d4cbf0b4f7a99f9935a55611658" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:53.976850 10846 tablet_bootstrap.cc:492] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:53.963807 10967 raft_consensus.cc:2749] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:53.965649 11074 raft_consensus.cc:493] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.983882 10799 raft_consensus.cc:3060] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:53.993976 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:53.995937 10619 leader_election.cc:304] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:53.997006 10847 raft_consensus.cc:359] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:53.997829 10847 raft_consensus.cc:385] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:53.998116 10847 raft_consensus.cc:740] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:53.981287 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:54.003659 10544 leader_election.cc:343] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.004190 10850 tablet_bootstrap.cc:654] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.009817 10848 tablet_bootstrap.cc:654] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.011935 11074 raft_consensus.cc:515] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:54.004191 10544 leader_election.cc:304] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:54.014462 11074 leader_election.cc:290] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:54.016947 11061 raft_consensus.cc:2749] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:54.019866 10650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97eca41f7f434c02976a77643991e556" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:54.048753 10544 leader_election.cc:343] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.055807 10847 consensus_queue.cc:260] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.080896 10847 ts_tablet_manager.cc:1434] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.125s	user 0.012s	sys 0.002s
W20260709 13:04:54.049245 10619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 63033.1us
I20260709 13:04:54.052925 11252 raft_consensus.cc:2749] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:54.102782 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:54.103453 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97eca41f7f434c02976a77643991e556" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:54.106127 10652 raft_consensus.cc:3060] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.110689 10874 consensus_queue.cc:1048] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:54.118094 10850 tablet_bootstrap.cc:492] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:54.119282 10850 ts_tablet_manager.cc:1403] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.274s	user 0.014s	sys 0.029s
I20260709 13:04:54.121223 10847 tablet_bootstrap.cc:492] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:54.106287 11000 raft_consensus.cc:493] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.130509 11000 raft_consensus.cc:515] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:54.140604 10544 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 53511.4us
I20260709 13:04:54.145300 10846 tablet_bootstrap.cc:654] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:54.150658 10544 leader_election.cc:343] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.153208 11254 raft_consensus.cc:493] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.155419 11254 raft_consensus.cc:515] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.156173 11000 leader_election.cc:290] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:54.157783 11200 consensus_queue.cc:1048] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:54.163250 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d33bf08ed78416ea29aaf59733e35a2" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:54.164048 10727 raft_consensus.cc:2468] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:54.168447 10764 leader_election.cc:304] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:54.153229 10544 leader_election.cc:304] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:54.173544 11156 raft_consensus.cc:2749] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:54.174376 11000 raft_consensus.cc:2804] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:54.174890 11000 raft_consensus.cc:493] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.175447 11000 raft_consensus.cc:3060] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.177309 11254 leader_election.cc:290] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:54.180498 10848 tablet_bootstrap.cc:492] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:54.181259 10848 ts_tablet_manager.cc:1403] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.250s	user 0.015s	sys 0.002s
I20260709 13:04:54.183194 11133 raft_consensus.cc:2749] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:54.199394 10847 tablet_bootstrap.cc:654] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.211673 10799 raft_consensus.cc:3060] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:54.194793 10619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 85608.5us
I20260709 13:04:54.227550 10850 raft_consensus.cc:359] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.228420 10850 raft_consensus.cc:385] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.228735 10850 raft_consensus.cc:740] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.239477 11203 raft_consensus.cc:493] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.240165 11203 raft_consensus.cc:515] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.233165 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:54.243527 11203 leader_election.cc:290] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:54.248293 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d33bf08ed78416ea29aaf59733e35a2" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:54.252038 10767 leader_election.cc:304] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:54.253697 11065 raft_consensus.cc:2749] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:54.259107 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:54.262756 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69cb859af9704c2596574e50f238ce52" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:54.273092 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69cb859af9704c2596574e50f238ce52" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:54.274025 10573 raft_consensus.cc:2468] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
W20260709 13:04:54.268635 10693 leader_election.cc:343] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.265857 10850 consensus_queue.cc:260] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
W20260709 13:04:54.277971 10541 leader_election.cc:343] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:54.288825 10544 leader_election.cc:343] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.289390 10544 leader_election.cc:304] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:54.290194 11074 raft_consensus.cc:2749] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:54.291267 10850 ts_tablet_manager.cc:1434] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.172s	user 0.007s	sys 0.004s
I20260709 13:04:54.284181 11255 consensus_queue.cc:1048] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:04:54.296274 10850 tablet_bootstrap.cc:492] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:54.307878 10692 leader_election.cc:304] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:54.309947 10848 raft_consensus.cc:359] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.311079 10848 raft_consensus.cc:385] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.311429 10848 raft_consensus.cc:740] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.320385 10848 consensus_queue.cc:260] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
W20260709 13:04:54.314487 10767 leader_election.cc:343] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.318231 11203 raft_consensus.cc:2804] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:54.339335 11203 raft_consensus.cc:493] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.341162 11203 raft_consensus.cc:3060] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.318048 11254 consensus_queue.cc:1048] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:04:54.344090 11200 consensus_queue.cc:1048] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:04:54.345921 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 56491.6us
I20260709 13:04:54.349166 10848 ts_tablet_manager.cc:1434] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.163s	user 0.010s	sys 0.000s
I20260709 13:04:54.352479 10846 tablet_bootstrap.cc:492] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:54.353085 10846 ts_tablet_manager.cc:1403] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.377s	user 0.025s	sys 0.012s
I20260709 13:04:54.360517 10848 tablet_bootstrap.cc:492] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:54.364645 11227 raft_consensus.cc:493] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.374940 11227 raft_consensus.cc:515] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.377573 11227 leader_election.cc:290] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:54.374647 10846 raft_consensus.cc:359] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.379590 10846 raft_consensus.cc:385] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.380184 10846 raft_consensus.cc:740] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.364185 10473 catalog_manager.cc:5697] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } }
I20260709 13:04:54.381431 10847 tablet_bootstrap.cc:492] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:54.382457 10847 ts_tablet_manager.cc:1403] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.269s	user 0.041s	sys 0.026s
W20260709 13:04:54.395478 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:54.402434 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:54.387894 11252 raft_consensus.cc:493] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.416244 11252 raft_consensus.cc:515] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.413367 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:54.430177 10846 consensus_queue.cc:260] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.438815 10764 leader_election.cc:304] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:54.456118 10850 tablet_bootstrap.cc:654] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.457526 10846 ts_tablet_manager.cc:1434] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.104s	user 0.013s	sys 0.001s
I20260709 13:04:54.463943 11252 leader_election.cc:290] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:54.466908 10847 raft_consensus.cc:359] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.467841 10847 raft_consensus.cc:385] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.468025 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6f6d7fae4f453994e7b12e063b6b49" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:54.470069 11058 raft_consensus.cc:2749] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.470247 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6f6d7fae4f453994e7b12e063b6b49" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:54.474889 10846 tablet_bootstrap.cc:492] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:54.468294 10847 raft_consensus.cc:740] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.481954 10847 consensus_queue.cc:260] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.491144 10847 ts_tablet_manager.cc:1434] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.108s	user 0.008s	sys 0.003s
I20260709 13:04:54.502775 11203 raft_consensus.cc:515] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.461746 11000 raft_consensus.cc:515] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:54.505568 10848 tablet_bootstrap.cc:654] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.510830 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69cb859af9704c2596574e50f238ce52" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:54.511716 10797 raft_consensus.cc:3060] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.518647 11000 leader_election.cc:290] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:54.526317 10616 leader_election.cc:304] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:54.524926 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d33bf08ed78416ea29aaf59733e35a2" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:54.528914 10723 raft_consensus.cc:3060] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.535483 11201 raft_consensus.cc:2749] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:54.537552 10723 raft_consensus.cc:2468] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:54.516382 11203 leader_election.cc:290] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:54.542824 10846 tablet_bootstrap.cc:654] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.546303 10847 tablet_bootstrap.cc:492] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:54.547858 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d33bf08ed78416ea29aaf59733e35a2" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:54.568332 10571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69cb859af9704c2596574e50f238ce52" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:54.568473 10764 leader_election.cc:304] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:54.572376 10571 raft_consensus.cc:3060] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.573527 10850 tablet_bootstrap.cc:492] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:54.574783 11251 raft_consensus.cc:2804] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:54.578588 11251 raft_consensus.cc:697] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:54.580008 11251 consensus_queue.cc:237] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:54.553411 10797 raft_consensus.cc:2468] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:54.589354 10847 tablet_bootstrap.cc:654] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.592303 10571 raft_consensus.cc:2468] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:54.598748 10850 ts_tablet_manager.cc:1403] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.306s	user 0.026s	sys 0.005s
I20260709 13:04:54.599615 10652 raft_consensus.cc:3060] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.613828 10693 leader_election.cc:304] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:54.616576 11247 raft_consensus.cc:2804] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:54.617197 11247 raft_consensus.cc:697] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:54.618139 11247 consensus_queue.cc:237] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.635771 10850 raft_consensus.cc:359] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:54.636808 10850 raft_consensus.cc:385] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.637133 10850 raft_consensus.cc:740] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.638140 10850 consensus_queue.cc:260] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:54.636497 10847 tablet_bootstrap.cc:492] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:54.639407 10847 ts_tablet_manager.cc:1403] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.114s	user 0.013s	sys 0.001s
W20260709 13:04:54.631028 10767 leader_election.cc:343] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.648736 11274 raft_consensus.cc:493] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.653960 11274 raft_consensus.cc:515] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.656906 11274 leader_election.cc:290] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:54.662750 10850 ts_tablet_manager.cc:1434] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.002s
I20260709 13:04:54.670037 10850 tablet_bootstrap.cc:492] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:54.665866 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c617206d4bb4c0d84e4f8f6cccdbf30" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:54.672258 10573 raft_consensus.cc:2468] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:54.673959 10692 leader_election.cc:304] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:54.675948 11247 raft_consensus.cc:2804] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:54.676403 11247 raft_consensus.cc:493] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.676774 11247 raft_consensus.cc:3060] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.658380 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c617206d4bb4c0d84e4f8f6cccdbf30" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:54.684074 10850 tablet_bootstrap.cc:654] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:54.688679 10693 leader_election.cc:343] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.695941 10847 raft_consensus.cc:359] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.697244 10847 raft_consensus.cc:385] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.697860 10847 raft_consensus.cc:740] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.700270 10848 tablet_bootstrap.cc:492] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:54.700985 10848 ts_tablet_manager.cc:1403] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.341s	user 0.021s	sys 0.012s
I20260709 13:04:54.723984 11247 raft_consensus.cc:515] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.726994 10848 raft_consensus.cc:359] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:54.727900 10848 raft_consensus.cc:385] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.729128 10847 consensus_queue.cc:260] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.732158 10847 ts_tablet_manager.cc:1434] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.092s	user 0.008s	sys 0.000s
I20260709 13:04:54.733352 10846 tablet_bootstrap.cc:492] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:54.733561 10847 tablet_bootstrap.cc:492] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:54.733979 10846 ts_tablet_manager.cc:1403] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.259s	user 0.026s	sys 0.057s
I20260709 13:04:54.735109 10475 catalog_manager.cc:5697] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:54.737046 10846 raft_consensus.cc:359] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:54.728235 10848 raft_consensus.cc:740] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.741554 10846 raft_consensus.cc:385] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.742054 10846 raft_consensus.cc:740] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.742825 10846 consensus_queue.cc:260] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:54.746528 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c617206d4bb4c0d84e4f8f6cccdbf30" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:54.747572 10573 raft_consensus.cc:3060] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.753288 10846 ts_tablet_manager.cc:1434] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.004s
I20260709 13:04:54.755147 10846 tablet_bootstrap.cc:492] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:54.761392 11155 raft_consensus.cc:493] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.762583 10846 tablet_bootstrap.cc:654] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.762394 11155 raft_consensus.cc:515] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.765848 11155 leader_election.cc:290] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:54.767218 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d1771d12424af18f74abf7650fab3a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:54.767051 10848 consensus_queue.cc:260] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:54.768229 10727 raft_consensus.cc:2468] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:54.769783 10848 ts_tablet_manager.cc:1434] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.004s
I20260709 13:04:54.771456 10541 leader_election.cc:304] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:54.773332 11247 leader_election.cc:290] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:54.777877 10848 tablet_bootstrap.cc:492] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:54.777737 11156 raft_consensus.cc:2804] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:54.779685 11156 raft_consensus.cc:493] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.780169 11156 raft_consensus.cc:3060] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.784724 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c617206d4bb4c0d84e4f8f6cccdbf30" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:54.790098 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d1771d12424af18f74abf7650fab3a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:54.792302 10544 leader_election.cc:343] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:54.792464 10693 leader_election.cc:343] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.796823 10848 tablet_bootstrap.cc:654] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.808091 10848 tablet_bootstrap.cc:492] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:54.808725 10848 ts_tablet_manager.cc:1403] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.001s
I20260709 13:04:54.812150 10848 raft_consensus.cc:359] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:54.813031 10848 raft_consensus.cc:385] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.814571 11156 raft_consensus.cc:515] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.814988 11272 raft_consensus.cc:493] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.815974 10847 tablet_bootstrap.cc:654] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.815685 11272 raft_consensus.cc:515] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:54.818606 11272 leader_election.cc:290] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:54.820365 10571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:54.821148 10571 raft_consensus.cc:2468] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:54.821463 10850 tablet_bootstrap.cc:492] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:54.822031 10850 ts_tablet_manager.cc:1403] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.153s	user 0.014s	sys 0.017s
I20260709 13:04:54.823573 10766 leader_election.cc:304] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:54.824842 11272 raft_consensus.cc:2804] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:54.825803 11272 raft_consensus.cc:493] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.826238 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:54.827665 10850 raft_consensus.cc:359] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:54.828488 10850 raft_consensus.cc:385] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.828764 10850 raft_consensus.cc:740] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
W20260709 13:04:54.829795 10767 leader_election.cc:343] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.826239 11272 raft_consensus.cc:3060] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.829746 10850 consensus_queue.cc:260] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:54.832113 10573 raft_consensus.cc:2468] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:54.813342 10848 raft_consensus.cc:740] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.836802 10692 leader_election.cc:304] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:54.836633 10848 consensus_queue.cc:260] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:54.837986 10850 ts_tablet_manager.cc:1434] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:04:54.839696 10850 tablet_bootstrap.cc:492] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:54.851485 11156 leader_election.cc:290] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:54.841462 11247 raft_consensus.cc:2804] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:54.863729 11247 raft_consensus.cc:697] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:54.839844 10848 ts_tablet_manager.cc:1434] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.003s
I20260709 13:04:54.864761 11247 consensus_queue.cc:237] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.866370 10848 tablet_bootstrap.cc:492] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:54.866750 10846 tablet_bootstrap.cc:492] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:54.867692 10846 ts_tablet_manager.cc:1403] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.113s	user 0.041s	sys 0.021s
I20260709 13:04:54.856812 10850 tablet_bootstrap.cc:654] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.857131 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d1771d12424af18f74abf7650fab3a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:54.869187 10727 raft_consensus.cc:3060] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.875741 10846 raft_consensus.cc:359] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.876758 10846 raft_consensus.cc:385] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.877058 10846 raft_consensus.cc:740] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.858111 10847 tablet_bootstrap.cc:492] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:54.877928 10846 consensus_queue.cc:260] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.879086 10847 ts_tablet_manager.cc:1403] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.146s	user 0.011s	sys 0.008s
I20260709 13:04:54.858696 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d1771d12424af18f74abf7650fab3a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:54.861156 10473 catalog_manager.cc:5697] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.880823 10846 ts_tablet_manager.cc:1434] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:04:54.883111 10727 raft_consensus.cc:2468] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 1.
W20260709 13:04:54.885571 10544 leader_election.cc:343] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.886234 10541 leader_election.cc:304] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:54.888520 11256 raft_consensus.cc:493] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.888892 10847 raft_consensus.cc:359] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:54.889997 11074 raft_consensus.cc:2804] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:54.890231 10847 raft_consensus.cc:385] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.890822 10847 raft_consensus.cc:740] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.890897 11074 raft_consensus.cc:697] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:04:54.891800 10847 consensus_queue.cc:260] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:54.889022 11256 raft_consensus.cc:515] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:54.892315 11074 consensus_queue.cc:237] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:54.896096 11256 leader_election.cc:290] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:54.896971 10847 ts_tablet_manager.cc:1434] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:04:54.900925 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b930f7fe8fb4ec88f4aa71cd7094c9e" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:54.903200 10847 tablet_bootstrap.cc:492] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:54.904389 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b930f7fe8fb4ec88f4aa71cd7094c9e" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:54.909914 10544 leader_election.cc:343] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:54.912462 10544 leader_election.cc:343] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.913107 10544 leader_election.cc:304] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:54.862701 11272 raft_consensus.cc:515] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:54.915756 11254 raft_consensus.cc:2749] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:54.922818 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:54.923640 10573 raft_consensus.cc:3060] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:54.933113 10573 raft_consensus.cc:2468] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:54.935237 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:54.937541 11251 raft_consensus.cc:493] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.938102 11251 raft_consensus.cc:515] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:54.939090 10767 leader_election.cc:343] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.935956 11272 leader_election.cc:290] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:54.942061 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "656245aa555f45a8a6ef044f0da06e46" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:54.944032 10767 leader_election.cc:343] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:54.944598 10846 tablet_bootstrap.cc:492] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:04:54.936695 10848 tablet_bootstrap.cc:654] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.949448 10766 leader_election.cc:304] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:54.951452 11227 raft_consensus.cc:2804] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:54.952035 11227 raft_consensus.cc:697] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:54.952960 11227 consensus_queue.cc:237] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:04:54.954840 11288 log.cc:927] Time spent T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:04:54.952345 10474 catalog_manager.cc:5697] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.960386 10850 tablet_bootstrap.cc:492] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:54.961187 10850 ts_tablet_manager.cc:1403] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.122s	user 0.012s	sys 0.009s
I20260709 13:04:54.964756 10850 raft_consensus.cc:359] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.966199 11251 leader_election.cc:290] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:54.966429 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "656245aa555f45a8a6ef044f0da06e46" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:54.969823 11203 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.990605 10727 raft_consensus.cc:2468] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:54.989198 10476 catalog_manager.cc:5697] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.991128 10850 raft_consensus.cc:385] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:54.993108 10850 raft_consensus.cc:740] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:54.972725 10847 tablet_bootstrap.cc:654] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:54.991791 11203 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:54.998157 10850 consensus_queue.cc:260] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:55.003000 10850 ts_tablet_manager.cc:1434] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 13:04:55.004349 10850 tablet_bootstrap.cc:492] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:55.005689 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:55.006451 11203 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:55.006635 10573 raft_consensus.cc:2468] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:55.007555 10846 tablet_bootstrap.cc:654] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:55.008862 10692 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:55.010198 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:55.011534 10764 leader_election.cc:304] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
W20260709 13:04:55.011876 10693 leader_election.cc:343] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:55.013252 11203 raft_consensus.cc:2804] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:55.013823 11203 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.015044 11203 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:55.008700 10473 catalog_manager.cc:5697] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:55.013444 11251 raft_consensus.cc:2804] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:55.024425 11251 raft_consensus.cc:493] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.047034 11251 raft_consensus.cc:3060] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:55.061559 10847 tablet_bootstrap.cc:492] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:55.062433 10847 ts_tablet_manager.cc:1403] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.164s	user 0.012s	sys 0.006s
I20260709 13:04:55.066294 11271 raft_consensus.cc:493] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.066928 11271 raft_consensus.cc:515] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:55.074332 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c617206d4bb4c0d84e4f8f6cccdbf30" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:55.075367 10576 raft_consensus.cc:2393] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 7ecd836bb7a144dc9aa07a046239d6ce in this term.
I20260709 13:04:55.075448 11203 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:55.077636 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c617206d4bb4c0d84e4f8f6cccdbf30" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:55.078197 11203 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:55.079440 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
W20260709 13:04:55.080843 10693 leader_election.cc:343] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:55.069186 11271 leader_election.cc:290] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:55.083966 10618 leader_election.cc:304] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:55.086688 11201 raft_consensus.cc:3060] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:55.145740 10848 tablet_bootstrap.cc:492] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:55.146981 10848 ts_tablet_manager.cc:1403] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.281s	user 0.017s	sys 0.016s
I20260709 13:04:55.153182 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:55.154080 10573 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:55.155375 10847 raft_consensus.cc:359] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:55.156288 10847 raft_consensus.cc:385] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:55.156595 10847 raft_consensus.cc:740] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:55.161904 10847 consensus_queue.cc:260] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:55.164801 10847 ts_tablet_manager.cc:1434] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.102s	user 0.011s	sys 0.010s
I20260709 13:04:55.166675 10847 tablet_bootstrap.cc:492] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce: Bootstrap starting.
I20260709 13:04:55.174283 11106 raft_consensus.cc:493] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.176973 10847 tablet_bootstrap.cc:654] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:55.196210 11203 raft_consensus.cc:493] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.204907 11203 raft_consensus.cc:515] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:55.206979 10847 tablet_bootstrap.cc:492] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce: No bootstrap required, opened a new log
I20260709 13:04:55.207185 11203 leader_election.cc:290] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:55.207739 10847 ts_tablet_manager.cc:1403] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.000s
I20260709 13:04:55.208218 11251 raft_consensus.cc:515] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:55.177353 11106 raft_consensus.cc:515] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:55.223927 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315bc88bbbfa4be190f5d7581ace8793" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:55.226378 10693 leader_election.cc:343] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:55.228518 10846 tablet_bootstrap.cc:492] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230: No bootstrap required, opened a new log
I20260709 13:04:55.231141 10846 ts_tablet_manager.cc:1403] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.287s	user 0.028s	sys 0.036s
I20260709 13:04:55.201198 10573 raft_consensus.cc:2468] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:55.230134 10848 raft_consensus.cc:359] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:55.232292 10848 raft_consensus.cc:385] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:55.232621 10848 raft_consensus.cc:740] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:55.233644 11200 raft_consensus.cc:493] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.234283 10848 consensus_queue.cc:260] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:55.225494 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315bc88bbbfa4be190f5d7581ace8793" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:55.237895 11200 raft_consensus.cc:515] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:55.239092 11251 leader_election.cc:290] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:55.243348 10653 raft_consensus.cc:3060] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:55.245174 11106 leader_election.cc:290] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:55.247414 10850 tablet_bootstrap.cc:654] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:55.229212 11247 consensus_queue.cc:1048] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:04:55.248059 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d33bf08ed78416ea29aaf59733e35a2" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:55.250312 10847 raft_consensus.cc:359] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:55.257218 10847 raft_consensus.cc:385] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:55.257655 10847 raft_consensus.cc:740] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Initialized, Role: FOLLOWER
I20260709 13:04:55.258507 10847 consensus_queue.cc:260] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:55.241055 11200 leader_election.cc:290] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:55.261582 10692 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:55.262638 10928 raft_consensus.cc:2804] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:55.263212 10928 raft_consensus.cc:697] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:55.267445 10725 raft_consensus.cc:2393] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 1: Already voted for candidate 5a12f1c20c674e7f8242df41a9acf5dc in this term.
I20260709 13:04:55.264467 10928 consensus_queue.cc:237] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:55.278568 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97eca41f7f434c02976a77643991e556" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:55.279465 10576 raft_consensus.cc:2468] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 0.
I20260709 13:04:55.280318 11201 raft_consensus.cc:2749] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:55.283375 11247 raft_consensus.cc:493] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.284862 11247 raft_consensus.cc:515] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
W20260709 13:04:55.287434 10693 leader_election.cc:343] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:55.287955 10693 leader_election.cc:304] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:55.291247 10650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "656245aa555f45a8a6ef044f0da06e46" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:55.291334 11203 raft_consensus.cc:2749] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:55.302685 11274 raft_consensus.cc:493] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.303310 11274 raft_consensus.cc:515] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:55.305980 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "656245aa555f45a8a6ef044f0da06e46" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:55.306787 10725 raft_consensus.cc:3060] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:55.311080 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97eca41f7f434c02976a77643991e556" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:55.313130 10619 leader_election.cc:343] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:55.314792 10847 ts_tablet_manager.cc:1434] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce: Time spent starting tablet: real 0.107s	user 0.011s	sys 0.000s
I20260709 13:04:55.318195 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:55.315855 11247 leader_election.cc:290] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:55.318959 10573 raft_consensus.cc:2468] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:55.320760 10618 leader_election.cc:304] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:04:55.321620 11296 raft_consensus.cc:2804] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:55.322132 11296 raft_consensus.cc:493] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.322471 11296 raft_consensus.cc:3060] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:55.324894 10692 leader_election.cc:304] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
W20260709 13:04:55.327109 10767 leader_election.cc:343] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:55.332319 11247 raft_consensus.cc:2804] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:55.332782 11247 raft_consensus.cc:493] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.333127 11247 raft_consensus.cc:3060] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:55.334640 11274 leader_election.cc:290] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:55.336445 11250 consensus_queue.cc:1048] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:55.337003 10571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e856ce94cd487395746d76e17c555a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:55.343386 11247 raft_consensus.cc:515] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:55.343952 11296 raft_consensus.cc:515] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:55.346246 11247 leader_election.cc:290] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:55.351176 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d33bf08ed78416ea29aaf59733e35a2" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:55.352788 11298 consensus_queue.cc:1048] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:55.353012 10619 leader_election.cc:304] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:55.354085 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:55.354902 10578 raft_consensus.cc:3060] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:55.355243 11247 consensus_queue.cc:1048] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:55.356433 11201 raft_consensus.cc:2749] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:55.283878 10846 raft_consensus.cc:359] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:55.364321 10846 raft_consensus.cc:385] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:55.366094 10846 raft_consensus.cc:740] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:04:55.367269 10846 consensus_queue.cc:260] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:55.390518 11296 leader_election.cc:290] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:55.393018 10571 raft_consensus.cc:2468] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 0.
I20260709 13:04:55.396400 10846 ts_tablet_manager.cc:1434] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.165s	user 0.008s	sys 0.002s
I20260709 13:04:55.397333 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97eca41f7f434c02976a77643991e556" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:55.398034 10573 raft_consensus.cc:3060] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:55.404623 10692 leader_election.cc:304] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:55.414808 10848 ts_tablet_manager.cc:1434] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.267s	user 0.012s	sys 0.021s
I20260709 13:04:55.416630 11263 raft_consensus.cc:2804] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:55.417140 11263 raft_consensus.cc:493] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.417510 11263 raft_consensus.cc:3060] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:55.436044 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97eca41f7f434c02976a77643991e556" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:55.451159 10725 raft_consensus.cc:2468] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
W20260709 13:04:55.453450 10544 consensus_peers.cc:597] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 -> Peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Couldn't send request to peer 5a12f1c20c674e7f8242df41a9acf5dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:55.465696 10764 leader_election.cc:304] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:55.433460 10578 raft_consensus.cc:2468] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:55.467136 11254 consensus_queue.cc:1048] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:55.459362 10573 raft_consensus.cc:2468] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:04:55.471805 11272 raft_consensus.cc:2804] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:55.471882 10618 leader_election.cc:304] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:55.472780 11272 raft_consensus.cc:697] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:55.472856 11200 raft_consensus.cc:2804] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:55.468657 10692 leader_election.cc:304] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:55.492127 10928 raft_consensus.cc:2804] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:55.473537 11200 raft_consensus.cc:697] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:55.495489 10928 raft_consensus.cc:697] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:55.497206 10848 tablet_bootstrap.cc:492] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:55.497409 10928 consensus_queue.cc:237] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:55.477272 11251 consensus_queue.cc:1048] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:04:55.542771 11200 consensus_queue.cc:237] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:55.527352 11272 consensus_queue.cc:237] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:55.534214 10619 leader_election.cc:343] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:55.574590 10619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 47334.1us
I20260709 13:04:55.603111 10850 tablet_bootstrap.cc:492] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:55.603741 10850 ts_tablet_manager.cc:1403] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.600s	user 0.026s	sys 0.004s
I20260709 13:04:55.611045 11263 raft_consensus.cc:515] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:55.616483 11227 raft_consensus.cc:493] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.615972 11263 leader_election.cc:290] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:55.617439 11227 raft_consensus.cc:515] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:55.661219 10850 raft_consensus.cc:359] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:55.662089 10850 raft_consensus.cc:385] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:55.662360 10850 raft_consensus.cc:740] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:55.662992 10850 consensus_queue.cc:260] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:55.676586 10850 ts_tablet_manager.cc:1434] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.000s
W20260709 13:04:55.665542 10767 consensus_peers.cc:597] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:55.692229 10573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b930f7fe8fb4ec88f4aa71cd7094c9e" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:55.694830 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e856ce94cd487395746d76e17c555a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:55.695550 10577 raft_consensus.cc:3060] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:55.701047 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 44424.2us
W20260709 13:04:55.703128 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:04:55.715006 10619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 121053us
W20260709 13:04:55.731900 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 444587us
I20260709 13:04:55.737036 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b930f7fe8fb4ec88f4aa71cd7094c9e" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:55.746183 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:55.749082 11227 leader_election.cc:290] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:55.710903 10573 raft_consensus.cc:2468] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:55.763038 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e856ce94cd487395746d76e17c555a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:55.763360 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
W20260709 13:04:55.767704 10544 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 72860.2us
I20260709 13:04:55.752877 10473 catalog_manager.cc:5697] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:55.779803 10766 leader_election.cc:304] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:55.788501 10799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e856ce94cd487395746d76e17c555a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:04:55.792580 11297 raft_consensus.cc:2804] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:04:55.809103 10767 leader_election.cc:343] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:55.940919 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 184853us
I20260709 13:04:55.818733 10577 raft_consensus.cc:2468] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:04:55.946461 10848 tablet_bootstrap.cc:654] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:55.849359 11297 raft_consensus.cc:493] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:55.963249 11309 log.cc:927] Time spent T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:04:55.956122 11297 raft_consensus.cc:3060] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:56.019007 10616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 96243.9us
W20260709 13:04:56.019033 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 68255.1us
W20260709 13:04:55.972592 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 169358us
W20260709 13:04:56.061195 10541 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 112178us
I20260709 13:04:56.084432 10850 tablet_bootstrap.cc:492] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:56.087589 10692 leader_election.cc:304] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:56.081497 11256 raft_consensus.cc:493] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:56.135051 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 113917us
I20260709 13:04:56.167722 11320 raft_consensus.cc:2804] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:56.168484 11320 raft_consensus.cc:697] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
W20260709 13:04:56.184731 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 48820.5us
W20260709 13:04:56.183360 10541 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 89927.4us
W20260709 13:04:56.188460 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 101038us
I20260709 13:04:56.200851 11320 consensus_queue.cc:237] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:56.142534 11256 raft_consensus.cc:515] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:04:56.232725 11321 log.cc:927] Time spent T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 13:04:56.234148 10693 leader_election.cc:343] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:56.236188 10544 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 102957us
W20260709 13:04:56.238096 10693 leader_election.cc:343] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:56.239905 10693 leader_election.cc:343] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:56.241403 10693 leader_election.cc:343] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:56.194864 11201 raft_consensus.cc:493] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.251609 10850 tablet_bootstrap.cc:654] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:56.259001 10848 tablet_bootstrap.cc:492] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:56.259660 10848 ts_tablet_manager.cc:1403] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.763s	user 0.024s	sys 0.002s
W20260709 13:04:56.256523 11317 log.cc:927] Time spent T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:04:56.245888 11307 consensus_queue.cc:1048] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.048s
I20260709 13:04:56.257545 11201 raft_consensus.cc:515] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:56.351012 11201 leader_election.cc:290] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:56.404196 10576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:56.414057 10651 raft_consensus.cc:3060] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:56.417627 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:56.418018 10848 raft_consensus.cc:359] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
W20260709 13:04:56.397890 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 45183.5us
I20260709 13:04:56.420475 10619 leader_election.cc:304] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:56.422202 10848 raft_consensus.cc:385] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:56.422431 11256 leader_election.cc:290] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:56.422605 10848 raft_consensus.cc:740] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:56.423946 10848 consensus_queue.cc:260] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:56.413601 11297 raft_consensus.cc:515] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:56.431706 11297 leader_election.cc:290] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:56.428907 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a27c6651a83844a69107fdd6f819ebde" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:56.437760 10544 leader_election.cc:343] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:56.447067 10848 ts_tablet_manager.cc:1434] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.187s	user 0.005s	sys 0.005s
I20260709 13:04:56.452198 10848 tablet_bootstrap.cc:492] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:56.443529 11325 consensus_queue.cc:1048] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
W20260709 13:04:56.460619 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 209546us
W20260709 13:04:56.462507 10693 consensus_peers.cc:597] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Couldn't send request to peer 5a12f1c20c674e7f8242df41a9acf5dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.463692 10850 tablet_bootstrap.cc:492] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:56.464365 10850 ts_tablet_manager.cc:1403] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.380s	user 0.015s	sys 0.010s
W20260709 13:04:56.480324 11344 log.cc:927] Time spent T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: Append to log took a long time: real 0.065s	user 0.003s	sys 0.000s
I20260709 13:04:56.490083 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b930f7fe8fb4ec88f4aa71cd7094c9e" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
W20260709 13:04:56.492527 10767 consensus_peers.cc:597] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.503734 11330 consensus_queue.cc:1048] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:04:56.504312 11339 log.cc:927] Time spent T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:04:56.507467 11331 consensus_queue.cc:1048] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:04:56.514796 10693 consensus_peers.cc:597] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.504352 10767 leader_election.cc:343] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:56.563046 10619 consensus_peers.cc:597] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Couldn't send request to peer 5a12f1c20c674e7f8242df41a9acf5dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.559619 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 55332.4us
I20260709 13:04:56.580334 10848 tablet_bootstrap.cc:654] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:56.585126 10850 raft_consensus.cc:359] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:56.596190 10850 raft_consensus.cc:385] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:56.596685 10850 raft_consensus.cc:740] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:56.598062 10850 consensus_queue.cc:260] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:56.607354 10575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b930f7fe8fb4ec88f4aa71cd7094c9e" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:56.607460 11341 consensus_queue.cc:1048] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
I20260709 13:04:56.608155 10575 raft_consensus.cc:3060] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:56.615367 10850 ts_tablet_manager.cc:1434] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.151s	user 0.009s	sys 0.001s
I20260709 13:04:56.640859 11297 raft_consensus.cc:493] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.641638 11297 raft_consensus.cc:515] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:56.644212 11297 leader_election.cc:290] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:56.645251 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d1771d12424af18f74abf7650fab3a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:56.655274 10850 tablet_bootstrap.cc:492] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:56.682703 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d1771d12424af18f74abf7650fab3a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:56.687633 11342 raft_consensus.cc:2749] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:56.694542 10798 raft_consensus.cc:3060] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:56.770960 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a27c6651a83844a69107fdd6f819ebde" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:56.775593 10544 leader_election.cc:343] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:56.776742 10544 leader_election.cc:304] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:56.791292 11327 raft_consensus.cc:2749] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:56.856196 10764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 83235.3us
I20260709 13:04:56.866869 10928 consensus_queue.cc:1048] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:56.882149 11351 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.883064 11351 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:56.885138 10850 tablet_bootstrap.cc:654] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:56.896476 11364 log.cc:927] Time spent T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:04:56.896848 11351 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:56.898384 10848 tablet_bootstrap.cc:492] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:56.899219 10848 ts_tablet_manager.cc:1403] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.447s	user 0.025s	sys 0.033s
I20260709 13:04:56.869706 10575 raft_consensus.cc:2468] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:56.920167 10848 raft_consensus.cc:359] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:56.924535 10848 raft_consensus.cc:385] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:56.925027 10848 raft_consensus.cc:740] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:56.926285 10848 consensus_queue.cc:260] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
W20260709 13:04:56.930068 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 49064.2us
I20260709 13:04:56.931547 10848 ts_tablet_manager.cc:1434] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:04:56.938880 10848 tablet_bootstrap.cc:492] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:56.939534 10850 tablet_bootstrap.cc:492] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:56.940116 10850 ts_tablet_manager.cc:1403] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.291s	user 0.007s	sys 0.013s
I20260709 13:04:56.943243 10850 raft_consensus.cc:359] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:56.953190 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:56.959128 11311 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.959807 10474 catalog_manager.cc:5697] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:56.977742 10766 leader_election.cc:304] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:56.991556 10474 catalog_manager.cc:5697] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:56.995221 10848 tablet_bootstrap.cc:654] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:56.989708 11311 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:56.995718 10850 raft_consensus.cc:385] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:57.009444 11311 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
W20260709 13:04:57.011662 10523 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:57.009871 10651 raft_consensus.cc:2468] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 0.
I20260709 13:04:57.010113 10850 raft_consensus.cc:740] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:56.997926 11251 raft_consensus.cc:2749] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:57.014685 10850 consensus_queue.cc:260] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:57.012941 10722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:57.017277 10850 ts_tablet_manager.cc:1434] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.002s
I20260709 13:04:57.020682 10848 tablet_bootstrap.cc:492] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:57.021358 10848 ts_tablet_manager.cc:1403] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.083s	user 0.011s	sys 0.001s
I20260709 13:04:57.022859 10850 tablet_bootstrap.cc:492] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:57.025986 10848 raft_consensus.cc:359] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:57.027019 10848 raft_consensus.cc:385] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:57.027451 10848 raft_consensus.cc:740] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
W20260709 13:04:57.001811 10544 consensus_peers.cc:597] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:57.028321 10848 consensus_queue.cc:260] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:04:57.031100 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 53490.7us
I20260709 13:04:57.039538 10850 tablet_bootstrap.cc:654] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:57.046587 10848 ts_tablet_manager.cc:1434] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:04:57.064574 10645 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:57.071890 10848 tablet_bootstrap.cc:492] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
W20260709 13:04:57.076493 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 44630.1us
I20260709 13:04:57.077622 10766 leader_election.cc:304] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:57.081153 10577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:57.085933 10544 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:57.104051 10618 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:57.106580 11349 raft_consensus.cc:2804] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:57.107201 11349 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7ecd836bb7a144dc9aa07a046239d6ce)
I20260709 13:04:57.107741 11349 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:57.119689 11307 consensus_queue.cc:1048] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:57.123508 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:57.105360 11251 raft_consensus.cc:2804] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:57.142704 11251 raft_consensus.cc:697] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:57.143963 11251 consensus_queue.cc:237] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:04:57.163769 10693 consensus_peers.cc:597] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707): Couldn't send request to peer 5a12f1c20c674e7f8242df41a9acf5dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.198238 10794 raft_consensus.cc:3060] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:57.208694 10848 tablet_bootstrap.cc:654] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:57.222682 11368 raft_consensus.cc:493] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.223471 11368 raft_consensus.cc:515] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:57.231871 10653 raft_consensus.cc:3060] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:57.235082 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 59027.1us
W20260709 13:04:57.236670 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 100042us
I20260709 13:04:57.205114 11349 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:57.251296 11349 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:57.252487 10652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:57.259503 11200 raft_consensus.cc:2749] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:57.265730 11368 leader_election.cc:290] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:57.267354 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:57.281834 10644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315bc88bbbfa4be190f5d7581ace8793" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:04:57.287572 10767 leader_election.cc:343] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:57.288729 10850 tablet_bootstrap.cc:492] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:57.294900 10850 ts_tablet_manager.cc:1403] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.272s	user 0.041s	sys 0.038s
W20260709 13:04:57.321867 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 60650.6us
I20260709 13:04:57.323350 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315bc88bbbfa4be190f5d7581ace8793" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:57.326330 10724 raft_consensus.cc:2468] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:57.333092 10541 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:04:57.347617 10764 leader_election.cc:304] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:57.339973 11310 raft_consensus.cc:2749] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:57.350888 10850 raft_consensus.cc:359] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:57.353073 10850 raft_consensus.cc:385] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:57.353327 10850 raft_consensus.cc:740] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:57.354110 10850 consensus_queue.cc:260] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:57.358640 10848 tablet_bootstrap.cc:492] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:57.359724 10848 ts_tablet_manager.cc:1403] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.288s	user 0.012s	sys 0.017s
I20260709 13:04:57.350967 11251 raft_consensus.cc:2804] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:57.370123 11251 raft_consensus.cc:493] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.370733 11251 raft_consensus.cc:3060] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:57.379561 10570 raft_consensus.cc:1240] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Rejecting Update request from peer 7ecd836bb7a144dc9aa07a046239d6ce for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:57.416191 10850 ts_tablet_manager.cc:1434] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.121s	user 0.009s	sys 0.000s
I20260709 13:04:57.399053 10848 raft_consensus.cc:359] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:57.419713 10848 raft_consensus.cc:385] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:57.420126 10848 raft_consensus.cc:740] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:57.428952 10848 consensus_queue.cc:260] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:57.445083 10928 consensus_queue.cc:1059] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:57.449385 10850 tablet_bootstrap.cc:492] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:57.457666 10848 ts_tablet_manager.cc:1434] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.097s	user 0.013s	sys 0.000s
I20260709 13:04:57.481638 10848 tablet_bootstrap.cc:492] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:57.516366 11247 raft_consensus.cc:3055] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:57.517048 11247 raft_consensus.cc:740] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:04:57.518137 11247 consensus_queue.cc:260] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:57.519815 11247 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:57.525301 10473 catalog_manager.cc:5697] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:57.529870 11393 log.cc:927] Time spent T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:04:57.581298 10474 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37090 (request call id 68) took 1250 ms. Trace:
I20260709 13:04:57.587386 11368 consensus_queue.cc:1048] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:57.528249 11251 raft_consensus.cc:515] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:57.601679 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315bc88bbbfa4be190f5d7581ace8793" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
W20260709 13:04:57.603372 10767 leader_election.cc:343] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:57.604671 11251 leader_election.cc:290] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:57.606988 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315bc88bbbfa4be190f5d7581ace8793" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:57.607704 10727 raft_consensus.cc:3060] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:57.606757 10473 catalog_manager.cc:5697] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 1 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } }
W20260709 13:04:57.608357 10618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 129495us
I20260709 13:04:57.611780 11343 raft_consensus.cc:493] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.615265 10848 tablet_bootstrap.cc:654] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:57.622527 10727 raft_consensus.cc:2468] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:57.624208 10764 leader_election.cc:304] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:57.635742 11369 raft_consensus.cc:2804] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:57.636588 11369 raft_consensus.cc:697] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:57.637542 11369 consensus_queue.cc:237] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:57.616380 11343 raft_consensus.cc:515] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:57.643894 10644 raft_consensus.cc:3060] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:57.653595 11343 leader_election.cc:290] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:57.654616 10575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b930f7fe8fb4ec88f4aa71cd7094c9e" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:57.675197 11328 consensus_queue.cc:1048] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:57.663128 10476 catalog_manager.cc:5697] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:57.582217 10474 rpcz_store.cc:276] 0709 13:04:56.342390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.345104 (+  2714us) service_pool.cc:224] Handling call
0709 13:04:56.525422 (+180318us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540002b054. stack: 00007fda245bb343 00007fda245bb240 00007fda23f4a1d4 00007fda2dc88304 00007fda2dc88299 00007fda27644090 00007fda276e2f3a 00007fda2d4991cc 00007fda2d59737e 00007fda2ba582a4 00007fda2ba58232 00007fda2ba58161 00007fda2ba580dc 00007fda2ba57392 00007fda25a1548c 00007fda25a148c6
0709 13:04:57.581154 (+1055732us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:57.031767 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:57.032423 (+   656us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:57.032465 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:04:57.032484 (+    19us) tablet.cc:662] Decoding operations
0709 13:04:57.032944 (+   460us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:57.033039 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:57.033069 (+    30us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:04:57.033436 (+   367us) tablet.cc:701] Row locks acquired
0709 13:04:57.033455 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:04:57.033675 (+   220us) write_op.cc:270] Start()
0709 13:04:57.033793 (+   118us) write_op.cc:276] Timestamp: P: 1783602297033622 usec, L: 0
0709 13:04:57.033821 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:04:57.034717 (+   896us) log.cc:844] Serialized 928 byte log entry
0709 13:04:57.277078 (+242361us) op_driver.cc:464] REPLICATION: finished
0709 13:04:57.442032 (+164954us) write_op.cc:301] APPLY: starting
0709 13:04:57.444460 (+  2428us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:57.451907 (+  7447us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:57.467240 (+ 15333us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:57.479724 (+ 12484us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:57.486553 (+  6829us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:04:57.486605 (+    52us) write_op.cc:312] APPLY: finished
0709 13:04:57.486831 (+   226us) log.cc:844] Serialized 28 byte log entry
0709 13:04:57.487011 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:57.487115 (+   104us) write_op.cc:454] Released schema lock
0709 13:04:57.487175 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:57.487549 (+   374us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:57.487578 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":233285376,"child_traces":[["op",{"apply.queue_time_us":150454,"apply.run_cpu_time_us":2763,"apply.run_wall_time_us":51549,"num_ops":2,"prepare.queue_time_us":31659,"prepare.run_cpu_time_us":3345,"prepare.run_wall_time_us":3345,"replication_time_us":243101,"thread_start_us":104695,"threads_started":2}]]}
W20260709 13:04:57.732618 10764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 108513us
I20260709 13:04:57.770104 10850 tablet_bootstrap.cc:654] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:57.837929 10764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 58378.9us
I20260709 13:04:57.861994 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b930f7fe8fb4ec88f4aa71cd7094c9e" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:04:57.857159 11397 raft_consensus.cc:493] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.865713 10619 leader_election.cc:304] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:57.864694 11397 raft_consensus.cc:515] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:57.900955 11343 raft_consensus.cc:2749] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:57.929801 11397 leader_election.cc:290] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:04:57.931720 10801 raft_consensus.cc:3060] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:57.936060 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e856ce94cd487395746d76e17c555a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:57.936456 10571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e856ce94cd487395746d76e17c555a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:57.941308 10766 leader_election.cc:304] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:57.949150 10848 tablet_bootstrap.cc:492] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:57.953472 10848 ts_tablet_manager.cc:1403] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.482s	user 0.034s	sys 0.028s
W20260709 13:04:57.993948 10767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 47368.3us
I20260709 13:04:58.004065 10848 raft_consensus.cc:359] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:58.004905 10848 raft_consensus.cc:385] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:58.005182 10848 raft_consensus.cc:740] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:58.019353 10848 consensus_queue.cc:260] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:58.037456 10848 ts_tablet_manager.cc:1434] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.000s
I20260709 13:04:58.056949 10850 tablet_bootstrap.cc:492] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:58.057730 10850 ts_tablet_manager.cc:1403] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.614s	user 0.021s	sys 0.026s
I20260709 13:04:58.060436 10848 tablet_bootstrap.cc:492] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc: Bootstrap starting.
I20260709 13:04:58.044909 11297 raft_consensus.cc:2749] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:58.061286 10850 raft_consensus.cc:359] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:58.062256 10850 raft_consensus.cc:385] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:58.062547 10850 raft_consensus.cc:740] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:58.063303 10850 consensus_queue.cc:260] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:58.066623 10850 ts_tablet_manager.cc:1434] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:04:58.067973 10850 tablet_bootstrap.cc:492] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:58.069244 10848 tablet_bootstrap.cc:654] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:58.075297 10850 tablet_bootstrap.cc:654] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:58.078043 10848 tablet_bootstrap.cc:492] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc: No bootstrap required, opened a new log
I20260709 13:04:58.078758 10848 ts_tablet_manager.cc:1403] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.001s
I20260709 13:04:58.082754 10848 raft_consensus.cc:359] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:58.157186 10848 raft_consensus.cc:385] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:58.158038 10848 raft_consensus.cc:740] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Initialized, Role: FOLLOWER
I20260709 13:04:58.160225 10848 consensus_queue.cc:260] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:04:58.176776 10850 tablet_bootstrap.cc:492] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:58.178495 10850 ts_tablet_manager.cc:1403] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.111s	user 0.015s	sys 0.004s
W20260709 13:04:58.184371 10767 consensus_peers.cc:597] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:58.196547 10850 raft_consensus.cc:359] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:58.199783 10850 raft_consensus.cc:385] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:58.200184 10850 raft_consensus.cc:740] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:58.203645 10850 consensus_queue.cc:260] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:58.226956 10850 ts_tablet_manager.cc:1434] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.048s	user 0.014s	sys 0.005s
I20260709 13:04:58.230196 10850 tablet_bootstrap.cc:492] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c: Bootstrap starting.
I20260709 13:04:58.286828 10848 ts_tablet_manager.cc:1434] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc: Time spent starting tablet: real 0.208s	user 0.009s	sys 0.008s
I20260709 13:04:58.290447 10476 catalog_manager.cc:5697] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:58.335819 10850 tablet_bootstrap.cc:654] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:58.337679 11402 raft_consensus.cc:493] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.338892 11402 raft_consensus.cc:515] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:58.344902 11323 raft_consensus.cc:493] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.346642 11323 raft_consensus.cc:515] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
W20260709 13:04:58.351639 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 151810us
I20260709 13:04:58.376055 10646 raft_consensus.cc:3060] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:58.373797 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 145282us
I20260709 13:04:58.384029 11402 leader_election.cc:290] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:58.405452 11323 leader_election.cc:290] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:58.445096 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a27c6651a83844a69107fdd6f819ebde" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:58.448132 10798 raft_consensus.cc:3060] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:58.450348 10571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a27c6651a83844a69107fdd6f819ebde" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:58.451659 10571 raft_consensus.cc:2468] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 0.
I20260709 13:04:58.472534 11402 consensus_queue.cc:1048] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 13:04:58.532179 10569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3d1fe775e74c4684020e6c5732ce13" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
W20260709 13:04:58.533341 10616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 97205.2us
W20260709 13:04:58.506126 10767 leader_election.cc:343] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:58.595273 10721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3d1fe775e74c4684020e6c5732ce13" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:58.606324 10644 raft_consensus.cc:3060] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:58.640956 10850 tablet_bootstrap.cc:492] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c: No bootstrap required, opened a new log
I20260709 13:04:58.641359 10766 leader_election.cc:304] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
W20260709 13:04:58.645090 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 112946us
I20260709 13:04:58.646196 11402 raft_consensus.cc:2804] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:58.646754 11402 raft_consensus.cc:493] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.647120 11402 raft_consensus.cc:3060] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:58.641688 10850 ts_tablet_manager.cc:1403] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent bootstrapping tablet: real 0.412s	user 0.014s	sys 0.020s
I20260709 13:04:58.686117 10616 leader_election.cc:304] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:58.719332 10850 raft_consensus.cc:359] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:58.724440 10850 raft_consensus.cc:385] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:58.740456 10850 raft_consensus.cc:740] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Initialized, Role: FOLLOWER
I20260709 13:04:58.744074 11411 raft_consensus.cc:493] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a12f1c20c674e7f8242df41a9acf5dc)
I20260709 13:04:58.745038 11411 raft_consensus.cc:515] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:04:58.759389 10850 consensus_queue.cc:260] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:58.752581 11402 raft_consensus.cc:515] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:58.813896 10850 ts_tablet_manager.cc:1434] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c: Time spent starting tablet: real 0.150s	user 0.009s	sys 0.001s
W20260709 13:04:58.824455 10544 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 74444.6us
I20260709 13:04:58.825294 11296 raft_consensus.cc:2749] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:58.836205 11402 leader_election.cc:290] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:04:58.841753 10646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a27c6651a83844a69107fdd6f819ebde" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:58.842679 10646 raft_consensus.cc:3060] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:58.862620 11411 leader_election.cc:290] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:04:58.864527 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315bc88bbbfa4be190f5d7581ace8793" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:58.867707 10801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315bc88bbbfa4be190f5d7581ace8793" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:04:58.890185 10541 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 77653.4us
I20260709 13:04:58.984860 10646 raft_consensus.cc:2468] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:58.961722 10619 leader_election.cc:304] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
W20260709 13:04:58.995128 10619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 113937us
I20260709 13:04:59.004640 11394 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7ecd836bb7a144dc9aa07a046239d6ce)
I20260709 13:04:59.005425 11394 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:59.008883 11394 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:58.995064 11296 raft_consensus.cc:2749] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:59.051270 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:59.052171 10724 raft_consensus.cc:2468] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 2.
I20260709 13:04:59.062498 10616 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:04:59.063794 11411 raft_consensus.cc:2804] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:59.068745 11411 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7ecd836bb7a144dc9aa07a046239d6ce)
I20260709 13:04:59.069195 11411 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:59.088481 10571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a27c6651a83844a69107fdd6f819ebde" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:59.089337 10571 raft_consensus.cc:3060] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:59.081843 10767 leader_election.cc:304] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:04:59.110385 11369 raft_consensus.cc:2804] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:59.111071 11369 raft_consensus.cc:697] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:04:59.112257 11369 consensus_queue.cc:237] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:04:59.128165 10571 raft_consensus.cc:2468] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:04:59.110127 10574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230" is_pre_election: true
I20260709 13:04:59.148337 10574 raft_consensus.cc:2393] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 2: Already voted for candidate bef3584ebb2c4a2ea619936c71900230 in this term.
I20260709 13:04:59.173005 11411 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:59.184566 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:59.187732 11411 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:59.200877 10570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
I20260709 13:04:59.202082 10570 raft_consensus.cc:2393] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 73db933c4a6848d1b2aeb56813f7bb2c in current term 2: Already voted for candidate bef3584ebb2c4a2ea619936c71900230 in this term.
I20260709 13:04:59.208369 10723 raft_consensus.cc:2468] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 2.
I20260709 13:04:59.210780 10616 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:04:59.211716 11411 raft_consensus.cc:2804] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:04:59.214255 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 54299.9us
I20260709 13:04:59.212430 11411 raft_consensus.cc:697] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:59.219758 11411 consensus_queue.cc:237] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:59.249734 10472 catalog_manager.cc:5697] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:59.258997 10474 catalog_manager.cc:5697] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 1 to 2, leader changed from 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195) to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 2 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:59.500037 11419 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.500741 11419 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:59.504979 11419 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:59.505328 10646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:04:59.505625 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:04:59.506974 10723 raft_consensus.cc:2468] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:04:59.508291 10541 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:04:59.509200 11419 raft_consensus.cc:2804] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:59.509889 11419 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.510344 11419 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:59.519407 11419 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:59.521648 11419 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:04:59.522428 10646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:04:59.522776 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:04:59.523334 10723 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:59.529874 10723 raft_consensus.cc:2468] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 3.
I20260709 13:04:59.531044 10541 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:04:59.531770 11419 raft_consensus.cc:2804] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:59.533048 11419 raft_consensus.cc:697] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:04:59.534054 11419 consensus_queue.cc:237] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:59.546943 10472 catalog_manager.cc:5697] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 2 to 3, leader changed from 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:59.681052 10723 raft_consensus.cc:1240] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Rejecting Update request from peer 73db933c4a6848d1b2aeb56813f7bb2c for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:59.682541 11411 consensus_queue.cc:1059] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:59.683329 11296 raft_consensus.cc:3055] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:59.683728 11296 raft_consensus.cc:740] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:04:59.684217 11432 consensus_queue.cc:1048] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:59.684566 11296 consensus_queue.cc:260] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:04:59.685860 11296 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:59.707466 11432 consensus_queue.cc:1048] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:59.953902 10723 raft_consensus.cc:1275] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:59.955619 11419 consensus_queue.cc:1048] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:59.975656 10653 raft_consensus.cc:1275] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:59.976130 10653 pending_rounds.cc:85] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c: Aborting all ops after (but not including) 1
I20260709 13:04:59.976409 10653 pending_rounds.cc:107] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:59.976697 10653 raft_consensus.cc:2889] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:59.979370 11349 consensus_queue.cc:1048] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:00.501277   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:05:00.505497 10618 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:00.510177 10618 consensus_peers.cc:597] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.554129   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:00.555938   431 tablet_replica.cc:333] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.557925   431 raft_consensus.cc:2243] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.558854   431 raft_consensus.cc:2272] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.564740   431 tablet_replica.cc:333] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.566135   431 raft_consensus.cc:2243] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.566912   431 raft_consensus.cc:2272] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.567690 10618 consensus_peers.cc:597] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.572310   431 tablet_replica.cc:333] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.574025   431 raft_consensus.cc:2243] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.574846   431 raft_consensus.cc:2272] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.579603   431 tablet_replica.cc:333] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.580916   431 raft_consensus.cc:2243] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.581743   431 raft_consensus.cc:2272] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.585966   431 tablet_replica.cc:333] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.587289   431 raft_consensus.cc:2243] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:00.588703   431 raft_consensus.cc:2272] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.592474 10692 consensus_peers.cc:597] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.593307   431 tablet_replica.cc:333] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.594661   431 raft_consensus.cc:2243] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:00.595317 10692 consensus_peers.cc:597] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.595573   431 raft_consensus.cc:2272] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.598537 10766 consensus_peers.cc:597] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.599609   431 tablet_replica.cc:333] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.600649   431 raft_consensus.cc:2243] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.601337   431 raft_consensus.cc:2272] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.606797   431 tablet_replica.cc:333] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.608036   431 raft_consensus.cc:2243] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.608799   431 raft_consensus.cc:2272] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.611650   431 tablet_replica.cc:333] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.612401   431 raft_consensus.cc:2243] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.613202   431 raft_consensus.cc:2272] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.616644   431 tablet_replica.cc:333] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.617771   431 raft_consensus.cc:2243] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:00.619081   431 raft_consensus.cc:2272] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.622889   431 tablet_replica.cc:333] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.623966   431 raft_consensus.cc:2243] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:00.625093   431 raft_consensus.cc:2272] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.629334 10692 consensus_peers.cc:597] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.629629   431 tablet_replica.cc:333] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.630553   431 raft_consensus.cc:2243] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.631181   431 raft_consensus.cc:2272] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.634923   431 tablet_replica.cc:333] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.636011   431 raft_consensus.cc:2243] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:00.637058   431 raft_consensus.cc:2272] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.640704   431 tablet_replica.cc:333] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.641669   431 raft_consensus.cc:2243] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.642251   431 raft_consensus.cc:2272] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.645282   431 tablet_replica.cc:333] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.646203   431 raft_consensus.cc:2243] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.646822   431 raft_consensus.cc:2272] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.650126   431 tablet_replica.cc:333] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.651432   431 raft_consensus.cc:2243] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.652223   431 raft_consensus.cc:2272] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.653141 10692 consensus_peers.cc:597] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.654320 10618 consensus_peers.cc:597] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.655085 10618 consensus_peers.cc:597] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.656759   431 tablet_replica.cc:333] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
W20260709 13:05:00.657738 10692 consensus_peers.cc:597] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.658082   431 raft_consensus.cc:2243] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.658854   431 raft_consensus.cc:2272] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.662451   431 tablet_replica.cc:333] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.663308   431 raft_consensus.cc:2243] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.663892   431 raft_consensus.cc:2272] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.663916 10766 consensus_peers.cc:597] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.665751 10692 consensus_peers.cc:597] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.665979 10766 consensus_peers.cc:597] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.667609 10692 consensus_peers.cc:597] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.668735   431 tablet_replica.cc:333] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.669991   431 raft_consensus.cc:2243] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.670879   431 raft_consensus.cc:2272] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.673650   431 tablet_replica.cc:333] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.674340   431 raft_consensus.cc:2243] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.674855   431 raft_consensus.cc:2272] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.678007   431 tablet_replica.cc:333] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.679126   431 raft_consensus.cc:2243] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:05:00.679685 10692 consensus_peers.cc:597] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.680307   431 raft_consensus.cc:2272] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.683369   431 tablet_replica.cc:333] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.684388   431 raft_consensus.cc:2243] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.685146   431 raft_consensus.cc:2272] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.688747   431 tablet_replica.cc:333] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.689847   431 raft_consensus.cc:2243] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:00.691133   431 raft_consensus.cc:2272] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.691246 10692 consensus_peers.cc:597] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.695595 10766 consensus_peers.cc:597] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.699748   431 tablet_replica.cc:333] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.700837   431 raft_consensus.cc:2243] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:00.702268   431 raft_consensus.cc:2272] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.706753   431 tablet_replica.cc:333] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.707741   431 raft_consensus.cc:2243] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.708451   431 raft_consensus.cc:2272] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.710378 10766 consensus_peers.cc:597] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.710378 10692 consensus_peers.cc:597] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.711941   431 tablet_replica.cc:333] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.712741   431 raft_consensus.cc:2243] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:00.713794   431 raft_consensus.cc:2272] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.716814   431 tablet_replica.cc:333] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.717581   431 raft_consensus.cc:2243] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.718109   431 raft_consensus.cc:2272] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.720722   431 tablet_replica.cc:333] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.721413   431 raft_consensus.cc:2243] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.721978   431 raft_consensus.cc:2272] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.725160   431 tablet_replica.cc:333] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.726006   431 raft_consensus.cc:2243] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:00.727352   431 raft_consensus.cc:2272] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.730875   431 tablet_replica.cc:333] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.731807   431 raft_consensus.cc:2243] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.732461   431 raft_consensus.cc:2272] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.732882 10766 consensus_peers.cc:597] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.736116   431 tablet_replica.cc:333] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.737425   431 raft_consensus.cc:2243] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.738119   431 raft_consensus.cc:2272] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.741676   431 tablet_replica.cc:333] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.742663   431 raft_consensus.cc:2243] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.743348   431 raft_consensus.cc:2272] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.746675   431 tablet_replica.cc:333] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.747674   431 raft_consensus.cc:2243] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.748337   431 raft_consensus.cc:2272] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.748652 10766 consensus_peers.cc:597] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.748728 10692 consensus_peers.cc:597] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.751360   431 tablet_replica.cc:333] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.752157   431 raft_consensus.cc:2243] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.752700   431 raft_consensus.cc:2272] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.755434   431 tablet_replica.cc:333] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.756114   431 raft_consensus.cc:2243] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:00.757225   431 raft_consensus.cc:2272] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.761039   431 tablet_replica.cc:333] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.762029   431 raft_consensus.cc:2243] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:05:00.762437 10618 consensus_peers.cc:597] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.763437   431 raft_consensus.cc:2272] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.767016   431 tablet_replica.cc:333] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
W20260709 13:05:00.767742 10692 consensus_peers.cc:597] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.767835   431 raft_consensus.cc:2243] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:00.769333   431 raft_consensus.cc:2272] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.771874   431 tablet_replica.cc:333] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.772500   431 raft_consensus.cc:2243] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.773015   431 raft_consensus.cc:2272] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.775475   431 tablet_replica.cc:333] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.776057   431 raft_consensus.cc:2243] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.776571   431 raft_consensus.cc:2272] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.779701   431 tablet_replica.cc:333] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.780494   431 raft_consensus.cc:2243] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.781090   431 raft_consensus.cc:2272] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.780910 10692 consensus_peers.cc:597] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.784268   431 tablet_replica.cc:333] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.784937   431 raft_consensus.cc:2243] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.785442   431 raft_consensus.cc:2272] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.788583   431 tablet_replica.cc:333] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
W20260709 13:05:00.789222 10618 consensus_peers.cc:597] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.789577   431 raft_consensus.cc:2243] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.790150   431 raft_consensus.cc:2272] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.792675   431 tablet_replica.cc:333] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.793383   431 raft_consensus.cc:2243] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.793896   431 raft_consensus.cc:2272] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.796344   431 tablet_replica.cc:333] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.796826   431 raft_consensus.cc:2243] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.797242   431 raft_consensus.cc:2272] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.799880   431 tablet_replica.cc:333] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230: stopping tablet replica
I20260709 13:05:00.800349   431 raft_consensus.cc:2243] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.800750   431 raft_consensus.cc:2272] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:00.811919 10618 consensus_peers.cc:597] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.820437 10766 consensus_peers.cc:597] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.841774 10692 consensus_peers.cc:597] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.848321 10692 consensus_peers.cc:597] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:00.853456 10766 consensus_peers.cc:597] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:00.901314   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:05:00.912421 10618 consensus_peers.cc:597] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:01.031726   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:01.054759 10692 consensus_peers.cc:597] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:01.055505 11453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:01.055902 11455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:01.059660   431 server_base.cc:1061] running on GCE node
W20260709 13:05:01.063670 11457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:01.068415   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:01.068826   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:01.069038   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602301069018 us; error 0 us; skew 500 ppm
I20260709 13:05:01.069905   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:01.077399   431 webserver.cc:533] Webserver started at http://127.0.107.193:33297/ using document root <none> and password file <none>
I20260709 13:05:01.078552   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:01.078883   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:05:01.084204   431 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:05:01.097227 11463 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:01.098352   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:05:01.098745   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-0-root
uuid: "bef3584ebb2c4a2ea619936c71900230"
format_stamp: "Formatted at 2026-07-09 13:04:42 on dist-test-slave-l8g5"
I20260709 13:05:01.099159   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601924925497-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:01.118283   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:01.120340   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:01.124382   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:01.149190 11470 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:05:01.592602   431 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:05:01.593124   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.459s	user 0.013s	sys 0.009s
I20260709 13:05:01.593554   431 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:05:01.602883 11470 tablet_bootstrap.cc:492] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:01.619647 11470 tablet_bootstrap.cc:492] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.620745 11470 tablet_bootstrap.cc:492] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:01.621559 11470 ts_tablet_manager.cc:1403] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 13:05:01.624910 11470 raft_consensus.cc:359] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:01.625881 11470 raft_consensus.cc:740] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.626868 11470 consensus_queue.cc:260] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:01.630235 11470 ts_tablet_manager.cc:1434] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:01.631282 11470 tablet_bootstrap.cc:492] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:01.647958 11470 tablet_bootstrap.cc:492] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.648754 11470 tablet_bootstrap.cc:492] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:01.649319 11470 ts_tablet_manager.cc:1403] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 13:05:01.651301 11470 raft_consensus.cc:359] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:01.651855 11470 raft_consensus.cc:740] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.652371 11470 consensus_queue.cc:260] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:01.653856 11470 ts_tablet_manager.cc:1434] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:05:01.654524 11470 tablet_bootstrap.cc:492] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:01.672799 11470 tablet_bootstrap.cc:492] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.672499 11460 raft_consensus.cc:493] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.673981 11470 tablet_bootstrap.cc:492] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:01.673688 11460 raft_consensus.cc:515] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:01.674770 11470 ts_tablet_manager.cc:1403] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 13:05:01.677647 11470 raft_consensus.cc:359] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:01.678638 11470 raft_consensus.cc:740] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.679399 11470 consensus_queue.cc:260] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:01.681499 11470 ts_tablet_manager.cc:1434] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:05:01.682440 11470 tablet_bootstrap.cc:492] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:01.682977 11460 leader_election.cc:290] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
W20260709 13:05:01.682507 10618 leader_election.cc:336] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:01.677588 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95de1505ade4f5389a46d7aca7d7136" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:01.686062 10616 leader_election.cc:304] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:01.687431 11460 raft_consensus.cc:2749] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:01.705245 11470 tablet_bootstrap.cc:492] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.706358 11470 tablet_bootstrap.cc:492] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:01.707057 11470 ts_tablet_manager.cc:1403] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.025s	user 0.022s	sys 0.000s
I20260709 13:05:01.709853 11470 raft_consensus.cc:359] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.710608 11470 raft_consensus.cc:740] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.714399 11470 consensus_queue.cc:260] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.716595 11470 ts_tablet_manager.cc:1434] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:05:01.718444 11470 tablet_bootstrap.cc:492] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:01.742530 11470 tablet_bootstrap.cc:492] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.743415 11470 tablet_bootstrap.cc:492] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:01.743999 11470 ts_tablet_manager.cc:1403] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.027s	user 0.022s	sys 0.000s
I20260709 13:05:01.746192 11470 raft_consensus.cc:359] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:01.746840 11470 raft_consensus.cc:740] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.747399 11470 consensus_queue.cc:260] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:01.748924 11470 ts_tablet_manager.cc:1434] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:01.749671 11470 tablet_bootstrap.cc:492] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:01.753783 11472 raft_consensus.cc:493] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.754444 11472 raft_consensus.cc:515] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
W20260709 13:05:01.767519 10766 leader_election.cc:336] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:01.768332 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d1771d12424af18f74abf7650fab3a" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:01.770340 10764 leader_election.cc:304] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:01.771143 11297 raft_consensus.cc:2749] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:01.771368 11472 leader_election.cc:290] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:01.771936 11470 tablet_bootstrap.cc:492] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.772976 11470 tablet_bootstrap.cc:492] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:01.772969 11450 raft_consensus.cc:493] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.774053 11470 ts_tablet_manager.cc:1403] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.004s
I20260709 13:05:01.773800 11450 raft_consensus.cc:515] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:01.775378 11407 raft_consensus.cc:493] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.775871 11407 raft_consensus.cc:515] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.776535 11450 leader_election.cc:290] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:01.778707 11407 leader_election.cc:290] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:01.776975 11470 raft_consensus.cc:359] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.780889 11470 raft_consensus.cc:740] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.781749 11470 consensus_queue.cc:260] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.779976 10646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3d1fe775e74c4684020e6c5732ce13" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:05:01.782915 10692 leader_election.cc:336] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
W20260709 13:05:01.783522 10692 leader_election.cc:336] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:01.780215 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:01.784862 10693 leader_election.cc:304] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:01.785573 11470 ts_tablet_manager.cc:1434] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:05:01.786217 10693 leader_election.cc:304] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:01.786629 11470 tablet_bootstrap.cc:492] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:01.786979 11450 raft_consensus.cc:2749] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:01.787699 11450 raft_consensus.cc:2749] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:01.822469 11470 tablet_bootstrap.cc:492] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.823673 11470 tablet_bootstrap.cc:492] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:01.824510 11470 ts_tablet_manager.cc:1403] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.007s
I20260709 13:05:01.827415 11470 raft_consensus.cc:359] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.828167 11470 raft_consensus.cc:740] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.829039 11470 consensus_queue.cc:260] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.831532 11470 ts_tablet_manager.cc:1434] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:05:01.832412 11470 tablet_bootstrap.cc:492] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:01.843641 11297 raft_consensus.cc:493] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.844656 11450 raft_consensus.cc:493] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.844368 11297 raft_consensus.cc:515] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:01.845371 11450 raft_consensus.cc:515] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.849423 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ecb24ddae546da9915163c05742a7e" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:01.851850 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:05:01.855438 10766 leader_election.cc:336] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:01.856050 10766 leader_election.cc:304] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:01.853824 11297 leader_election.cc:290] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:01.857640 11297 raft_consensus.cc:2749] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:01.859977 11450 leader_election.cc:290] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:01.861865 11470 tablet_bootstrap.cc:492] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:01.861732 10692 leader_election.cc:336] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:01.862407 10692 leader_election.cc:304] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:01.863025 11470 tablet_bootstrap.cc:492] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:01.863243 11450 raft_consensus.cc:2749] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:01.863791 11470 ts_tablet_manager.cc:1403] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.008s
I20260709 13:05:01.866729 11470 raft_consensus.cc:359] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.867482 11470 raft_consensus.cc:740] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.868549 11470 consensus_queue.cc:260] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.871641 11470 ts_tablet_manager.cc:1434] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:01.872504 11470 tablet_bootstrap.cc:492] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:01.881345 11472 raft_consensus.cc:493] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.882201 11472 raft_consensus.cc:515] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.883543 11450 raft_consensus.cc:493] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.884238 11450 raft_consensus.cc:515] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.888195 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:01.889019 10723 raft_consensus.cc:2468] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:05:01.889372 11450 leader_election.cc:290] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:01.890465 10764 leader_election.cc:304] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:01.892027 11297 raft_consensus.cc:2804] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:01.892426 11297 raft_consensus.cc:493] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.892795 11297 raft_consensus.cc:3060] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:01.895726 11470 tablet_bootstrap.cc:492] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.896085 11450 raft_consensus.cc:493] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.896916 11470 tablet_bootstrap.cc:492] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:01.896647 11450 raft_consensus.cc:515] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.897722 11470 ts_tablet_manager.cc:1403] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.007s
W20260709 13:05:01.897795 10766 leader_election.cc:336] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:01.890923 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d1771d12424af18f74abf7650fab3a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:01.900746 11472 leader_election.cc:290] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:01.901268 11470 raft_consensus.cc:359] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:01.901496 11297 raft_consensus.cc:515] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.902483 11470 raft_consensus.cc:740] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.904937 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:01.905560 10723 raft_consensus.cc:3060] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:01.903249 11470 consensus_queue.cc:260] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:01.907289 11297 leader_election.cc:290] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:01.914680 11470 ts_tablet_manager.cc:1434] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.005s
I20260709 13:05:01.907276   431 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:05:01.916154   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.323s	user 0.298s	sys 0.002s
I20260709 13:05:01.914214 11450 leader_election.cc:290] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:01.913589 10723 raft_consensus.cc:2468] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
I20260709 13:05:01.901157 10800 raft_consensus.cc:2468] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:05:01.915850 11470 tablet_bootstrap.cc:492] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:01.915632 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46823d8cc8df4f4bafb078e496b9a7d2" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:05:01.918682 10766 leader_election.cc:336] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:01.920513 10764 leader_election.cc:304] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:01.920946 10693 leader_election.cc:304] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:01.921862 11450 raft_consensus.cc:2804] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:01.922264 11450 raft_consensus.cc:493] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.922657 11450 raft_consensus.cc:3060] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:01.925917 11445 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.926563 11432 raft_consensus.cc:2804] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:01.927115 11432 raft_consensus.cc:697] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:01.926573 11445 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:01.928188 11432 consensus_queue.cc:237] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.943714 11445 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:01.945891 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
W20260709 13:05:01.946751 10618 leader_election.cc:336] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:01.950480 10616 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:01.958879 11445 raft_consensus.cc:2749] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:01.967342 11470 tablet_bootstrap.cc:492] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:01.968705 11470 tablet_bootstrap.cc:492] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:01.969494 11470 ts_tablet_manager.cc:1403] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.054s	user 0.033s	sys 0.016s
I20260709 13:05:01.972244 11445 raft_consensus.cc:493] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.972887 11445 raft_consensus.cc:515] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:01.975215 11445 leader_election.cc:290] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:01.976337 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3d1fe775e74c4684020e6c5732ce13" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:01.977038 10723 raft_consensus.cc:2468] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:05:01.976059 10474 catalog_manager.cc:5697] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 1 to 2, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.972892 11470 raft_consensus.cc:359] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.977957 11470 raft_consensus.cc:740] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.978773 10616 leader_election.cc:304] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:01.980027 11445 raft_consensus.cc:2804] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:01.980619 11445 raft_consensus.cc:493] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.981169 11445 raft_consensus.cc:3060] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:01.982023 11386 raft_consensus.cc:493] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.982623 11386 raft_consensus.cc:515] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.991832 11407 raft_consensus.cc:493] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:01.992621 11407 raft_consensus.cc:515] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.993345 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:01.993448 11470 consensus_queue.cc:260] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:01.996866 11386 leader_election.cc:290] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:01.998335 11407 leader_election.cc:290] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.000173 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:02.002465 11470 ts_tablet_manager.cc:1434] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:05:02.004187 11407 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.005229 11470 tablet_bootstrap.cc:492] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:02.004752 11407 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:02.000204 11473 raft_consensus.cc:493] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.006983 11473 raft_consensus.cc:515] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
W20260709 13:05:02.007624 10618 leader_election.cc:336] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
W20260709 13:05:02.010130 10692 leader_election.cc:336] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.010772 10692 leader_election.cc:304] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.011948 11386 raft_consensus.cc:2749] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:02.009243 11473 leader_election.cc:290] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.008649 11407 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:02.021106 11450 raft_consensus.cc:515] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.033708 11472 raft_consensus.cc:493] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.034518 11472 raft_consensus.cc:515] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.040086 11472 leader_election.cc:290] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.033581 11445 raft_consensus.cc:515] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:02.040024 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ecb24ddae546da9915163c05742a7e" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:02.043722 10723 raft_consensus.cc:2468] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:05:02.045179 10764 leader_election.cc:304] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:02.056138 11432 raft_consensus.cc:2804] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:02.059265 11472 raft_consensus.cc:493] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
W20260709 13:05:02.066448 10766 leader_election.cc:336] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.066464 11472 raft_consensus.cc:515] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.063391 11432 raft_consensus.cc:493] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.069224 11432 raft_consensus.cc:3060] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.071029 11472 leader_election.cc:290] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.071662 11472 raft_consensus.cc:493] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.072185 11472 raft_consensus.cc:515] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.078418 11444 raft_consensus.cc:493] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.079231 11444 raft_consensus.cc:515] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:02.082700 11444 leader_election.cc:290] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:02.085421 10646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:02.086359 10646 raft_consensus.cc:2468] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:05:02.097780 11445 leader_election.cc:290] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:02.098978 10646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:02.099763 10646 raft_consensus.cc:2468] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 3.
I20260709 13:05:02.100427 11450 leader_election.cc:290] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.100255 10722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3d1fe775e74c4684020e6c5732ce13" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:02.101936 10722 raft_consensus.cc:3060] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.102761 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:02.103459 10800 raft_consensus.cc:2468] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:05:02.105640 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d1771d12424af18f74abf7650fab3a" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:02.106261 10800 raft_consensus.cc:3060] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:02.107842 10692 leader_election.cc:336] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
W20260709 13:05:02.111872 10618 leader_election.cc:336] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.112186 11297 raft_consensus.cc:493] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.112823 11297 raft_consensus.cc:515] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.118768 11297 leader_election.cc:290] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
W20260709 13:05:02.121268 10766 leader_election.cc:336] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.123724 10767 leader_election.cc:304] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:05:02.125061 11297 raft_consensus.cc:2804] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:02.126336 11470 tablet_bootstrap.cc:492] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:02.127687 11470 tablet_bootstrap.cc:492] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:02.128528 11470 ts_tablet_manager.cc:1403] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.125s	user 0.027s	sys 0.003s
I20260709 13:05:02.125556 11297 raft_consensus.cc:493] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.129526 11297 raft_consensus.cc:3060] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.131736 11432 raft_consensus.cc:515] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
W20260709 13:05:02.135733 10766 leader_election.cc:336] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.137930 10693 leader_election.cc:304] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:02.147471 11470 raft_consensus.cc:359] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:02.153292 11386 raft_consensus.cc:2804] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:02.153983 11386 raft_consensus.cc:493] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.154112 11472 leader_election.cc:290] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.154515 11386 raft_consensus.cc:3060] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.156592 11432 leader_election.cc:290] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.163744 11470 raft_consensus.cc:740] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:02.169778 11470 consensus_queue.cc:260] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
W20260709 13:05:02.170264 10692 leader_election.cc:336] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.178043 11297 raft_consensus.cc:515] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:05:02.179508 10766 leader_election.cc:336] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
W20260709 13:05:02.180236 10766 leader_election.cc:336] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
W20260709 13:05:02.177356 10692 leader_election.cc:336] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.178498 10800 raft_consensus.cc:2468] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 2.
I20260709 13:05:02.181823 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:02.185297 10722 raft_consensus.cc:2468] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 2.
I20260709 13:05:02.186003 10653 raft_consensus.cc:3060] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.176149 11472 raft_consensus.cc:493] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.188083 11472 raft_consensus.cc:515] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.183045 11407 raft_consensus.cc:493] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.189847 11407 raft_consensus.cc:515] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
W20260709 13:05:02.190001 10692 leader_election.cc:336] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.191768 11407 leader_election.cc:290] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:02.182557 11297 leader_election.cc:290] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:02.195458 11386 raft_consensus.cc:515] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.192629 11472 leader_election.cc:290] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.192608 10616 leader_election.cc:304] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
W20260709 13:05:02.192193 10766 leader_election.cc:336] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.198885 11491 raft_consensus.cc:2804] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:02.199402 11491 raft_consensus.cc:697] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:05:02.200322 11491 consensus_queue.cc:237] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
W20260709 13:05:02.203004 10766 leader_election.cc:336] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.203826 10653 raft_consensus.cc:2468] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
I20260709 13:05:02.193347 10692 leader_election.cc:304] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.205506 11386 leader_election.cc:290] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.206082 11407 raft_consensus.cc:2749] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:02.207118 10692 leader_election.cc:336] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.207686 10692 leader_election.cc:304] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.208585 11407 raft_consensus.cc:2749] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:02.208979 10766 leader_election.cc:336] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.184239 10693 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:02.213986 11407 raft_consensus.cc:2804] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:02.214425 11407 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.214773 11407 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:02.222410 10692 leader_election.cc:336] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.226517 11491 raft_consensus.cc:493] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.228088 10646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95de1505ade4f5389a46d7aca7d7136" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:02.229076 10646 raft_consensus.cc:2468] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 1.
I20260709 13:05:02.225566 10474 catalog_manager.cc:5697] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 1 to 2, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 2 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:02.227571 11491 raft_consensus.cc:515] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:02.233644 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:02.233989 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:02.234578 10724 raft_consensus.cc:2468] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:05:02.235129 10723 raft_consensus.cc:2393] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a12f1c20c674e7f8242df41a9acf5dc in current term 2: Already voted for candidate 7ecd836bb7a144dc9aa07a046239d6ce in this term.
I20260709 13:05:02.232963 10722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:02.235888 11491 leader_election.cc:290] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.236449 10722 raft_consensus.cc:2468] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:05:02.241503 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46823d8cc8df4f4bafb078e496b9a7d2" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:02.242408 10726 raft_consensus.cc:2468] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 1.
I20260709 13:05:02.245155 11470 ts_tablet_manager.cc:1434] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.116s	user 0.009s	sys 0.000s
I20260709 13:05:02.246620 11470 tablet_bootstrap.cc:492] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:02.246205 10767 leader_election.cc:304] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.248124 11472 raft_consensus.cc:2804] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:02.248889 11472 raft_consensus.cc:697] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:02.245409 10764 leader_election.cc:304] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.252113 11297 raft_consensus.cc:2804] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:02.252475 11297 raft_consensus.cc:493] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.251515 11472 consensus_queue.cc:237] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:02.252797 11297 raft_consensus.cc:3060] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.253909 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:02.254091 10722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ecb24ddae546da9915163c05742a7e" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:02.258909 10722 raft_consensus.cc:3060] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.262032 10693 leader_election.cc:304] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.263749 11297 raft_consensus.cc:515] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.265348 11386 raft_consensus.cc:2804] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:02.262390 10764 leader_election.cc:304] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.266012 11386 raft_consensus.cc:697] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:02.266408 11444 raft_consensus.cc:3060] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.267138 10764 leader_election.cc:304] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.267274 11386 consensus_queue.cc:237] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.268162 11432 raft_consensus.cc:2804] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:02.268564 11432 raft_consensus.cc:493] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.268944 11432 raft_consensus.cc:3060] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:02.257776 10618 leader_election.cc:336] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.277436 11444 raft_consensus.cc:2749] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:02.254603 10800 raft_consensus.cc:2468] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 1.
I20260709 13:05:02.253260 11407 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:02.286393 10693 leader_election.cc:304] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:02.294868 11407 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:02.296123 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:02.296788 10653 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:02.298573 11297 leader_election.cc:290] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.291865 11386 raft_consensus.cc:2804] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:02.303411 10472 catalog_manager.cc:5697] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 1 to 2, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 2 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:02.309725 11386 raft_consensus.cc:493] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
W20260709 13:05:02.329173 10692 leader_election.cc:336] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
W20260709 13:05:02.331035 10692 leader_election.cc:336] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.325889 11386 raft_consensus.cc:3060] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.330367 11432 raft_consensus.cc:515] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.343856 11432 leader_election.cc:290] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.349502 10722 raft_consensus.cc:2468] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
W20260709 13:05:02.352311 10692 leader_election.cc:336] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.360086 10653 raft_consensus.cc:2468] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 4.
I20260709 13:05:02.366950 10472 catalog_manager.cc:5697] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 1 to 2, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:02.375728 10693 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.377779 11407 raft_consensus.cc:2804] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:02.379469 11407 raft_consensus.cc:697] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
W20260709 13:05:02.380638 10764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 98016.8us
I20260709 13:05:02.383302 10764 leader_election.cc:304] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.384634 11297 raft_consensus.cc:2804] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:02.385077 11297 raft_consensus.cc:493] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.385852 11297 raft_consensus.cc:3060] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.386914 11407 consensus_queue.cc:237] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
W20260709 13:05:02.399351 10766 leader_election.cc:336] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
W20260709 13:05:02.400352 10766 leader_election.cc:336] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.401659 10764 leader_election.cc:304] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.403556 10619 leader_election.cc:304] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.404407 11297 raft_consensus.cc:515] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.406754 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:02.416507 10800 raft_consensus.cc:2468] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 2.
I20260709 13:05:02.420976 11297 leader_election.cc:290] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.425889 11490 raft_consensus.cc:2804] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:02.426540 11490 raft_consensus.cc:493] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:02.426947 11490 raft_consensus.cc:3060] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.430253 10722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:02.431329 10722 raft_consensus.cc:3060] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:02.431962 10766 leader_election.cc:336] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.411480 11432 raft_consensus.cc:2804] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:02.433894 11432 raft_consensus.cc:697] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:02.434988 11432 consensus_queue.cc:237] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.439376 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:02.441612 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46823d8cc8df4f4bafb078e496b9a7d2" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:02.393091 11386 raft_consensus.cc:515] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:02.451294 10723 raft_consensus.cc:3060] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.453220 11386 leader_election.cc:290] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 election: Requested vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:02.460198 10724 raft_consensus.cc:3060] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.462291 10727 raft_consensus.cc:1275] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:02.473730 11444 consensus_queue.cc:1048] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:02.475654 11470 tablet_bootstrap.cc:492] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:02.477188 11470 tablet_bootstrap.cc:492] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:02.478271 11470 ts_tablet_manager.cc:1403] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.232s	user 0.026s	sys 0.005s
W20260709 13:05:02.469038 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 89932.4us
I20260709 13:05:02.485935 10723 raft_consensus.cc:2468] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
I20260709 13:05:02.499877 10764 leader_election.cc:304] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.500891 11444 raft_consensus.cc:2804] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:02.501969 11444 raft_consensus.cc:697] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
W20260709 13:05:02.502300 10692 leader_election.cc:336] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.503118 11444 consensus_queue.cc:237] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.503708 10644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95de1505ade4f5389a46d7aca7d7136" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:02.505157 10644 raft_consensus.cc:3060] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.508966 10693 leader_election.cc:304] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.496924 11470 raft_consensus.cc:359] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:02.510504 11470 raft_consensus.cc:740] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:02.482790 10722 raft_consensus.cc:2468] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
I20260709 13:05:02.512768 10764 leader_election.cc:304] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.513780 11507 raft_consensus.cc:2804] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:02.514581 11507 raft_consensus.cc:697] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:02.515429 11507 consensus_queue.cc:237] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.473165 11490 raft_consensus.cc:515] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:02.524190 10798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:02.524936 10798 raft_consensus.cc:3060] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:02.532886 10766 consensus_peers.cc:597] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:02.529086 10472 catalog_manager.cc:5697] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 1 to 2, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:02.535413 11510 raft_consensus.cc:2804] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:02.538663 11510 raft_consensus.cc:697] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:02.539635 11510 consensus_queue.cc:237] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.541301 11490 leader_election.cc:290] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:02.535506 10798 raft_consensus.cc:2468] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 2.
I20260709 13:05:02.543529 10619 leader_election.cc:304] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:05:02.513756 11470 consensus_queue.cc:260] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:02.547210 11490 raft_consensus.cc:2804] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:02.549219 11490 raft_consensus.cc:697] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:05:02.549921 11470 ts_tablet_manager.cc:1434] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.003s
I20260709 13:05:02.550261 11490 consensus_queue.cc:237] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:02.573376 10474 catalog_manager.cc:5697] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 1 to 2, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:02.580183 10474 catalog_manager.cc:5697] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 1 to 2, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:02.577389 11470 tablet_bootstrap.cc:492] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:02.584474 10724 raft_consensus.cc:2468] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
I20260709 13:05:02.598050 10764 leader_election.cc:304] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.599773 11507 raft_consensus.cc:2804] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:02.600512 11507 raft_consensus.cc:697] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:02.601719 11507 consensus_queue.cc:237] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.619098 11470 tablet_bootstrap.cc:492] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:02.620410 11470 tablet_bootstrap.cc:492] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:02.621261 11470 ts_tablet_manager.cc:1403] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.071s	user 0.018s	sys 0.003s
W20260709 13:05:02.629308 10618 leader_election.cc:336] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:02.629729 11470 raft_consensus.cc:359] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:02.631211 11470 raft_consensus.cc:740] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:02.632210 11470 consensus_queue.cc:260] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:02.634649 11470 ts_tablet_manager.cc:1434] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:05:02.635553 11470 tablet_bootstrap.cc:492] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:02.657425 10644 raft_consensus.cc:2468] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 2.
I20260709 13:05:02.658939 10693 leader_election.cc:304] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:02.660437 11407 raft_consensus.cc:2804] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:02.660946 11407 raft_consensus.cc:697] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:02.661829 11407 consensus_queue.cc:237] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:02.663407 11470 tablet_bootstrap.cc:492] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:02.664671 11470 tablet_bootstrap.cc:492] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:02.665593 11470 ts_tablet_manager.cc:1403] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.004s
I20260709 13:05:02.636546 10472 catalog_manager.cc:5697] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 3 to 4, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 4 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:02.668853 11470 raft_consensus.cc:359] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.672451 11470 raft_consensus.cc:740] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:02.673507 11470 consensus_queue.cc:260] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.680405 11470 ts_tablet_manager.cc:1434] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.004s
I20260709 13:05:02.681432 11470 tablet_bootstrap.cc:492] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:02.697461 10476 catalog_manager.cc:5697] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 1 to 2, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:02.712579 10473 catalog_manager.cc:5697] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 1 to 2, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 2 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:02.715535 10473 catalog_manager.cc:5697] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce reported cstate change: term changed from 1 to 2, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195). New cstate: current_term: 2 leader_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:02.684540 10474 catalog_manager.cc:5697] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 1 to 2, leader changed from bef3584ebb2c4a2ea619936c71900230 (127.0.107.193) to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 2 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:02.728318 10692 consensus_peers.cc:597] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:02.741225 10644 raft_consensus.cc:1275] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:02.744769 11507 consensus_queue.cc:1048] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:02.748750 10801 raft_consensus.cc:1275] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Refusing update from remote peer 7ecd836bb7a144dc9aa07a046239d6ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:02.752833 11407 consensus_queue.cc:1048] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:02.786569 10766 consensus_peers.cc:597] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.793398 11470 tablet_bootstrap.cc:492] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:02.799757 10618 consensus_peers.cc:597] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:02.794718 11470 tablet_bootstrap.cc:492] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:02.805706 11470 ts_tablet_manager.cc:1403] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.125s	user 0.020s	sys 0.019s
I20260709 13:05:02.809270 11470 raft_consensus.cc:359] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.810154 11470 raft_consensus.cc:740] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:02.818841 10723 raft_consensus.cc:1275] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 73db933c4a6848d1b2aeb56813f7bb2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:02.820640 11490 consensus_queue.cc:1048] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:02.826167 11470 consensus_queue.cc:260] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:02.832129 11470 ts_tablet_manager.cc:1434] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.003s
I20260709 13:05:02.833214 11470 tablet_bootstrap.cc:492] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
W20260709 13:05:02.836925 10766 consensus_peers.cc:597] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:02.885241 11470 tablet_bootstrap.cc:492] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:02.886343 11470 tablet_bootstrap.cc:492] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:02.887050 11470 ts_tablet_manager.cc:1403] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.012s
I20260709 13:05:02.888160 10653 raft_consensus.cc:1275] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 4 FOLLOWER]: Refusing update from remote peer 7ecd836bb7a144dc9aa07a046239d6ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:02.889829 11470 raft_consensus.cc:359] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:02.890656 11470 raft_consensus.cc:740] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:02.891858 11470 consensus_queue.cc:260] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:05:02.894367 10766 consensus_peers.cc:597] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.895081 11475 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.895663 11475 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:02.894570 11470 ts_tablet_manager.cc:1434] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:05:02.901571 11470 tablet_bootstrap.cc:492] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:02.899775 11510 consensus_queue.cc:1048] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:05:02.903731 11475 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
W20260709 13:05:02.932683 10766 consensus_peers.cc:597] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.039779 10766 consensus_peers.cc:597] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:03.041852 10766 consensus_peers.cc:597] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:03.060967 10618 consensus_peers.cc:597] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.063297 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 50881.7us
I20260709 13:05:03.061185 10722 raft_consensus.cc:1275] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:03.073302 11507 consensus_queue.cc:1048] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:03.079886 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:03.080480 10723 raft_consensus.cc:1275] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:03.087715 10766 consensus_peers.cc:597] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:03.090463 10692 consensus_peers.cc:597] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:03.082808 11444 consensus_queue.cc:1048] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
W20260709 13:05:03.103866 10618 consensus_peers.cc:597] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:03.108366 10692 consensus_peers.cc:597] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:03.109248 10723 raft_consensus.cc:1275] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:03.118211 11432 consensus_queue.cc:1048] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:03.124887 10800 raft_consensus.cc:1275] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Refusing update from remote peer 73db933c4a6848d1b2aeb56813f7bb2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:03.132771 10800 raft_consensus.cc:1275] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Refusing update from remote peer 7ecd836bb7a144dc9aa07a046239d6ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:03.148283 10692 consensus_peers.cc:597] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.148865 10692 consensus_peers.cc:597] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.149292 10692 consensus_peers.cc:597] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.155709 10692 consensus_peers.cc:597] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:03.146323 11512 consensus_queue.cc:1048] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:05:03.170985 10726 raft_consensus.cc:1275] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:03.147904 11493 consensus_queue.cc:1048] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:05:03.192251 11470 tablet_bootstrap.cc:492] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:03.193665 11475 raft_consensus.cc:493] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.198472 11470 tablet_bootstrap.cc:492] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:03.199431 11470 ts_tablet_manager.cc:1403] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.299s	user 0.030s	sys 0.006s
I20260709 13:05:03.204632 11444 consensus_queue.cc:1048] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:03.209689 10618 consensus_peers.cc:597] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.219899 10766 consensus_peers.cc:597] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:03.229562 11470 raft_consensus.cc:359] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:03.230619 11470 raft_consensus.cc:740] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:03.232103 11470 consensus_queue.cc:260] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:05:03.242244 10766 consensus_peers.cc:597] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:03.197213 11475 raft_consensus.cc:515] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:03.260000 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:03.295315 11470 ts_tablet_manager.cc:1434] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.095s	user 0.017s	sys 0.004s
I20260709 13:05:03.303361 11470 tablet_bootstrap.cc:492] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
W20260709 13:05:03.315584 10764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 120991us
W20260709 13:05:03.324136 10692 consensus_peers.cc:597] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:03.390825 11475 leader_election.cc:290] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:03.391840 11475 raft_consensus.cc:493] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.342062 11578 raft_consensus.cc:493] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.412134 10653 raft_consensus.cc:1275] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Refusing update from remote peer 7ecd836bb7a144dc9aa07a046239d6ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:03.392411 11475 raft_consensus.cc:515] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:03.420148 11512 consensus_queue.cc:1048] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:03.425661 11578 raft_consensus.cc:515] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:03.428280 11578 leader_election.cc:290] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
W20260709 13:05:03.438611 10692 consensus_peers.cc:597] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.473374 10692 consensus_peers.cc:597] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:03.484901 11475 leader_election.cc:290] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:03.491153 11581 raft_consensus.cc:493] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.491914 11581 raft_consensus.cc:515] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
W20260709 13:05:03.494861 10618 consensus_peers.cc:597] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:03.415568 11464 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
W20260709 13:05:03.495529 10618 consensus_peers.cc:597] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:03.475481 11578 raft_consensus.cc:493] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:03.504734 10618 consensus_peers.cc:597] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.529448 10618 consensus_peers.cc:597] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.544056 10766 consensus_peers.cc:597] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:03.562646 11581 leader_election.cc:290] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:03.548341 11578 raft_consensus.cc:515] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:03.571348 10653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca28e86e81447cb899fbb41f02a7f03" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:03.534229 11475 raft_consensus.cc:2749] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:05:03.594983 10692 consensus_peers.cc:597] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.596809 10692 consensus_peers.cc:597] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.597433 10692 consensus_peers.cc:597] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.598547 10692 consensus_peers.cc:597] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.599285 10692 consensus_peers.cc:597] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.600886 10692 consensus_peers.cc:597] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.611864 10766 consensus_peers.cc:597] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:03.585544 11464 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 220354us
I20260709 13:05:03.597299 11586 raft_consensus.cc:493] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:03.612572 10766 consensus_peers.cc:597] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:03.639397 11578 leader_election.cc:290] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:03.624217 11586 raft_consensus.cc:515] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
W20260709 13:05:03.694126 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 77672.3us
I20260709 13:05:03.701740 11586 leader_election.cc:290] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:03.703377 10644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:03.705518 11578 raft_consensus.cc:493] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.706616 11578 raft_consensus.cc:515] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:03.708914 11475 raft_consensus.cc:493] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.709748 11475 raft_consensus.cc:515] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:03.712394 11475 leader_election.cc:290] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:03.713030 11475 raft_consensus.cc:493] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.721560 10644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:03.729321 11475 raft_consensus.cc:515] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:03.737046 10644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:03.739862 11475 leader_election.cc:290] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
W20260709 13:05:03.740037 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 121454us
I20260709 13:05:03.780252 11475 raft_consensus.cc:493] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.780905 11475 raft_consensus.cc:515] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
W20260709 13:05:03.778235 10692 consensus_peers.cc:597] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:03.789788 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da58d0f3541543dc8678d61df75295f9" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
W20260709 13:05:03.785553 10766 consensus_peers.cc:597] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:03.761022 11578 leader_election.cc:290] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:03.790426 11475 leader_election.cc:290] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:03.823256 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca28e86e81447cb899fbb41f02a7f03" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:03.871212 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46823d8cc8df4f4bafb078e496b9a7d2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:03.839732 10721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f96bbe7d674a5485410f0aa8ec40e6" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:03.844175 11475 raft_consensus.cc:493] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.881517 10727 raft_consensus.cc:2393] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bef3584ebb2c4a2ea619936c71900230 in current term 2: Already voted for candidate 5a12f1c20c674e7f8242df41a9acf5dc in this term.
I20260709 13:05:03.853886 10722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:03.887337 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da58d0f3541543dc8678d61df75295f9" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
W20260709 13:05:03.898777 10766 consensus_peers.cc:597] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:03.927201 11578 raft_consensus.cc:493] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.936650 10719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ecb24ddae546da9915163c05742a7e" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:03.937392 11578 raft_consensus.cc:515] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:03.959985 11470 tablet_bootstrap.cc:492] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:03.888916 11475 raft_consensus.cc:515] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:03.992789 11470 tablet_bootstrap.cc:492] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:03.998518 11470 ts_tablet_manager.cc:1403] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.702s	user 0.027s	sys 0.009s
W20260709 13:05:04.012580 10764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 45574.4us
I20260709 13:05:04.027572 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ecb24ddae546da9915163c05742a7e" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.021567 11578 leader_election.cc:290] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:04.035606 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da7141649b494890b1b693fbfc917db0" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.039546 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33dcc19b71a2421eab2c5d20d5affc10" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.040652 11475 leader_election.cc:290] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:04.051471 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8082b4eacd414faa49143ba833d6ed" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.054839 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46823d8cc8df4f4bafb078e496b9a7d2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.055958 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b14050b5d4a4fb4aa74de480c7d9cdc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.058133 11470 raft_consensus.cc:359] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.059165 11470 raft_consensus.cc:740] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.059970 11470 consensus_queue.cc:260] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.062497 11470 ts_tablet_manager.cc:1434] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.000s
I20260709 13:05:04.063431 11470 tablet_bootstrap.cc:492] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
W20260709 13:05:04.086846 10692 consensus_peers.cc:597] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:04.087693 10692 consensus_peers.cc:597] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:04.090163 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f96bbe7d674a5485410f0aa8ec40e6" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.093977 11464 leader_election.cc:304] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
W20260709 13:05:04.099069 10618 consensus_peers.cc:597] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:04.104264 11475 raft_consensus.cc:2749] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.109849 11467 leader_election.cc:304] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:05:04.115159 11475 raft_consensus.cc:2749] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.140525 10647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:04.140960 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:04.133623 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.169893 10645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:04.169364 11464 leader_election.cc:304] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:05:04.171378 11475 raft_consensus.cc:2749] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.134994 11467 leader_election.cc:304] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:05:04.174018 11464 leader_election.cc:304] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:05:04.177779 11475 raft_consensus.cc:2749] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.186293 11470 tablet_bootstrap.cc:492] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.195791 11470 tablet_bootstrap.cc:492] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:04.196683 11470 ts_tablet_manager.cc:1403] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.133s	user 0.018s	sys 0.008s
I20260709 13:05:04.197201 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:05:04.201967 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 62832.3us
W20260709 13:05:04.176657 11467 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 52420us
I20260709 13:05:04.204640 11475 raft_consensus.cc:2749] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.216391 11470 raft_consensus.cc:359] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.230877 11470 raft_consensus.cc:740] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.235829 11470 consensus_queue.cc:260] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.238365 11464 leader_election.cc:304] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:05:04.241744 11470 ts_tablet_manager.cc:1434] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.045s	user 0.010s	sys 0.000s
W20260709 13:05:04.243791 10618 consensus_peers.cc:597] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:04.244843 11470 tablet_bootstrap.cc:492] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:04.241153 11475 raft_consensus.cc:2749] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.247627 11467 leader_election.cc:304] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:05:04.257207 11475 raft_consensus.cc:3060] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.257967 11467 leader_election.cc:304] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:05:04.259079 11578 raft_consensus.cc:2749] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.259619 11467 leader_election.cc:304] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:05:04.262117 11578 raft_consensus.cc:2749] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.269043 11467 leader_election.cc:304] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:05:04.283305 11475 raft_consensus.cc:2749] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.283506 11578 raft_consensus.cc:2749] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.285684 11467 leader_election.cc:304] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:05:04.294175 11475 raft_consensus.cc:2749] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:04.288923 10766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 77192.2us
I20260709 13:05:04.309342 11470 tablet_bootstrap.cc:492] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.311250 11470 tablet_bootstrap.cc:492] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:04.312104 11470 ts_tablet_manager.cc:1403] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.067s	user 0.025s	sys 0.004s
I20260709 13:05:04.315584 11470 raft_consensus.cc:359] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.316676 11470 raft_consensus.cc:740] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.317833 11470 consensus_queue.cc:260] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.320004 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:04.324079 11470 ts_tablet_manager.cc:1434] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:05:04.325526 11470 tablet_bootstrap.cc:492] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:04.327823 11464 leader_election.cc:304] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:05:04.328894 11475 raft_consensus.cc:2749] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:04.347690 10692 consensus_peers.cc:597] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:04.353061 11475 raft_consensus.cc:493] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.354116 11475 raft_consensus.cc:515] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.359989 10645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b930f7fe8fb4ec88f4aa71cd7094c9e" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:04.360071 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b930f7fe8fb4ec88f4aa71cd7094c9e" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.381531 11467 leader_election.cc:304] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:05:04.383399 11578 raft_consensus.cc:2749] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.357210 11475 leader_election.cc:290] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:04.444586 11475 raft_consensus.cc:493] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.445433 11475 raft_consensus.cc:515] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:04.452307 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:04.454455 11475 leader_election.cc:290] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:04.463711 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.464963 11470 tablet_bootstrap.cc:492] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.471984 11470 tablet_bootstrap.cc:492] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:04.472358 11467 leader_election.cc:304] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:05:04.473003 11470 ts_tablet_manager.cc:1403] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.148s	user 0.020s	sys 0.005s
I20260709 13:05:04.474028 11475 raft_consensus.cc:2749] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.476230 11470 raft_consensus.cc:359] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.484587 11470 raft_consensus.cc:740] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.485610 11470 consensus_queue.cc:260] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.491173 11470 ts_tablet_manager.cc:1434] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:05:04.492179 11470 tablet_bootstrap.cc:492] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:04.561404 11470 tablet_bootstrap.cc:492] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.561095 11475 raft_consensus.cc:493] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.562007 11475 raft_consensus.cc:515] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:04.566692 11475 leader_election.cc:290] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:04.567793 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.568552 11470 tablet_bootstrap.cc:492] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:04.569373 11470 ts_tablet_manager.cc:1403] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.077s	user 0.019s	sys 0.018s
I20260709 13:05:04.570190 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb134b566b6443f887986585ebc296c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:04.571789 11464 leader_election.cc:304] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce
I20260709 13:05:04.572654 11470 raft_consensus.cc:359] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:04.573117 11475 raft_consensus.cc:2749] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.573450 11470 raft_consensus.cc:740] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.574297 11470 consensus_queue.cc:260] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:04.580081 11470 ts_tablet_manager.cc:1434] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.003s
I20260709 13:05:04.581094 11470 tablet_bootstrap.cc:492] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:04.602916   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:44795
I20260709 13:05:04.603055 11644 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:44795 every 8 connection(s)
I20260709 13:05:04.613644 11470 tablet_bootstrap.cc:492] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.615078 11470 tablet_bootstrap.cc:492] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:04.616752 11470 ts_tablet_manager.cc:1403] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.009s
I20260709 13:05:04.619817 11470 raft_consensus.cc:359] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.620492 11470 raft_consensus.cc:740] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.627588 11470 consensus_queue.cc:260] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.636112 11470 ts_tablet_manager.cc:1434] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.004s
I20260709 13:05:04.637267 11470 tablet_bootstrap.cc:492] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:04.745580 11475 raft_consensus.cc:493] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.746307 11475 raft_consensus.cc:515] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.750509 10800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2988fb83d740c6a6b692c5fc8d8105" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:04.757869 11475 leader_election.cc:290] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:04.762554 10645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e2988fb83d740c6a6b692c5fc8d8105" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" is_pre_election: true
I20260709 13:05:04.772706 11467 leader_election.cc:304] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bef3584ebb2c4a2ea619936c71900230; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:05:04.774545 11475 raft_consensus.cc:2749] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:04.780650 11577 raft_consensus.cc:3060] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.788146 11645 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34253
I20260709 13:05:04.788866 11645 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:04.790283 11645 heartbeater.cc:507] Master 127.0.107.254:34253 requested a full tablet report, sending...
I20260709 13:05:04.816465 11583 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:04.822480 11470 tablet_bootstrap.cc:492] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.823801 11470 tablet_bootstrap.cc:492] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:04.832161 11470 ts_tablet_manager.cc:1403] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.195s	user 0.052s	sys 0.017s
I20260709 13:05:04.835983 11470 raft_consensus.cc:359] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:04.836930 11470 raft_consensus.cc:740] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.838044 11470 consensus_queue.cc:260] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:04.842088 11470 ts_tablet_manager.cc:1434] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:05:04.843400 11470 tablet_bootstrap.cc:492] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:04.862991 11470 tablet_bootstrap.cc:492] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.864547 11470 tablet_bootstrap.cc:492] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:04.865679 11470 ts_tablet_manager.cc:1403] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.007s
I20260709 13:05:04.868638 11470 raft_consensus.cc:359] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:04.869572 11470 raft_consensus.cc:740] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.870489 11470 consensus_queue.cc:260] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:04.872855 11470 ts_tablet_manager.cc:1434] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:05:04.873942 11470 tablet_bootstrap.cc:492] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:04.876490 10475 ts_manager.cc:194] Re-registered known tserver with Master: bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:04.899569 11470 tablet_bootstrap.cc:492] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.901436 11470 tablet_bootstrap.cc:492] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:04.902330 11470 ts_tablet_manager.cc:1403] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.012s
I20260709 13:05:04.906653 11470 raft_consensus.cc:359] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.907451 11470 raft_consensus.cc:740] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.907869 11583 raft_consensus.cc:3060] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.908324 11470 consensus_queue.cc:260] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.910914 11470 ts_tablet_manager.cc:1434] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:04.911811 11470 tablet_bootstrap.cc:492] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:04.931577 11583 raft_consensus.cc:3060] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.938293 11470 tablet_bootstrap.cc:492] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.937837 10475 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55608
I20260709 13:05:04.939703 11470 tablet_bootstrap.cc:492] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:04.940855 11470 ts_tablet_manager.cc:1403] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.029s	user 0.024s	sys 0.000s
I20260709 13:05:04.944199 11470 raft_consensus.cc:359] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.945043 11470 raft_consensus.cc:740] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.946209 11470 consensus_queue.cc:260] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:04.947243 11645 heartbeater.cc:499] Master 127.0.107.254:34253 was elected leader, sending a full tablet report...
I20260709 13:05:04.949720 11470 ts_tablet_manager.cc:1434] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.004s
I20260709 13:05:04.950726 11470 tablet_bootstrap.cc:492] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:04.962468 11570 raft_consensus.cc:3060] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.991496 11470 tablet_bootstrap.cc:492] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:04.992663 11470 tablet_bootstrap.cc:492] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:04.993527 11470 ts_tablet_manager.cc:1403] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.017s
I20260709 13:05:04.996378 11470 raft_consensus.cc:359] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:04.997078 11470 raft_consensus.cc:740] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.997993 11470 consensus_queue.cc:260] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:05.000365 11470 ts_tablet_manager.cc:1434] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:05.001368 11470 tablet_bootstrap.cc:492] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:05.037647 11470 tablet_bootstrap.cc:492] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:05.039304 11470 tablet_bootstrap.cc:492] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:05.040369 11470 ts_tablet_manager.cc:1403] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.005s
I20260709 13:05:05.045841 11470 raft_consensus.cc:359] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:05.046923 11470 raft_consensus.cc:740] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.049297 11470 consensus_queue.cc:260] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:05.051887 11470 ts_tablet_manager.cc:1434] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:05:05.053102 11470 tablet_bootstrap.cc:492] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:05.053298 11570 raft_consensus.cc:3060] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:05.103122 10692 consensus_peers.cc:597] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:05.105645 11470 tablet_bootstrap.cc:492] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:05.107366 11470 tablet_bootstrap.cc:492] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:05.108210 11470 ts_tablet_manager.cc:1403] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.056s	user 0.018s	sys 0.008s
I20260709 13:05:05.111332 11470 raft_consensus.cc:359] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:05.112080 11470 raft_consensus.cc:740] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.112886 11470 consensus_queue.cc:260] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:05.115226 11470 ts_tablet_manager.cc:1434] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:05:05.116591 11470 tablet_bootstrap.cc:492] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:05.148752 11470 tablet_bootstrap.cc:492] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:05.149986 11470 tablet_bootstrap.cc:492] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:05.150727 11470 ts_tablet_manager.cc:1403] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.009s
I20260709 13:05:05.163168 11470 raft_consensus.cc:359] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:05.164249 11470 raft_consensus.cc:740] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.171547 11470 consensus_queue.cc:260] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:05.174130 11470 ts_tablet_manager.cc:1434] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.010s
I20260709 13:05:05.175136 11470 tablet_bootstrap.cc:492] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:05.190376 11577 raft_consensus.cc:3060] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.208248 11583 raft_consensus.cc:3060] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.225039 11470 tablet_bootstrap.cc:492] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:05.234688 11470 tablet_bootstrap.cc:492] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:05.235530 11470 ts_tablet_manager.cc:1403] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.061s	user 0.027s	sys 0.003s
I20260709 13:05:05.238596 11470 raft_consensus.cc:359] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:05.239389 11470 raft_consensus.cc:740] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.240396 11470 consensus_queue.cc:260] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:05.243872 11470 ts_tablet_manager.cc:1434] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:05:05.245020 11470 tablet_bootstrap.cc:492] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
W20260709 13:05:05.312674 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 55618.3us
W20260709 13:05:05.323748 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 65232.8us
I20260709 13:05:05.382047 11470 tablet_bootstrap.cc:492] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:05.395452 10618 consensus_peers.cc:597] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:05.409036 11470 tablet_bootstrap.cc:492] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:05.444475 11470 ts_tablet_manager.cc:1403] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.200s	user 0.028s	sys 0.031s
I20260709 13:05:05.454711 11470 raft_consensus.cc:359] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:05.455763 11470 raft_consensus.cc:740] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
W20260709 13:05:05.456751 10764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 57914.8us
I20260709 13:05:05.467424 11470 consensus_queue.cc:260] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:05.476593 11470 ts_tablet_manager.cc:1434] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:05:05.489218 11470 tablet_bootstrap.cc:492] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
W20260709 13:05:05.566922 10766 consensus_peers.cc:597] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:05.590143 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 78598.4us
W20260709 13:05:05.649032 10764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 59150.6us
W20260709 13:05:05.809547 10766 consensus_peers.cc:597] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:05.820832 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 60053.5us
I20260709 13:05:05.833696 11470 tablet_bootstrap.cc:492] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:05.834936 11470 tablet_bootstrap.cc:492] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:05.836103 11470 ts_tablet_manager.cc:1403] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.359s	user 0.117s	sys 0.154s
I20260709 13:05:05.853168 11470 raft_consensus.cc:359] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:05.854187 11470 raft_consensus.cc:740] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.855239 11470 consensus_queue.cc:260] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:05.889935 11470 ts_tablet_manager.cc:1434] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.053s	user 0.009s	sys 0.004s
W20260709 13:05:05.903617 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 79410.9us
W20260709 13:05:05.905701 10692 consensus_peers.cc:597] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:05.908179 11470 tablet_bootstrap.cc:492] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:05.962548 11470 tablet_bootstrap.cc:492] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:05.964385 11470 tablet_bootstrap.cc:492] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:05.965507 11470 ts_tablet_manager.cc:1403] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.058s	user 0.024s	sys 0.001s
I20260709 13:05:05.968830 11470 raft_consensus.cc:359] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:05.969756 11470 raft_consensus.cc:740] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.970743 11470 consensus_queue.cc:260] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:05.973156 11470 ts_tablet_manager.cc:1434] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
W20260709 13:05:05.972965 10766 consensus_peers.cc:597] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:05.974908 11470 tablet_bootstrap.cc:492] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:06.000851 11470 tablet_bootstrap.cc:492] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.002580 11470 tablet_bootstrap.cc:492] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:06.003553 11470 ts_tablet_manager.cc:1403] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.029s	user 0.021s	sys 0.003s
I20260709 13:05:06.007612 11470 raft_consensus.cc:359] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:06.008705 11470 raft_consensus.cc:740] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:06.009445 11470 consensus_queue.cc:260] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:06.012040 11470 ts_tablet_manager.cc:1434] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:05:06.012964 11470 tablet_bootstrap.cc:492] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
W20260709 13:05:06.034744 10692 consensus_peers.cc:597] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:06.037732 11470 tablet_bootstrap.cc:492] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.039405 11470 tablet_bootstrap.cc:492] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:06.040690 11470 ts_tablet_manager.cc:1403] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.008s
I20260709 13:05:06.044023 11470 raft_consensus.cc:359] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:06.044842 11470 raft_consensus.cc:740] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:06.045572 11470 consensus_queue.cc:260] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:06.048156 11470 ts_tablet_manager.cc:1434] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:06.049723 11470 tablet_bootstrap.cc:492] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:06.067713 11470 tablet_bootstrap.cc:492] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.069219 11470 tablet_bootstrap.cc:492] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:06.070007 11470 ts_tablet_manager.cc:1403] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.003s
I20260709 13:05:06.073005 11470 raft_consensus.cc:359] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:06.073628 11470 raft_consensus.cc:740] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:06.074361 11470 consensus_queue.cc:260] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:06.077034 11470 ts_tablet_manager.cc:1434] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:05:06.078243 11470 tablet_bootstrap.cc:492] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:06.099558 11470 tablet_bootstrap.cc:492] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.100906 11470 tablet_bootstrap.cc:492] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:06.101747 11470 ts_tablet_manager.cc:1403] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.003s
I20260709 13:05:06.104763 11470 raft_consensus.cc:359] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:06.105656 11470 raft_consensus.cc:740] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:06.106396 11470 consensus_queue.cc:260] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:06.107991 11470 ts_tablet_manager.cc:1434] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:06.112424 11470 tablet_bootstrap.cc:492] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:06.134006 11470 tablet_bootstrap.cc:492] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.134832 11470 tablet_bootstrap.cc:492] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:06.135450 11470 ts_tablet_manager.cc:1403] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.001s
I20260709 13:05:06.137619 11470 raft_consensus.cc:359] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:06.138294 11470 raft_consensus.cc:740] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:06.139051 11470 consensus_queue.cc:260] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:06.140781 11470 ts_tablet_manager.cc:1434] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:06.142369 11470 tablet_bootstrap.cc:492] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:06.191103 11470 tablet_bootstrap.cc:492] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.192263 11470 tablet_bootstrap.cc:492] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:06.193048 11470 ts_tablet_manager.cc:1403] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.014s
I20260709 13:05:06.196310 11470 raft_consensus.cc:359] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:06.197280 11470 raft_consensus.cc:740] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:06.198333 11470 consensus_queue.cc:260] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:06.207218 11470 ts_tablet_manager.cc:1434] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 13:05:06.208266 11470 tablet_bootstrap.cc:492] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:06.236968 11470 tablet_bootstrap.cc:492] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.238634 11470 tablet_bootstrap.cc:492] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:06.240267 11470 ts_tablet_manager.cc:1403] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.003s
I20260709 13:05:06.248682 11470 raft_consensus.cc:359] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:06.249703 11470 raft_consensus.cc:740] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:06.250470 11470 consensus_queue.cc:260] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:06.253365 11470 ts_tablet_manager.cc:1434] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:05:06.254457 11470 tablet_bootstrap.cc:492] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:06.272300 11470 tablet_bootstrap.cc:492] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.273517 11470 tablet_bootstrap.cc:492] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:06.274319 11470 ts_tablet_manager.cc:1403] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.020s	user 0.019s	sys 0.000s
I20260709 13:05:06.276903 11470 raft_consensus.cc:359] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:06.277879 11470 raft_consensus.cc:740] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:06.279220 11470 consensus_queue.cc:260] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:06.282415 11470 ts_tablet_manager.cc:1434] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:06.283536 11470 tablet_bootstrap.cc:492] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:06.304214 11470 tablet_bootstrap.cc:492] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.305661 11470 tablet_bootstrap.cc:492] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:06.306533 11470 ts_tablet_manager.cc:1403] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.003s
I20260709 13:05:06.310719 11470 raft_consensus.cc:359] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:06.311662 11470 raft_consensus.cc:740] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:06.312558 11470 consensus_queue.cc:260] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:06.315598 11470 ts_tablet_manager.cc:1434] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:05:06.317134 11470 tablet_bootstrap.cc:492] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230: Bootstrap starting.
I20260709 13:05:06.344413 11470 tablet_bootstrap.cc:492] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.345721 11470 tablet_bootstrap.cc:492] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230: Bootstrap complete.
I20260709 13:05:06.346472 11470 ts_tablet_manager.cc:1403] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.014s
I20260709 13:05:06.349388 11470 raft_consensus.cc:359] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:06.350193 11470 raft_consensus.cc:740] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Initialized, Role: FOLLOWER
I20260709 13:05:06.350883 11470 consensus_queue.cc:260] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:06.353047 11470 ts_tablet_manager.cc:1434] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:05:06.526197 11563 raft_consensus.cc:3060] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.659307 11563 raft_consensus.cc:3060] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.738824 11583 raft_consensus.cc:3060] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:07.608451 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 79122.9us
W20260709 13:05:08.510867 10619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 71851.9us
I20260709 13:05:14.719359   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:14.953462 10793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f26ffe11b8143728804f33b884ed36a"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
mode: GRACEFUL
new_leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
 from {username='slave'} at 127.0.0.1:33360
I20260709 13:05:14.955389 10793 raft_consensus.cc:606] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Received request to transfer leadership to TS 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:05:14.973066 10724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a48d84eceb174792919f6256ee4fbb21"
dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
mode: GRACEFUL
new_leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
 from {username='slave'} at 127.0.0.1:38526
I20260709 13:05:14.973773 10724 raft_consensus.cc:606] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Received request to transfer leadership to TS 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:05:14.976763 10793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:33360
I20260709 13:05:14.977358 10793 raft_consensus.cc:606] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:14.980175 10793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5349592564d4f4d863236517a765d59"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:33360
I20260709 13:05:14.980870 10793 raft_consensus.cc:606] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:14.983687 10793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec71cde09fe64d089f95382f32269db5"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
mode: GRACEFUL
new_leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
 from {username='slave'} at 127.0.0.1:33360
I20260709 13:05:14.984475 10793 raft_consensus.cc:606] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Received request to transfer leadership to TS 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:05:14.987345 10793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fee952b378da49509e7ced417a0093cc"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:33360
I20260709 13:05:14.987890 10793 raft_consensus.cc:606] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:14.989054   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:14.990734   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:15.044984 11739 raft_consensus.cc:993] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:15.046211 11734 raft_consensus.cc:1081] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:15.048662 11557 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5349592564d4f4d863236517a765d59"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56924
I20260709 13:05:15.049522 11557 raft_consensus.cc:493] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:15.050320 11557 raft_consensus.cc:3060] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:15.059859 11557 raft_consensus.cc:515] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:15.062572 11557 leader_election.cc:290] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:15.063901 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:15.064036 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:15.065099 10724 raft_consensus.cc:3060] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:15.065173 10793 raft_consensus.cc:3055] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:15.065653 10793 raft_consensus.cc:740] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:15.066828 10793 consensus_queue.cc:260] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:15.068513 10793 raft_consensus.cc:3060] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:15.076570 10724 raft_consensus.cc:2468] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 3.
I20260709 13:05:15.078526 11464 leader_election.cc:304] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:15.079272 10793 raft_consensus.cc:2468] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 3.
I20260709 13:05:15.080852 11755 raft_consensus.cc:2804] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:15.082598 11755 raft_consensus.cc:697] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:15.083761 11755 consensus_queue.cc:237] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:15.100472 10473 catalog_manager.cc:5697] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 2 to 3, leader changed from 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:15.141414 11734 raft_consensus.cc:993] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:15.142102 11739 raft_consensus.cc:1081] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:15.144922 11557 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fee952b378da49509e7ced417a0093cc"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56924
I20260709 13:05:15.145620 11557 raft_consensus.cc:493] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:15.146028 11557 raft_consensus.cc:3060] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:15.161681 11557 raft_consensus.cc:515] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:15.165144 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:15.165936 10793 raft_consensus.cc:3055] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:15.166287 10793 raft_consensus.cc:740] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:15.166904 10646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee952b378da49509e7ced417a0093cc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:15.167609 10646 raft_consensus.cc:3060] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:15.167325 10793 consensus_queue.cc:260] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:15.166487 11557 leader_election.cc:290] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:15.171806 10793 raft_consensus.cc:3060] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:15.180553 11734 raft_consensus.cc:993] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:15.181437 11737 raft_consensus.cc:1081] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:15.182879 10646 raft_consensus.cc:2468] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 3.
I20260709 13:05:15.183494 11577 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56924
I20260709 13:05:15.184239 11577 raft_consensus.cc:493] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:15.184657 11467 leader_election.cc:304] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:15.184669 11577 raft_consensus.cc:3060] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.185725 11755 raft_consensus.cc:2804] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:15.186296 11755 raft_consensus.cc:697] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:15.187484 11755 consensus_queue.cc:237] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:15.184713 10793 raft_consensus.cc:2468] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 3.
I20260709 13:05:15.196178 11577 raft_consensus.cc:515] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:15.201566 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:15.202235 10723 raft_consensus.cc:3060] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.206676 10793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:15.207636 10793 raft_consensus.cc:3055] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:15.208074 10793 raft_consensus.cc:740] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:15.209220 10793 consensus_queue.cc:260] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:15.211380 10793 raft_consensus.cc:3060] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.228921 10723 raft_consensus.cc:2468] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:15.233289 11464 leader_election.cc:304] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:15.234521 11756 raft_consensus.cc:2804] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:15.236248 11577 leader_election.cc:290] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:15.240681 10793 raft_consensus.cc:2468] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:15.253865 11745 raft_consensus.cc:993] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc: : Instructing follower 73db933c4a6848d1b2aeb56813f7bb2c to start an election
I20260709 13:05:15.254460 11745 raft_consensus.cc:1081] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Signalling peer 73db933c4a6848d1b2aeb56813f7bb2c to start an election
I20260709 13:05:15.258024 10646 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f26ffe11b8143728804f33b884ed36a"
dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
 from {username='slave'} at 127.0.0.1:48306
I20260709 13:05:15.258662 10646 raft_consensus.cc:493] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:15.259073 10646 raft_consensus.cc:3060] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.261143 11756 raft_consensus.cc:697] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:15.262574 11756 consensus_queue.cc:237] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:15.262831 10473 catalog_manager.cc:5697] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 2 to 3, leader changed from 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:15.271271 10646 raft_consensus.cc:515] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:15.292390 11741 raft_consensus.cc:993] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce: : Instructing follower 5a12f1c20c674e7f8242df41a9acf5dc to start an election
I20260709 13:05:15.293146 11741 raft_consensus.cc:1081] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Signalling peer 5a12f1c20c674e7f8242df41a9acf5dc to start an election
I20260709 13:05:15.295230 10792 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a48d84eceb174792919f6256ee4fbb21"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
 from {username='slave'} at 127.0.0.1:34798
I20260709 13:05:15.296017 10792 raft_consensus.cc:493] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:15.296501 10792 raft_consensus.cc:3060] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.297111 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f26ffe11b8143728804f33b884ed36a" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:15.297813 10723 raft_consensus.cc:3060] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.300364 10646 leader_election.cc:290] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:15.305019 10723 raft_consensus.cc:2468] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 2.
I20260709 13:05:15.306811 10797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f26ffe11b8143728804f33b884ed36a" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:15.307610 10797 raft_consensus.cc:3055] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:15.307741 10616 leader_election.cc:304] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:15.308025 10797 raft_consensus.cc:740] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:15.309187 10797 consensus_queue.cc:260] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:15.310156 11744 raft_consensus.cc:2804] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:15.310768 11744 raft_consensus.cc:697] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:05:15.310846 10797 raft_consensus.cc:3060] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.311838 11744 consensus_queue.cc:237] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:15.322402 10475 catalog_manager.cc:5697] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 1 to 2, leader changed from 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:15.319339 10792 raft_consensus.cc:515] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:15.329129 10792 leader_election.cc:290] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:15.325166 10797 raft_consensus.cc:2468] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 2.
I20260709 13:05:15.330724 10718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48d84eceb174792919f6256ee4fbb21" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:15.332062 10718 raft_consensus.cc:3055] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:15.332435 10718 raft_consensus.cc:740] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:15.333542 10718 consensus_queue.cc:260] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:15.335242 10718 raft_consensus.cc:3060] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.343386 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a48d84eceb174792919f6256ee4fbb21" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:15.345213 10651 raft_consensus.cc:3060] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.356714 10718 raft_consensus.cc:2468] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
I20260709 13:05:15.359239 10764 leader_election.cc:304] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:15.360280 11739 raft_consensus.cc:2804] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:15.360788 11739 raft_consensus.cc:697] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:15.361938 11739 consensus_queue.cc:237] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:15.368733 10473 catalog_manager.cc:5697] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 1 to 2, leader changed from 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196) to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 2 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:15.371512 10651 raft_consensus.cc:2468] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
I20260709 13:05:15.387746 11734 raft_consensus.cc:993] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc: : Instructing follower 73db933c4a6848d1b2aeb56813f7bb2c to start an election
I20260709 13:05:15.392081 10475 catalog_manager.cc:5697] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 1 to 2, leader changed from 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195) to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:15.393731 11734 raft_consensus.cc:1081] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Signalling peer 73db933c4a6848d1b2aeb56813f7bb2c to start an election
I20260709 13:05:15.397830 10646 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec71cde09fe64d089f95382f32269db5"
dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
 from {username='slave'} at 127.0.0.1:48306
I20260709 13:05:15.398579 10646 raft_consensus.cc:493] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:15.399070 10646 raft_consensus.cc:3060] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.407761 10646 raft_consensus.cc:515] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:15.410745 10646 leader_election.cc:290] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:15.411088 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec71cde09fe64d089f95382f32269db5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:15.411808 10796 raft_consensus.cc:3055] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:15.412170 10796 raft_consensus.cc:740] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:15.413067 10796 consensus_queue.cc:260] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:15.413408 10718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec71cde09fe64d089f95382f32269db5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:15.414168 10718 raft_consensus.cc:3060] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.414681 10796 raft_consensus.cc:3060] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:15.423357 10796 raft_consensus.cc:2468] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 2.
I20260709 13:05:15.425590 10619 leader_election.cc:304] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:05:15.426661 11744 raft_consensus.cc:2804] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:15.427187 11744 raft_consensus.cc:697] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:05:15.428120 11744 consensus_queue.cc:237] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:15.430128 10718 raft_consensus.cc:2468] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 2.
I20260709 13:05:15.449606 10475 catalog_manager.cc:5697] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 1 to 2, leader changed from 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196) to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 2 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:15.638700 10796 raft_consensus.cc:1275] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:15.640098 11756 consensus_queue.cc:1048] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:15.652964 10651 raft_consensus.cc:1275] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:15.658540 11756 consensus_queue.cc:1048] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:05:15.665405 10507 debug-util.cc:398] Leaking SignalData structure 0x7b080010ab40 after lost signal to thread 434
W20260709 13:05:15.667150 10507 debug-util.cc:398] Leaking SignalData structure 0x7b080023ae20 after lost signal to thread 10510
W20260709 13:05:15.668967 10507 debug-util.cc:398] Leaking SignalData structure 0x7b0800216ea0 after lost signal to thread 10677
W20260709 13:05:15.673237 10507 debug-util.cc:398] Leaking SignalData structure 0x7b08002ce4e0 after lost signal to thread 11644
I20260709 13:05:15.674108 10718 raft_consensus.cc:1275] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:15.676293 11763 consensus_queue.cc:1048] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:15.686290 10797 raft_consensus.cc:1275] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:15.693951 11763 consensus_queue.cc:1048] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:15.832881 10718 raft_consensus.cc:1275] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:15.835505 11763 consensus_queue.cc:1048] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:15.899394 10797 raft_consensus.cc:1275] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:15.902769 10646 raft_consensus.cc:1275] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:15.909868 11739 consensus_queue.cc:1048] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:15.914767 10727 raft_consensus.cc:1275] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 73db933c4a6848d1b2aeb56813f7bb2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:15.922225 10727 raft_consensus.cc:1275] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:15.916771 11763 consensus_queue.cc:1048] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:15.962615 10800 raft_consensus.cc:1275] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Refusing update from remote peer 73db933c4a6848d1b2aeb56813f7bb2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:15.997925   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:16.116438 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 107874us
W20260709 13:05:16.117395 10764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 60105.7us
I20260709 13:05:16.099946 11734 consensus_queue.cc:1048] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
W20260709 13:05:16.134085 10618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 114133us
W20260709 13:05:16.135746 11789 log.cc:927] Time spent T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:05:16.221392   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:16.246536 10619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 270404us
W20260709 13:05:16.257679 10616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 366818us
I20260709 13:05:16.279186 11785 consensus_queue.cc:1048] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:05:16.286156 10724 raft_consensus.cc:1275] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 73db933c4a6848d1b2aeb56813f7bb2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:16.356326 10800 raft_consensus.cc:1275] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Refusing update from remote peer 73db933c4a6848d1b2aeb56813f7bb2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:16.394825 11744 consensus_queue.cc:1048] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
W20260709 13:05:16.434641 10619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 125231us
I20260709 13:05:16.378021 11778 consensus_queue.cc:1048] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:16.505666 11727 consensus_queue.cc:1048] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 13:05:16.535241 10616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 143885us
W20260709 13:05:16.865846 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 110385us
W20260709 13:05:16.951233 10619 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 75895.9us
W20260709 13:05:16.968895 10693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43755, user_credentials={real_user=slave}} blocked reactor thread for 94841.4us
W20260709 13:05:17.033034 10618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 79705.1us
I20260709 13:05:17.048847 10647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33e1dc0bb8f54bd2ba02310d2d9d3bdc"
dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:59746
I20260709 13:05:17.049633 10647 raft_consensus.cc:606] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:17.055965 10647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be"
dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
mode: GRACEFUL
new_leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
 from {username='slave'} at 127.0.0.1:59746
I20260709 13:05:17.056772 10647 raft_consensus.cc:606] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Received request to transfer leadership to TS 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:05:17.061281 10796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:33360
I20260709 13:05:17.062003 10796 raft_consensus.cc:606] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:17.065347 10723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f95de1505ade4f5389a46d7aca7d7136"
dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:38526
I20260709 13:05:17.065977 10723 raft_consensus.cc:606] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:17.070842 10796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
mode: GRACEFUL
new_leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
 from {username='slave'} at 127.0.0.1:33360
I20260709 13:05:17.071590 10796 raft_consensus.cc:606] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Received request to transfer leadership to TS 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:05:17.075227   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:17.077069   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:17.093922 11799 raft_consensus.cc:993] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:17.094632 11814 raft_consensus.cc:1081] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:17.096457 11557 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f95de1505ade4f5389a46d7aca7d7136"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56908
I20260709 13:05:17.097033 11557 raft_consensus.cc:493] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:17.097450 11557 raft_consensus.cc:3060] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:17.107823 11801 raft_consensus.cc:993] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:17.108484 11801 raft_consensus.cc:1081] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:17.110771 11559 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56924
I20260709 13:05:17.111464 11559 raft_consensus.cc:493] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:17.111902 11559 raft_consensus.cc:3060] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.114990 11557 raft_consensus.cc:515] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:17.125340 11557 leader_election.cc:290] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:17.126713 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95de1505ade4f5389a46d7aca7d7136" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:17.127470 10723 raft_consensus.cc:3055] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:17.127820 10723 raft_consensus.cc:740] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:17.131852 10647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95de1505ade4f5389a46d7aca7d7136" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:17.131803 10723 consensus_queue.cc:260] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:17.142614 10723 raft_consensus.cc:3060] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:17.134878 10647 raft_consensus.cc:3060] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:17.151891 11559 raft_consensus.cc:515] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:17.155828 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:17.156490 10796 raft_consensus.cc:3055] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:17.156791 10796 raft_consensus.cc:740] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:17.157825 10796 consensus_queue.cc:260] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:17.158984 10796 raft_consensus.cc:3060] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.159021 10723 raft_consensus.cc:2468] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 3.
I20260709 13:05:17.160365 11464 leader_election.cc:304] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:17.161316 11755 raft_consensus.cc:2804] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:17.161840 11755 raft_consensus.cc:697] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:17.162779 11755 consensus_queue.cc:237] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:17.163928 10647 raft_consensus.cc:2468] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 3.
I20260709 13:05:17.178391 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bad0f5b06cbf47c5a6ae75b972ee23a6" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:17.179291 10651 raft_consensus.cc:3060] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.181525 11559 leader_election.cc:290] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:17.184173 10796 raft_consensus.cc:2468] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:17.185642 11467 leader_election.cc:304] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:17.186607 11756 raft_consensus.cc:2804] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:17.187099 11756 raft_consensus.cc:697] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:17.188028 11756 consensus_queue.cc:237] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:17.190498 10651 raft_consensus.cc:2468] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:17.190848 11803 raft_consensus.cc:993] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c: : Instructing follower 5a12f1c20c674e7f8242df41a9acf5dc to start an election
I20260709 13:05:17.191403 11803 raft_consensus.cc:1081] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Signalling peer 5a12f1c20c674e7f8242df41a9acf5dc to start an election
I20260709 13:05:17.193596 10796 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
 from {username='slave'} at 127.0.0.1:34784
I20260709 13:05:17.194165 10796 raft_consensus.cc:493] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:17.194834 10796 raft_consensus.cc:3060] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.197933 10475 catalog_manager.cc:5697] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 2 to 3, leader changed from 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.202073 10796 raft_consensus.cc:515] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:17.206885 10796 leader_election.cc:290] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:17.209215 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:17.209932 10725 raft_consensus.cc:3060] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.214030 10651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9887fc1ac0ac404e9f590987c2f0b0be" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:17.216168 10651 raft_consensus.cc:3055] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:17.216563 10651 raft_consensus.cc:740] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:05:17.219013 10651 consensus_queue.cc:260] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:17.225257 10651 raft_consensus.cc:3060] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.228685 10725 raft_consensus.cc:2468] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
I20260709 13:05:17.238225 10764 leader_election.cc:304] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:17.239774 11791 raft_consensus.cc:2804] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:17.240321 11791 raft_consensus.cc:697] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:17.241219 11791 consensus_queue.cc:237] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:17.244729 10651 raft_consensus.cc:2468] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
I20260709 13:05:17.286017 11798 raft_consensus.cc:993] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:17.286630 11798 raft_consensus.cc:1081] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:17.290354 10473 catalog_manager.cc:5697] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 1 to 2, leader changed from 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194) to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.294309 11559 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33e1dc0bb8f54bd2ba02310d2d9d3bdc"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56902
I20260709 13:05:17.295105 11559 raft_consensus.cc:493] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:17.295493 11559 raft_consensus.cc:3060] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.316318 10475 catalog_manager.cc:5697] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 1 to 2, leader changed from 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.328555 11559 raft_consensus.cc:515] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:17.332261 11559 leader_election.cc:290] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:17.332568 10647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e1dc0bb8f54bd2ba02310d2d9d3bdc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:17.333345 10647 raft_consensus.cc:3055] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:17.333703 10647 raft_consensus.cc:740] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:05:17.334651 10647 consensus_queue.cc:260] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:17.335964 10647 raft_consensus.cc:3060] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.338467 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33e1dc0bb8f54bd2ba02310d2d9d3bdc" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:17.339211 10796 raft_consensus.cc:3060] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.351161 10647 raft_consensus.cc:2468] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:17.363222 10796 raft_consensus.cc:2468] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:17.366874 11467 leader_election.cc:304] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:17.369230 11756 raft_consensus.cc:2804] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:17.370345 11756 raft_consensus.cc:697] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:17.371527 11756 consensus_queue.cc:237] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:17.409256 11791 raft_consensus.cc:993] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc: : Instructing follower 73db933c4a6848d1b2aeb56813f7bb2c to start an election
I20260709 13:05:17.409824 11791 raft_consensus.cc:1081] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Signalling peer 73db933c4a6848d1b2aeb56813f7bb2c to start an election
I20260709 13:05:17.414301 10651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5"
dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
 from {username='slave'} at 127.0.0.1:48306
I20260709 13:05:17.414911 10651 raft_consensus.cc:493] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:17.415287 10651 raft_consensus.cc:3060] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.447541 10473 catalog_manager.cc:5697] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 1 to 2, leader changed from 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.449606 10651 raft_consensus.cc:515] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:17.453699 10718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:17.454174 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb17118da73149cbb340865c5c6959c5" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:17.455000 10792 raft_consensus.cc:3055] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:17.455327 10792 raft_consensus.cc:740] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:17.454312 10718 raft_consensus.cc:3060] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.456168 10792 consensus_queue.cc:260] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:17.459211 10792 raft_consensus.cc:3060] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:17.467088 10651 leader_election.cc:290] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:17.468991 10792 raft_consensus.cc:2468] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 2.
I20260709 13:05:17.470229 10718 raft_consensus.cc:2468] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 2.
I20260709 13:05:17.470703 10619 leader_election.cc:304] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:05:17.471778 11805 raft_consensus.cc:2804] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:17.472582 11805 raft_consensus.cc:697] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:05:17.473467 11805 consensus_queue.cc:237] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:17.484951 10473 catalog_manager.cc:5697] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 1 to 2, leader changed from 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196) to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 2 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.590425 10718 raft_consensus.cc:1275] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:17.592308 11755 consensus_queue.cc:1048] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:17.609763 10651 raft_consensus.cc:1275] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:17.613626 11756 consensus_queue.cc:1048] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:17.629312 10725 raft_consensus.cc:1275] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:17.631551 11791 consensus_queue.cc:1048] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:17.715407 10651 raft_consensus.cc:1275] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:17.716918 11755 consensus_queue.cc:1048] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:17.718204 10796 raft_consensus.cc:1275] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:17.727139 11755 consensus_queue.cc:1048] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:17.762616 10800 raft_consensus.cc:1275] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:17.764359 11755 consensus_queue.cc:1048] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:17.802125 10646 raft_consensus.cc:1275] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:17.804275 11739 consensus_queue.cc:1048] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:17.869844 10647 raft_consensus.cc:1275] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:17.874491 11763 consensus_queue.cc:1048] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:17.904984 10796 raft_consensus.cc:1275] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Refusing update from remote peer 73db933c4a6848d1b2aeb56813f7bb2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:17.910342 11744 consensus_queue.cc:1048] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:17.933176 10718 raft_consensus.cc:1275] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 73db933c4a6848d1b2aeb56813f7bb2c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:17.943948 11805 consensus_queue.cc:1048] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:18.078132   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:18.119151   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:18.182437 10792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:33360
I20260709 13:05:18.183188 10792 raft_consensus.cc:606] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:18.186408 10725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69cb859af9704c2596574e50f238ce52"
dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:38526
I20260709 13:05:18.187048 10725 raft_consensus.cc:606] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:18.189779 10725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79"
dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
mode: GRACEFUL
new_leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
 from {username='slave'} at 127.0.0.1:38526
I20260709 13:05:18.190274 10725 raft_consensus.cc:606] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Received request to transfer leadership to TS 5a12f1c20c674e7f8242df41a9acf5dc
I20260709 13:05:18.193044 10646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3"
dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:59746
I20260709 13:05:18.193753 10646 raft_consensus.cc:606] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:18.199425 10725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2bb2563ed2a407c9c219a632057e54a"
dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:38526
I20260709 13:05:18.200012 10725 raft_consensus.cc:606] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:18.201220   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:18.203167   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:18.243076 11784 raft_consensus.cc:993] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:18.243830 11815 raft_consensus.cc:1081] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:18.245744 11559 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e2bb2563ed2a407c9c219a632057e54a"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56908
I20260709 13:05:18.246305 11559 raft_consensus.cc:493] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.246676 11559 raft_consensus.cc:3060] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.273387 11559 raft_consensus.cc:515] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:18.275889 11803 raft_consensus.cc:993] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:18.276571 11803 raft_consensus.cc:1081] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:18.277320 11559 leader_election.cc:290] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:18.279603 11559 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56902
I20260709 13:05:18.279690 10647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2bb2563ed2a407c9c219a632057e54a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:18.280534 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2bb2563ed2a407c9c219a632057e54a" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:18.282351 10725 raft_consensus.cc:3055] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:18.282914 10725 raft_consensus.cc:740] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:18.283879 10725 consensus_queue.cc:260] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:18.285390 10725 raft_consensus.cc:3060] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.290180 11559 raft_consensus.cc:493] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.290629 11559 raft_consensus.cc:3060] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:18.290822 10647 raft_consensus.cc:3060] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.298980 10725 raft_consensus.cc:2468] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:18.300449 11464 leader_election.cc:304] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:18.301915 11755 raft_consensus.cc:2804] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:18.303350 11755 raft_consensus.cc:697] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:18.310773 10647 raft_consensus.cc:2468] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:18.315886 11559 raft_consensus.cc:515] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:18.347990 11559 leader_election.cc:290] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:18.347990 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:18.369277 10792 raft_consensus.cc:3060] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:18.374344 10646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9323c8a136794ad692b7dbff074333f3" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:18.375136 10646 raft_consensus.cc:3055] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:18.375442 10646 raft_consensus.cc:740] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:05:18.376281 10646 consensus_queue.cc:260] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:18.304350 11755 consensus_queue.cc:237] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:18.377450 10646 raft_consensus.cc:3060] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:18.380909 10792 raft_consensus.cc:2468] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 3.
I20260709 13:05:18.381799 11739 raft_consensus.cc:993] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:18.382324 11467 leader_election.cc:304] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:18.382792 11734 raft_consensus.cc:1081] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:18.384135 11763 raft_consensus.cc:2804] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:18.385061 11763 raft_consensus.cc:697] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:18.386533 11559 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56924
I20260709 13:05:18.387156 11559 raft_consensus.cc:493] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.387508 11559 raft_consensus.cc:3060] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:18.387537 11763 consensus_queue.cc:237] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:18.396457 11559 raft_consensus.cc:515] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:18.403944 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:18.404664 10792 raft_consensus.cc:3055] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:18.404994 10792 raft_consensus.cc:740] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:18.405930 10792 consensus_queue.cc:260] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:18.406874 10792 raft_consensus.cc:3060] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:18.408568 10646 raft_consensus.cc:2468] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 3.
I20260709 13:05:18.414245 11559 leader_election.cc:290] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:18.425571 10792 raft_consensus.cc:2468] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 3.
I20260709 13:05:18.429107 11467 leader_election.cc:304] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:18.430258 11763 raft_consensus.cc:2804] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:18.430775 11763 raft_consensus.cc:697] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:18.431823 11763 consensus_queue.cc:237] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:18.549235 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:18.551338 10727 raft_consensus.cc:3060] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:18.550315 10472 catalog_manager.cc:5697] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 1 to 2, leader changed from 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.588317 11790 raft_consensus.cc:993] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:18.597431 11790 raft_consensus.cc:1081] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:18.711218 10727 raft_consensus.cc:2468] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 3.
I20260709 13:05:18.732707 11551 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69cb859af9704c2596574e50f238ce52"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56908
I20260709 13:05:18.733858 11551 raft_consensus.cc:493] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.734664 11551 raft_consensus.cc:3060] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.727190 11815 raft_consensus.cc:993] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce: : Instructing follower 5a12f1c20c674e7f8242df41a9acf5dc to start an election
I20260709 13:05:18.742559 11815 raft_consensus.cc:1081] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Signalling peer 5a12f1c20c674e7f8242df41a9acf5dc to start an election
I20260709 13:05:18.759224 10800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
 from {username='slave'} at 127.0.0.1:34798
I20260709 13:05:18.760188 10800 raft_consensus.cc:493] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.760689 10800 raft_consensus.cc:3060] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.800320 10800 raft_consensus.cc:515] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:18.806211 10800 leader_election.cc:290] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:18.826711 10646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:18.827227 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:18.827744 10646 raft_consensus.cc:3060] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.828051 10723 raft_consensus.cc:3055] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:18.828392 10723 raft_consensus.cc:740] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:18.831007 10723 consensus_queue.cc:260] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:18.843591 10723 raft_consensus.cc:3060] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.850561 10646 raft_consensus.cc:2468] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
I20260709 13:05:18.858098 10767 leader_election.cc:304] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 73db933c4a6848d1b2aeb56813f7bb2c; no voters: 
I20260709 13:05:18.873185 11858 raft_consensus.cc:2804] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:18.876518 11551 raft_consensus.cc:515] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:18.877297 11858 raft_consensus.cc:697] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Leader. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:18.885890 11858 consensus_queue.cc:237] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:18.892275 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69cb859af9704c2596574e50f238ce52" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:18.894423 10724 raft_consensus.cc:3055] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:18.894831 10724 raft_consensus.cc:740] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:18.903862 10724 consensus_queue.cc:260] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:18.905790 11551 leader_election.cc:290] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:18.913268 10651 raft_consensus.cc:1275] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:18.918053 10724 raft_consensus.cc:3060] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.921053 10723 raft_consensus.cc:2468] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a12f1c20c674e7f8242df41a9acf5dc in term 2.
I20260709 13:05:18.924786 10797 raft_consensus.cc:1275] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:18.922171 10472 catalog_manager.cc:5697] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 2 to 3, leader changed from 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.925702 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69cb859af9704c2596574e50f238ce52" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:18.928777 10792 raft_consensus.cc:3060] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.930994 11841 consensus_queue.cc:1048] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:18.938181 10472 catalog_manager.cc:5697] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 2 to 3, leader changed from 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:18.941087 11763 consensus_queue.cc:1048] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:05:18.954782 10724 raft_consensus.cc:2468] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:18.960202 11464 leader_election.cc:304] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:18.961556 11763 raft_consensus.cc:2804] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:18.962247 11763 raft_consensus.cc:697] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:18.963554 11763 consensus_queue.cc:237] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:18.969589 10651 raft_consensus.cc:1275] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:18.973263 11756 consensus_queue.cc:1048] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:18.982708 10476 catalog_manager.cc:5697] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc reported cstate change: term changed from 1 to 2, leader changed from 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195) to 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:18.991662 10723 raft_consensus.cc:1275] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:18.996973 11756 consensus_queue.cc:1048] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:19.019101 10723 raft_consensus.cc:1275] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:19.035769 11756 consensus_queue.cc:1048] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:05:19.059219 10792 raft_consensus.cc:2468] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:19.091372 10796 raft_consensus.cc:1275] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:19.109870 11756 consensus_queue.cc:1048] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:05:19.250438   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:19.263079 10472 catalog_manager.cc:5697] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 1 to 2, leader changed from 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:19.312292 11467 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38707, user_credentials={real_user=slave}} blocked reactor thread for 95878.7us
I20260709 13:05:19.340317 10725 raft_consensus.cc:1275] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.393328 11857 consensus_queue.cc:1048] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:05:19.425637 10725 raft_consensus.cc:1275] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.437592 11879 consensus_queue.cc:1048] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:19.440272 10646 raft_consensus.cc:1275] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Refusing update from remote peer 5a12f1c20c674e7f8242df41a9acf5dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.468336 11860 consensus_queue.cc:1048] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:19.503819 10796 raft_consensus.cc:1275] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:19.559159 11841 consensus_queue.cc:1048] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.629516   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:19.721254 10727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "514efa59c7814384b4f80d29eea50908"
dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:38526
I20260709 13:05:19.723394 10727 raft_consensus.cc:606] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:19.730310 10796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79"
dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
mode: GRACEFUL
new_leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
 from {username='slave'} at 127.0.0.1:33360
I20260709 13:05:19.730945 10796 raft_consensus.cc:606] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Received request to transfer leadership to TS 73db933c4a6848d1b2aeb56813f7bb2c
I20260709 13:05:19.734131 10727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c"
dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:38526
I20260709 13:05:19.734720 10727 raft_consensus.cc:606] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:19.741578 10725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55"
dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:38526
I20260709 13:05:19.742321 10725 raft_consensus.cc:606] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:19.743659   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:05:19.746060   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:20.008689 11846 raft_consensus.cc:993] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:20.010250 11846 raft_consensus.cc:1081] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:20.011759 11885 raft_consensus.cc:993] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:20.012434 11790 raft_consensus.cc:1081] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:20.013789 11557 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56908
I20260709 13:05:20.014560 11557 raft_consensus.cc:493] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:20.015051 11557 raft_consensus.cc:3060] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.015228 11554 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "514efa59c7814384b4f80d29eea50908"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56908
I20260709 13:05:20.015926 11554 raft_consensus.cc:493] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:20.016314 11554 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:20.026520 11554 raft_consensus.cc:515] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:20.027412 11557 raft_consensus.cc:515] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:20.029930 11554 leader_election.cc:290] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 5 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:20.030130 10645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:20.030777 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:20.031023 10645 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:20.031440 10724 raft_consensus.cc:3055] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:20.031787 10724 raft_consensus.cc:740] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:20.032629 10724 consensus_queue.cc:260] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:20.033839 10724 raft_consensus.cc:3060] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.033957 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514efa59c7814384b4f80d29eea50908" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:20.034662 10727 raft_consensus.cc:3055] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:20.034992 10727 raft_consensus.cc:740] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:20.037216 10727 consensus_queue.cc:260] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:20.038738 10727 raft_consensus.cc:3060] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:20.044090 11557 leader_election.cc:290] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:20.056075 10727 raft_consensus.cc:2468] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 5.
I20260709 13:05:20.056339 10724 raft_consensus.cc:2468] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:20.058377 11464 leader_election.cc:304] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:20.059602 11464 leader_election.cc:304] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:20.060242 11879 raft_consensus.cc:2804] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:20.060544 11841 raft_consensus.cc:2804] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:20.062005 11841 raft_consensus.cc:697] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:20.063192 11841 consensus_queue.cc:237] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:20.067819 10792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bc0b50fa5648f2bd176e361fdf2b55" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:20.068792 10792 raft_consensus.cc:3060] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.071766 10645 raft_consensus.cc:2468] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 5.
I20260709 13:05:20.061710 11879 raft_consensus.cc:697] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 5 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:20.083307 11879 consensus_queue.cc:237] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:20.110553 10792 raft_consensus.cc:2468] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:20.146888 10472 catalog_manager.cc:5697] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 4 to 5, leader changed from 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:20.151731 11795 raft_consensus.cc:993] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc: : Instructing follower 73db933c4a6848d1b2aeb56813f7bb2c to start an election
I20260709 13:05:20.152486 11795 raft_consensus.cc:1081] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Signalling peer 73db933c4a6848d1b2aeb56813f7bb2c to start an election
I20260709 13:05:20.152850 10472 catalog_manager.cc:5697] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 1 to 2, leader changed from 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:20.154388 10645 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79"
dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
 from {username='slave'} at 127.0.0.1:48306
I20260709 13:05:20.155042 10645 raft_consensus.cc:493] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:20.155396 10645 raft_consensus.cc:3060] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:20.163209 10645 raft_consensus.cc:515] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:20.167294 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:20.171398 10727 raft_consensus.cc:3060] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:20.179381 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a5f26bcd6404b9b9df57d2c62ee79" candidate_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:20.180539 10796 raft_consensus.cc:3055] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:20.181013 10796 raft_consensus.cc:740] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5a12f1c20c674e7f8242df41a9acf5dc, State: Running, Role: LEADER
I20260709 13:05:20.182375 10727 raft_consensus.cc:2468] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 3.
I20260709 13:05:20.184227 10796 consensus_queue.cc:260] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:20.185937 10796 raft_consensus.cc:3060] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:20.190258 10616 leader_election.cc:304] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:20.192198 10645 leader_election.cc:290] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [CANDIDATE]: Term 3 election: Requested vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707)
I20260709 13:05:20.195323 11877 raft_consensus.cc:2804] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:20.195964 11877 raft_consensus.cc:697] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 LEADER]: Becoming Leader. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:05:20.196926 11877 consensus_queue.cc:237] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } }
I20260709 13:05:20.215209 10796 raft_consensus.cc:2468] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73db933c4a6848d1b2aeb56813f7bb2c in term 3.
I20260709 13:05:20.223330 11790 raft_consensus.cc:993] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:20.224377 11846 raft_consensus.cc:1081] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:20.223824 10475 catalog_manager.cc:5697] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c reported cstate change: term changed from 2 to 3, leader changed from 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196) to 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194). New cstate: current_term: 3 leader_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:20.226933 11551 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56908
I20260709 13:05:20.227573 11551 raft_consensus.cc:493] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:20.228020 11551 raft_consensus.cc:3060] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.241074 11551 raft_consensus.cc:515] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:20.247468 10645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:20.248157 10645 raft_consensus.cc:3060] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.262045 10645 raft_consensus.cc:2468] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:20.263193 11551 leader_election.cc:290] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:20.264434 11467 leader_election.cc:304] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:20.265411 11841 raft_consensus.cc:2804] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:20.265775 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c676741447624a8080294db4e3b3cd4c" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:20.265952 11841 raft_consensus.cc:697] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:20.266563 10727 raft_consensus.cc:3055] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:20.267001 10727 raft_consensus.cc:740] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:20.267572 11841 consensus_queue.cc:237] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:20.267989 10727 consensus_queue.cc:260] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:20.269979 10727 raft_consensus.cc:3060] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:20.282581 10727 raft_consensus.cc:2468] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:20.288481 10475 catalog_manager.cc:5697] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 1 to 2, leader changed from 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:20.501463 10727 raft_consensus.cc:1275] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:20.503525 11841 consensus_queue.cc:1048] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:20.545636 10793 raft_consensus.cc:1275] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:20.547778 11879 consensus_queue.cc:1048] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:20.639505 10724 raft_consensus.cc:1275] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 5 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:20.641299 11880 consensus_queue.cc:1048] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:20.657258 10649 raft_consensus.cc:1275] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 5 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:20.662520 11841 consensus_queue.cc:1048] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:20.683972 10724 raft_consensus.cc:1275] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Refusing update from remote peer 73db933c4a6848d1b2aeb56813f7bb2c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:20.686156 11886 consensus_queue.cc:1048] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:20.726301 10793 raft_consensus.cc:1275] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Refusing update from remote peer 73db933c4a6848d1b2aeb56813f7bb2c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:20.737277 11877 consensus_queue.cc:1048] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:20.745157 10649 raft_consensus.cc:1275] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:20.747133   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:20.746965 11880 consensus_queue.cc:1048] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:20.756515 10725 raft_consensus.cc:1275] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:20.758729 11841 consensus_queue.cc:1048] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:20.955152   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:21.016345 10649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b"
dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
mode: GRACEFUL
new_leader_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:59746
I20260709 13:05:21.017105 10649 raft_consensus.cc:606] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Received request to transfer leadership to TS bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:21.018775   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:05:21.021283   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:21.167774 11886 raft_consensus.cc:993] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c: : Instructing follower bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:21.168509 11886 raft_consensus.cc:1081] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Signalling peer bef3584ebb2c4a2ea619936c71900230 to start an election
I20260709 13:05:21.172994 11551 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b"
dest_uuid: "bef3584ebb2c4a2ea619936c71900230"
 from {username='slave'} at 127.0.0.1:56902
I20260709 13:05:21.173890 11551 raft_consensus.cc:493] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:21.175585 11551 raft_consensus.cc:3060] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.187429 11551 raft_consensus.cc:515] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:21.190382 11551 leader_election.cc:290] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Requested vote from peers 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755), 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475)
I20260709 13:05:21.200918 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce"
I20260709 13:05:21.201730 10724 raft_consensus.cc:3060] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.206213 10649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a502395920c4836b0dfb455c04e108b" candidate_uuid: "bef3584ebb2c4a2ea619936c71900230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73db933c4a6848d1b2aeb56813f7bb2c"
I20260709 13:05:21.213305 10649 raft_consensus.cc:3055] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:21.213939 10649 raft_consensus.cc:740] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73db933c4a6848d1b2aeb56813f7bb2c, State: Running, Role: LEADER
I20260709 13:05:21.215054 10649 consensus_queue.cc:260] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:21.216946 10649 raft_consensus.cc:3060] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.212530 10724 raft_consensus.cc:2468] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:21.222698 11464 leader_election.cc:304] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230; no voters: 
I20260709 13:05:21.223832 11879 raft_consensus.cc:2804] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:21.224378 11879 raft_consensus.cc:697] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Becoming Leader. State: Replica: bef3584ebb2c4a2ea619936c71900230, State: Running, Role: LEADER
I20260709 13:05:21.225390 11879 consensus_queue.cc:237] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:21.250326 10649 raft_consensus.cc:2468] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef3584ebb2c4a2ea619936c71900230 in term 2.
I20260709 13:05:21.263396 10472 catalog_manager.cc:5697] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 reported cstate change: term changed from 1 to 2, leader changed from 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194) to bef3584ebb2c4a2ea619936c71900230 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "bef3584ebb2c4a2ea619936c71900230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:21.651163 10645 raft_consensus.cc:1275] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:21.654321 11880 consensus_queue.cc:1048] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:21.715026 10727 raft_consensus.cc:1275] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Refusing update from remote peer bef3584ebb2c4a2ea619936c71900230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:21.717327 11911 consensus_queue.cc:1048] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:22.022634   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:22.103233   431 tablet_server.cc:179] TabletServer@127.0.107.193:44795 shutting down...
W20260709 13:05:22.121089 10618 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 330 similar messages]
W20260709 13:05:22.160931 10618 consensus_peers.cc:597] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.196107 10618 consensus_peers.cc:597] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.221837 10618 consensus_peers.cc:597] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.226804 10692 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:44795, user_credentials={real_user=slave}} blocked reactor thread for 82540.3us
W20260709 13:05:22.294384 10692 consensus_peers.cc:597] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.297596 10692 consensus_peers.cc:597] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.298453 10616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34475, user_credentials={real_user=slave}} blocked reactor thread for 85815.9us
W20260709 13:05:22.298686 10692 consensus_peers.cc:597] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.309924 10692 consensus_peers.cc:597] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.310824 10618 consensus_peers.cc:597] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.359380 10766 consensus_peers.cc:597] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.369225 10692 consensus_peers.cc:597] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.379124 10692 consensus_peers.cc:597] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.388998 10692 consensus_peers.cc:597] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.394132   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:22.396622   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:05:22.397606 10618 consensus_peers.cc:597] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.397945   431 raft_consensus.cc:2243] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.398659   431 raft_consensus.cc:2272] T 95882d5aa57c46ba86be3160648fc85a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.399982 10692 consensus_peers.cc:597] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.402055   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.403002   431 raft_consensus.cc:2243] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.403688   431 raft_consensus.cc:2272] T a5d1771d12424af18f74abf7650fab3a P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.407776   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.408866   431 raft_consensus.cc:2243] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:22.409090 10692 consensus_peers.cc:597] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.409443   431 raft_consensus.cc:2272] T dbf062bca1cd416181f4c017fecb396c P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.413882   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.414999   431 raft_consensus.cc:2243] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.415686   431 raft_consensus.cc:2272] T be849d4cbf0b4f7a99f9935a55611658 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.418794   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.419632   431 raft_consensus.cc:2243] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.420125   431 raft_consensus.cc:2272] T 5ec45eaa322240d2af46cc177f87d32f P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.423552   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.424695   431 raft_consensus.cc:2243] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.425292   431 raft_consensus.cc:2272] T 31e856ce94cd487395746d76e17c555a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.428467   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.429194   431 raft_consensus.cc:2243] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.429713   431 raft_consensus.cc:2272] T 6e6f6d7fae4f453994e7b12e063b6b49 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.433789   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.434736   431 raft_consensus.cc:2243] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.435266   431 raft_consensus.cc:2272] T 72c64556f12543bfbe8833b32eaf8366 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.437892 10692 consensus_peers.cc:597] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.439910   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.440948   431 raft_consensus.cc:2243] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.441628   431 raft_consensus.cc:2272] T 1d4d2b26b86c4f22976db2fc47a3ec1a P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.443684 10766 consensus_peers.cc:597] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.445679   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.446980   431 raft_consensus.cc:2243] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.448146   431 raft_consensus.cc:2272] T b5349592564d4f4d863236517a765d59 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.452068   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.453187   431 raft_consensus.cc:2243] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.453944   431 raft_consensus.cc:2272] T a99af7813c7b40da8a055799f327ac72 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.455124 10766 consensus_peers.cc:597] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.457839 10766 consensus_peers.cc:597] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.458328   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.459637   431 raft_consensus.cc:2243] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.460228   431 raft_consensus.cc:2272] T 44e728368868473181686e35396f3ae5 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.463570 10692 consensus_peers.cc:597] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.464020   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:05:22.464561 10692 consensus_peers.cc:597] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.465098   431 raft_consensus.cc:2243] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.465719   431 raft_consensus.cc:2272] T 5a09eebcf668422aa3c84e1950676dcb P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.469084   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.470044   431 raft_consensus.cc:2243] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.471206   431 raft_consensus.cc:2272] T 9323c8a136794ad692b7dbff074333f3 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.474627   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.475399   431 raft_consensus.cc:2243] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.476372   431 raft_consensus.cc:2272] T 8a502395920c4836b0dfb455c04e108b P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.480222   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.481163   431 raft_consensus.cc:2243] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.482466   431 raft_consensus.cc:2272] T f95de1505ade4f5389a46d7aca7d7136 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.486313   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.487290   431 raft_consensus.cc:2243] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.487957   431 raft_consensus.cc:2272] T b02d8d78f8e0419186c6c09d9cc307d7 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.491492   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:05:22.492261 10618 consensus_peers.cc:597] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.492344   431 raft_consensus.cc:2243] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.493311   431 raft_consensus.cc:2272] T e97c60086a204747ab7fc8b714daa4ea P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.496495   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.497262   431 raft_consensus.cc:2243] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.498265   431 raft_consensus.cc:2272] T 46a5186a491b4b87b1ab0c692ba8c7f2 P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.501068   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.501855   431 raft_consensus.cc:2243] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.502858   431 raft_consensus.cc:2272] T bad0f5b06cbf47c5a6ae75b972ee23a6 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.505795   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.506804   431 raft_consensus.cc:2243] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.507431   431 raft_consensus.cc:2272] T a27c6651a83844a69107fdd6f819ebde P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.511225   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.512228   431 raft_consensus.cc:2243] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.512760   431 raft_consensus.cc:2272] T a5f96bbe7d674a5485410f0aa8ec40e6 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.516188   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.516991   431 raft_consensus.cc:2243] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.517678   431 raft_consensus.cc:2272] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.521370   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.522455   431 raft_consensus.cc:2243] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.523115   431 raft_consensus.cc:2272] T 6e2988fb83d740c6a6b692c5fc8d8105 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.526938   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.528157   431 raft_consensus.cc:2243] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.529543   431 raft_consensus.cc:2272] T e4bc0b50fa5648f2bd176e361fdf2b55 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.533553   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.534560   431 raft_consensus.cc:2243] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.535144   431 raft_consensus.cc:2272] T 97eca41f7f434c02976a77643991e556 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.538532   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.539455   431 raft_consensus.cc:2243] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:05:22.540280 10766 consensus_peers.cc:597] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.540748   431 raft_consensus.cc:2272] T 69cb859af9704c2596574e50f238ce52 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.544302   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.545044   431 raft_consensus.cc:2243] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.545570   431 raft_consensus.cc:2272] T 33dcc19b71a2421eab2c5d20d5affc10 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.548496   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.549257   431 raft_consensus.cc:2243] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.549855   431 raft_consensus.cc:2272] T 46823d8cc8df4f4bafb078e496b9a7d2 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.553184   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.554150   431 raft_consensus.cc:2243] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.555434   431 raft_consensus.cc:2272] T fee952b378da49509e7ced417a0093cc P bef3584ebb2c4a2ea619936c71900230 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.557735 10766 consensus_peers.cc:597] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.559181   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.560346   431 raft_consensus.cc:2243] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.561553   431 raft_consensus.cc:2272] T 514efa59c7814384b4f80d29eea50908 P bef3584ebb2c4a2ea619936c71900230 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.565237   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.566270   431 raft_consensus.cc:2243] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.567517   431 raft_consensus.cc:2272] T c676741447624a8080294db4e3b3cd4c P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.569366 10618 consensus_peers.cc:597] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.571175   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.572283   431 raft_consensus.cc:2243] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.573400   431 raft_consensus.cc:2272] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.578814 10766 consensus_peers.cc:597] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.579922   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.581080   431 raft_consensus.cc:2243] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.581833   431 raft_consensus.cc:2272] T 3bb134b566b6443f887986585ebc296c P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.591208 10766 consensus_peers.cc:597] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.591208   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.593199   431 raft_consensus.cc:2243] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.593924   431 raft_consensus.cc:2272] T add2b1482b6e41b3ba139426c7da8f53 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.597991   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.599123   431 raft_consensus.cc:2243] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.600464   431 raft_consensus.cc:2272] T a65b76e958854f86b7ccf675dfeb85db P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.604477   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.605568   431 raft_consensus.cc:2243] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.607549   431 raft_consensus.cc:2272] T d6ecb24ddae546da9915163c05742a7e P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.611152   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.612100   431 raft_consensus.cc:2243] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.612756   431 raft_consensus.cc:2272] T 8e3d1fe775e74c4684020e6c5732ce13 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.616496   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.617504   431 raft_consensus.cc:2243] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.618170   431 raft_consensus.cc:2272] T 5b14050b5d4a4fb4aa74de480c7d9cdc P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.622066   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.623119   431 raft_consensus.cc:2243] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.623765   431 raft_consensus.cc:2272] T eca28e86e81447cb899fbb41f02a7f03 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.627645   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.629812   431 raft_consensus.cc:2243] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.630508   431 raft_consensus.cc:2272] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.636404 10692 consensus_peers.cc:597] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.646116 10692 consensus_peers.cc:597] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.648721   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.650035   431 raft_consensus.cc:2243] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.650839   431 raft_consensus.cc:2272] T da7141649b494890b1b693fbfc917db0 P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.654935   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.655813   431 raft_consensus.cc:2243] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:22.657141   431 raft_consensus.cc:2272] T e2bb2563ed2a407c9c219a632057e54a P bef3584ebb2c4a2ea619936c71900230 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.663527 10766 consensus_peers.cc:597] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.666137   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.667621   431 raft_consensus.cc:2243] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.668246   431 raft_consensus.cc:2272] T da58d0f3541543dc8678d61df75295f9 P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:22.671707   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:22.672420   431 raft_consensus.cc:2243] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:22.672983   431 raft_consensus.cc:2272] T 2a8082b4eacd414faa49143ba833d6ed P bef3584ebb2c4a2ea619936c71900230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:22.719565 10766 consensus_peers.cc:597] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Couldn't send request to peer bef3584ebb2c4a2ea619936c71900230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.804713   431 tablet_server.cc:196] TabletServer@127.0.107.193:44795 shutdown complete.
I20260709 13:05:23.064663   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:05:23.079077 10693 consensus_peers.cc:597] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.086400 10693 consensus_peers.cc:597] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.134976 10693 consensus_peers.cc:597] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.146876 10693 consensus_peers.cc:597] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.159466 10767 consensus_peers.cc:597] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.167815 10767 consensus_peers.cc:597] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.192557 10767 consensus_peers.cc:597] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.201146 10693 consensus_peers.cc:597] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.225930   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:23.227118   431 tablet_replica.cc:333] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.228211   431 raft_consensus.cc:2243] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.228875   431 raft_consensus.cc:2272] T 315bc88bbbfa4be190f5d7581ace8793 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.232264   431 tablet_replica.cc:333] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.233307   431 raft_consensus.cc:2243] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.233930   431 raft_consensus.cc:2272] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.238040   431 tablet_replica.cc:333] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.239077   431 raft_consensus.cc:2243] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.239864   431 raft_consensus.cc:2272] T 2a8082b4eacd414faa49143ba833d6ed P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.240437 10693 consensus_peers.cc:597] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.243726   431 tablet_replica.cc:333] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
W20260709 13:05:23.244321 10693 consensus_peers.cc:597] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.244928   431 raft_consensus.cc:2243] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.245457   431 raft_consensus.cc:2272] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.248028   431 tablet_replica.cc:333] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.248699   431 raft_consensus.cc:2243] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.249675   431 raft_consensus.cc:2272] T 97eca41f7f434c02976a77643991e556 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.253530   431 tablet_replica.cc:333] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.254526   431 raft_consensus.cc:2243] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.255173   431 raft_consensus.cc:2272] T 514efa59c7814384b4f80d29eea50908 P 73db933c4a6848d1b2aeb56813f7bb2c [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.257098 10693 consensus_peers.cc:597] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.259374   431 tablet_replica.cc:333] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.260326   431 raft_consensus.cc:2243] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.260898   431 raft_consensus.cc:2272] T c676741447624a8080294db4e3b3cd4c P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.262094 10767 consensus_peers.cc:597] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.264150   431 tablet_replica.cc:333] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.265290   431 raft_consensus.cc:2243] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.266628   431 raft_consensus.cc:2272] T fb17118da73149cbb340865c5c6959c5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.268743 10767 consensus_peers.cc:597] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.270190   431 tablet_replica.cc:333] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.271281   431 raft_consensus.cc:2243] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.271914   431 raft_consensus.cc:2272] T 8a502395920c4836b0dfb455c04e108b P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.274895   431 tablet_replica.cc:333] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.275734   431 raft_consensus.cc:2243] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.276177   431 raft_consensus.cc:2272] T 6d33bf08ed78416ea29aaf59733e35a2 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.278738   431 tablet_replica.cc:333] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.279492   431 raft_consensus.cc:2243] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.279999   431 raft_consensus.cc:2272] T 6e6f6d7fae4f453994e7b12e063b6b49 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.282559   431 tablet_replica.cc:333] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.283255   431 raft_consensus.cc:2243] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.284206   431 raft_consensus.cc:2272] T 413b57e5acb54f9b86a0741e6f235ad3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.287025   431 tablet_replica.cc:333] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.287968   431 raft_consensus.cc:2243] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.288550   431 raft_consensus.cc:2272] T 9323c8a136794ad692b7dbff074333f3 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.291318   431 tablet_replica.cc:333] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.292094   431 raft_consensus.cc:2243] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.292595   431 raft_consensus.cc:2272] T be849d4cbf0b4f7a99f9935a55611658 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.295416   431 tablet_replica.cc:333] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
W20260709 13:05:23.295993 10767 consensus_peers.cc:597] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.296329   431 raft_consensus.cc:2243] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.297242   431 raft_consensus.cc:2272] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.299476   431 tablet_replica.cc:333] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.300057   431 raft_consensus.cc:2243] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.300843   431 raft_consensus.cc:2272] T 6e2988fb83d740c6a6b692c5fc8d8105 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.303270   431 tablet_replica.cc:333] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.303928   431 raft_consensus.cc:2243] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.304456   431 raft_consensus.cc:2272] T a48d84eceb174792919f6256ee4fbb21 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.306779   431 tablet_replica.cc:333] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.307462   431 raft_consensus.cc:2243] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.307934   431 raft_consensus.cc:2272] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.311080   431 tablet_replica.cc:333] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.311722   431 raft_consensus.cc:2243] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:23.311762 10693 consensus_peers.cc:597] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.312968   431 raft_consensus.cc:2272] T 12b99c17dd874ae2bd745e4ed1ba0330 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.316078   431 tablet_replica.cc:333] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.317023   431 raft_consensus.cc:2243] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.317739   431 raft_consensus.cc:2272] T 72c64556f12543bfbe8833b32eaf8366 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.322041   431 tablet_replica.cc:333] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.322736   431 raft_consensus.cc:2243] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.323699   431 raft_consensus.cc:2272] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.326534   431 tablet_replica.cc:333] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.327324   431 raft_consensus.cc:2243] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.327919   431 raft_consensus.cc:2272] T f95de1505ade4f5389a46d7aca7d7136 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.330950   431 tablet_replica.cc:333] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.331893   431 raft_consensus.cc:2243] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.332650   431 raft_consensus.cc:2272] T 95882d5aa57c46ba86be3160648fc85a P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.333400 10767 consensus_peers.cc:597] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.336192   431 tablet_replica.cc:333] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.337607   431 raft_consensus.cc:2243] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.338353   431 raft_consensus.cc:2272] T fee952b378da49509e7ced417a0093cc P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.342293   431 tablet_replica.cc:333] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.343308   431 raft_consensus.cc:2243] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.344039   431 raft_consensus.cc:2272] T 33dcc19b71a2421eab2c5d20d5affc10 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.344357 10767 consensus_peers.cc:597] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.347700   431 tablet_replica.cc:333] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.348629   431 raft_consensus.cc:2243] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.349293   431 raft_consensus.cc:2272] T da58d0f3541543dc8678d61df75295f9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.352767   431 tablet_replica.cc:333] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.353657   431 raft_consensus.cc:2243] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.354153   431 raft_consensus.cc:2272] T e2bb2563ed2a407c9c219a632057e54a P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.356338 10767 consensus_peers.cc:597] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.357437   431 tablet_replica.cc:333] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.358332   431 raft_consensus.cc:2243] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.359242   431 raft_consensus.cc:2272] T 8e3d1fe775e74c4684020e6c5732ce13 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.361727   431 tablet_replica.cc:333] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.362365   431 raft_consensus.cc:2243] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.363237   431 raft_consensus.cc:2272] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.365578   431 tablet_replica.cc:333] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.366263   431 raft_consensus.cc:2243] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.367192   431 raft_consensus.cc:2272] T 5b4c2675dde04ec982395518aa83bfb0 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.369738   431 tablet_replica.cc:333] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.370487   431 raft_consensus.cc:2243] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.371608   431 raft_consensus.cc:2272] T eca28e86e81447cb899fbb41f02a7f03 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.375118   431 tablet_replica.cc:333] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.375944   431 raft_consensus.cc:2243] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.377054   431 raft_consensus.cc:2272] T e97c60086a204747ab7fc8b714daa4ea P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.380381   431 tablet_replica.cc:333] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.381052   431 raft_consensus.cc:2243] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.381700   431 raft_consensus.cc:2272] T b02d8d78f8e0419186c6c09d9cc307d7 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.384780   431 tablet_replica.cc:333] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.385684   431 raft_consensus.cc:2243] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.386221   431 raft_consensus.cc:2272] T a27c6651a83844a69107fdd6f819ebde P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.389038   431 tablet_replica.cc:333] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.390007   431 raft_consensus.cc:2243] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.390988   431 raft_consensus.cc:2272] T 9f26ffe11b8143728804f33b884ed36a P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.393325   431 tablet_replica.cc:333] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.394029   431 raft_consensus.cc:2243] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.394482   431 raft_consensus.cc:2272] T 656245aa555f45a8a6ef044f0da06e46 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.396625   431 tablet_replica.cc:333] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.397207   431 raft_consensus.cc:2243] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.397714   431 raft_consensus.cc:2272] T 5ec45eaa322240d2af46cc177f87d32f P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.399904   431 tablet_replica.cc:333] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.400463   431 raft_consensus.cc:2243] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.400910   431 raft_consensus.cc:2272] T 44e728368868473181686e35396f3ae5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.403038   431 tablet_replica.cc:333] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.403604   431 raft_consensus.cc:2243] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.404505   431 raft_consensus.cc:2272] T 6224db6d69b24c2ba435311454b28dc9 P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.406970   431 tablet_replica.cc:333] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.407554   431 raft_consensus.cc:2243] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.408386   431 raft_consensus.cc:2272] T 756a5f26bcd6404b9b9df57d2c62ee79 P 73db933c4a6848d1b2aeb56813f7bb2c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.410981   431 tablet_replica.cc:333] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.411672   431 raft_consensus.cc:2243] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.412238   431 raft_consensus.cc:2272] T 5a09eebcf668422aa3c84e1950676dcb P 73db933c4a6848d1b2aeb56813f7bb2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.414597   431 tablet_replica.cc:333] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.415175   431 raft_consensus.cc:2243] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.416064   431 raft_consensus.cc:2272] T ec71cde09fe64d089f95382f32269db5 P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.418402   431 tablet_replica.cc:333] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.418968   431 raft_consensus.cc:2243] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.419549   431 raft_consensus.cc:2272] T 9887fc1ac0ac404e9f590987c2f0b0be P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.422722   431 tablet_replica.cc:333] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c: stopping tablet replica
I20260709 13:05:23.423480   431 raft_consensus.cc:2243] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.424106   431 raft_consensus.cc:2272] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 73db933c4a6848d1b2aeb56813f7bb2c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.429384 10767 consensus_peers.cc:597] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.450024 10767 consensus_peers.cc:597] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Couldn't send request to peer 73db933c4a6848d1b2aeb56813f7bb2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.468180 11795 raft_consensus.cc:493] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:23.468850 11795 raft_consensus.cc:515] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:23.471501 11795 leader_election.cc:290] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:23.471777 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69cb859af9704c2596574e50f238ce52" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
W20260709 13:05:23.476070 10766 leader_election.cc:336] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:23.476900 10766 leader_election.cc:304] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:23.477788 11795 raft_consensus.cc:2749] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:23.493170 11795 raft_consensus.cc:493] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:23.493954 11795 raft_consensus.cc:515] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:23.496196 11795 leader_election.cc:290] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
W20260709 13:05:23.501950 10766 leader_election.cc:336] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:23.507027 11795 raft_consensus.cc:493] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:23.507673 11795 raft_consensus.cc:515] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:23.510488 11795 leader_election.cc:290] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:23.510794 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" is_pre_election: true
I20260709 13:05:23.511586   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:05:23.513924 10767 leader_election.cc:336] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43755: connect: Connection refused (error 111)
I20260709 13:05:23.514506 10767 leader_election.cc:304] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:23.515744 11795 raft_consensus.cc:2749] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:05:23.517623 10766 leader_election.cc:336] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:23.518100 10766 leader_election.cc:304] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:23.518863 11929 raft_consensus.cc:2749] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:23.554351 11846 raft_consensus.cc:493] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:23.555089 11846 raft_consensus.cc:515] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:23.557255 11846 leader_election.cc:290] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:23.558193 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65b76e958854f86b7ccf675dfeb85db" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
W20260709 13:05:23.562289 10692 leader_election.cc:336] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:23.562810 10692 leader_election.cc:304] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:23.563609 11846 raft_consensus.cc:2749] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:23.614838   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:05:23.642740 11885 raft_consensus.cc:493] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:23.643503 11885 raft_consensus.cc:515] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:23.646237 11885 leader_election.cc:290] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
W20260709 13:05:23.656061 10692 leader_election.cc:336] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
W20260709 13:05:23.657347 10693 leader_election.cc:336] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43755: connect: Connection refused (error 111)
I20260709 13:05:23.658046 10693 leader_election.cc:304] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:23.658813 11846 raft_consensus.cc:2749] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:23.666811 10764 consensus_peers.cc:597] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Couldn't send request to peer 7ecd836bb7a144dc9aa07a046239d6ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.686708   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:23.688027   431 tablet_replica.cc:333] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.689126   431 raft_consensus.cc:2243] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.690286   431 raft_consensus.cc:2272] T 31e856ce94cd487395746d76e17c555a P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.692938   431 tablet_replica.cc:333] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.693655   431 raft_consensus.cc:2243] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.694099   431 raft_consensus.cc:2272] T 315bc88bbbfa4be190f5d7581ace8793 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.696774   431 tablet_replica.cc:333] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.697751   431 raft_consensus.cc:2243] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.698333   431 raft_consensus.cc:2272] T 8a502395920c4836b0dfb455c04e108b P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.701613   431 tablet_replica.cc:333] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.702567   431 raft_consensus.cc:2243] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.704181   431 raft_consensus.cc:2272] T a5d1771d12424af18f74abf7650fab3a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.707505   431 tablet_replica.cc:333] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.708549   431 raft_consensus.cc:2243] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.709583   431 raft_consensus.cc:2272] T 6c617206d4bb4c0d84e4f8f6cccdbf30 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.713675   431 tablet_replica.cc:333] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.714473   431 raft_consensus.cc:2243] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.715402   431 raft_consensus.cc:2272] T da7141649b494890b1b693fbfc917db0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.717937   431 tablet_replica.cc:333] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.718607   431 raft_consensus.cc:2243] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.718691 11795 raft_consensus.cc:493] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:23.719095   431 raft_consensus.cc:2272] T 6d33bf08ed78416ea29aaf59733e35a2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.719282 11795 raft_consensus.cc:515] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:23.721164 11795 leader_election.cc:290] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:23.722165   431 tablet_replica.cc:333] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.723201   431 raft_consensus.cc:2243] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.724186   431 raft_consensus.cc:2272] T 6e6f6d7fae4f453994e7b12e063b6b49 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.725560 10766 leader_election.cc:336] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
W20260709 13:05:23.726261 10767 leader_election.cc:336] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43755: connect: Connection refused (error 111)
I20260709 13:05:23.726678 10767 leader_election.cc:304] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:23.727340 11795 raft_consensus.cc:2749] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:23.727672   431 tablet_replica.cc:333] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.728888   431 raft_consensus.cc:2243] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.729579   431 raft_consensus.cc:2272] T 413b57e5acb54f9b86a0741e6f235ad3 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.732533   431 tablet_replica.cc:333] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.733347   431 raft_consensus.cc:2243] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.733986   431 raft_consensus.cc:2272] T 514efa59c7814384b4f80d29eea50908 P 7ecd836bb7a144dc9aa07a046239d6ce [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.735733 11931 raft_consensus.cc:493] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:23.736333 11931 raft_consensus.cc:515] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:23.737327   431 tablet_replica.cc:333] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.738471   431 raft_consensus.cc:2243] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.739136 11846 raft_consensus.cc:493] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:23.739761 11846 raft_consensus.cc:515] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } } peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } }
I20260709 13:05:23.740468 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:23.739148   431 raft_consensus.cc:2272] T c676741447624a8080294db4e3b3cd4c P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.741209 10796 raft_consensus.cc:2468] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 3.
I20260709 13:05:23.739045 11931 leader_election.cc:290] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:23.741919 11846 leader_election.cc:290] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:23.744191   431 tablet_replica.cc:333] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.745687   431 raft_consensus.cc:2243] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.746400   431 raft_consensus.cc:2272] T fb17118da73149cbb340865c5c6959c5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.749809   431 tablet_replica.cc:333] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.750730   431 raft_consensus.cc:2243] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.750236 10693 leader_election.cc:304] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 
I20260709 13:05:23.752161   431 raft_consensus.cc:2272] T 69cb859af9704c2596574e50f238ce52 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.752241 11885 raft_consensus.cc:2804] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:23.752771 11885 raft_consensus.cc:493] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:23.753173 11885 raft_consensus.cc:3060] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:23.760452 10692 leader_election.cc:336] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
W20260709 13:05:23.764832 10764 consensus_peers.cc:597] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Couldn't send request to peer 7ecd836bb7a144dc9aa07a046239d6ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.765534   431 tablet_replica.cc:333] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
W20260709 13:05:23.765594 10692 leader_election.cc:336] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:23.766760 11846 raft_consensus.cc:493] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
W20260709 13:05:23.767443 10693 leader_election.cc:336] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43755: connect: Connection refused (error 111)
I20260709 13:05:23.767978 10693 leader_election.cc:304] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:23.768651 11931 raft_consensus.cc:2749] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:23.767252 11846 raft_consensus.cc:515] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:23.769797 11940 raft_consensus.cc:493] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:23.770397 11940 raft_consensus.cc:515] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:23.771255 11846 leader_election.cc:290] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:23.774091 11940 leader_election.cc:290] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:23.774474   431 raft_consensus.cc:2243] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.775537   431 raft_consensus.cc:2272] T 46823d8cc8df4f4bafb078e496b9a7d2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.778045 10766 leader_election.cc:336] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:23.778055 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a5186a491b4b87b1ab0c692ba8c7f2" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" is_pre_election: true
I20260709 13:05:23.779871   431 tablet_replica.cc:333] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.781008   431 raft_consensus.cc:2243] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.780694 11885 raft_consensus.cc:515] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:23.781785   431 raft_consensus.cc:2272] T 9887fc1ac0ac404e9f590987c2f0b0be P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.785663 11885 leader_election.cc:290] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 election: Requested vote from peers 5a12f1c20c674e7f8242df41a9acf5dc (127.0.107.196:38707), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:23.786422 10796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5349592564d4f4d863236517a765d59" candidate_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a12f1c20c674e7f8242df41a9acf5dc"
I20260709 13:05:23.787212   431 tablet_replica.cc:333] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
W20260709 13:05:23.788203 10767 leader_election.cc:336] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43755: connect: Connection refused (error 111)
W20260709 13:05:23.788280 10692 leader_election.cc:336] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:23.788794 10796 raft_consensus.cc:3060] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:23.788976 10767 leader_election.cc:304] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:23.789412 10692 leader_election.cc:304] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ecd836bb7a144dc9aa07a046239d6ce; no voters: 5a12f1c20c674e7f8242df41a9acf5dc, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:23.789676   431 raft_consensus.cc:2243] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.790457   431 raft_consensus.cc:2272] T a48d84eceb174792919f6256ee4fbb21 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.791669 11885 raft_consensus.cc:2749] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:23.794292   431 tablet_replica.cc:333] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.795431   431 raft_consensus.cc:2243] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.796167 11795 raft_consensus.cc:2749] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:23.796774   431 raft_consensus.cc:2272] T dbf062bca1cd416181f4c017fecb396c P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.797251 10692 leader_election.cc:336] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:23.798323 10796 raft_consensus.cc:2468] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ecd836bb7a144dc9aa07a046239d6ce in term 4.
I20260709 13:05:23.800302 10693 leader_election.cc:304] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc, 7ecd836bb7a144dc9aa07a046239d6ce; no voters: bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:23.801395   431 tablet_replica.cc:333] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.801334 11885 raft_consensus.cc:2804] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:23.802282 11885 raft_consensus.cc:697] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 LEADER]: Becoming Leader. State: Replica: 7ecd836bb7a144dc9aa07a046239d6ce, State: Running, Role: LEADER
I20260709 13:05:23.802569   431 raft_consensus.cc:2243] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.803220   431 raft_consensus.cc:2272] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.806550   431 tablet_replica.cc:333] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.806725 11885 consensus_queue.cc:237] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:23.807569   431 raft_consensus.cc:2243] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.808226   431 raft_consensus.cc:2272] T 3bb134b566b6443f887986585ebc296c P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.811735   431 tablet_replica.cc:333] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.812500   431 raft_consensus.cc:2243] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.813679   431 raft_consensus.cc:2272] T a5f96bbe7d674a5485410f0aa8ec40e6 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.817926   431 tablet_replica.cc:333] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.819020   431 raft_consensus.cc:2243] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.819787   431 raft_consensus.cc:2272] T ec71cde09fe64d089f95382f32269db5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.823506   431 tablet_replica.cc:333] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.824492   431 raft_consensus.cc:2243] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.825865   431 raft_consensus.cc:2272] T 12b99c17dd874ae2bd745e4ed1ba0330 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.827378 10764 consensus_peers.cc:597] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Couldn't send request to peer 7ecd836bb7a144dc9aa07a046239d6ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.828840   431 tablet_replica.cc:333] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.829700   431 raft_consensus.cc:2243] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.830269   431 raft_consensus.cc:2272] T f95de1505ade4f5389a46d7aca7d7136 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.832847   431 tablet_replica.cc:333] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.833628   431 raft_consensus.cc:2243] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.834587   431 raft_consensus.cc:2272] T 72c64556f12543bfbe8833b32eaf8366 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.837126   431 tablet_replica.cc:333] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.837886   431 raft_consensus.cc:2243] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.838639   431 raft_consensus.cc:2272] T 9f26ffe11b8143728804f33b884ed36a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.841933   431 tablet_replica.cc:333] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.842625   431 raft_consensus.cc:2243] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.843151   431 raft_consensus.cc:2272] T d6ecb24ddae546da9915163c05742a7e P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.845593   431 tablet_replica.cc:333] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.845834 11940 raft_consensus.cc:493] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:23.846632   431 raft_consensus.cc:2243] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.846621 11940 raft_consensus.cc:515] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:23.847968   431 raft_consensus.cc:2272] T da58d0f3541543dc8678d61df75295f9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.849251 11940 leader_election.cc:290] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
I20260709 13:05:23.851553   431 tablet_replica.cc:333] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
W20260709 13:05:23.852385 10764 leader_election.cc:336] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:23.852468   431 raft_consensus.cc:2243] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.853404   431 raft_consensus.cc:2272] T e2bb2563ed2a407c9c219a632057e54a P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.854099 10766 leader_election.cc:336] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:23.854516 10766 leader_election.cc:304] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:23.855161 11940 raft_consensus.cc:2749] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:23.856454   431 tablet_replica.cc:333] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.857324   431 raft_consensus.cc:2243] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.858486   431 raft_consensus.cc:2272] T b02d8d78f8e0419186c6c09d9cc307d7 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.862057   431 tablet_replica.cc:333] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.863027   431 raft_consensus.cc:2243] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.863657   431 raft_consensus.cc:2272] T a99af7813c7b40da8a055799f327ac72 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.866995   431 tablet_replica.cc:333] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.867949   431 raft_consensus.cc:2243] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.868952   431 pending_rounds.cc:70] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce: Trying to abort 1 pending ops.
I20260709 13:05:23.869257   431 pending_rounds.cc:77] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305635118325329920 op_type: NO_OP noop_request { }
I20260709 13:05:23.869735   431 raft_consensus.cc:2889] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:23.870119   431 raft_consensus.cc:2272] T b5349592564d4f4d863236517a765d59 P 7ecd836bb7a144dc9aa07a046239d6ce [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.872757 10764 consensus_peers.cc:597] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Couldn't send request to peer 7ecd836bb7a144dc9aa07a046239d6ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.873674   431 tablet_replica.cc:333] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.874454   431 raft_consensus.cc:2243] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.875041   431 raft_consensus.cc:2272] T 756a5f26bcd6404b9b9df57d2c62ee79 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.878225   431 tablet_replica.cc:333] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.878948   431 raft_consensus.cc:2243] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.879913   431 raft_consensus.cc:2272] T add2b1482b6e41b3ba139426c7da8f53 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.883483   431 tablet_replica.cc:333] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.884262   431 raft_consensus.cc:2243] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.884953   431 raft_consensus.cc:2272] T a65b76e958854f86b7ccf675dfeb85db P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.888036   431 tablet_replica.cc:333] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.888883   431 raft_consensus.cc:2243] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.889631   431 raft_consensus.cc:2272] T 656245aa555f45a8a6ef044f0da06e46 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.892925   431 tablet_replica.cc:333] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.893885   431 raft_consensus.cc:2243] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.895058   431 raft_consensus.cc:2272] T 5ec45eaa322240d2af46cc177f87d32f P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.897967   431 tablet_replica.cc:333] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.898706   431 raft_consensus.cc:2243] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.899335   431 raft_consensus.cc:2272] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 7ecd836bb7a144dc9aa07a046239d6ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.902665   431 tablet_replica.cc:333] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.903555   431 raft_consensus.cc:2243] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.904803   431 raft_consensus.cc:2272] T 44e728368868473181686e35396f3ae5 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.908023   431 tablet_replica.cc:333] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.908669   431 raft_consensus.cc:2243] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.909163   431 raft_consensus.cc:2272] T 8e3d1fe775e74c4684020e6c5732ce13 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.911656   431 tablet_replica.cc:333] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.912281   431 raft_consensus.cc:2243] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.912787   431 raft_consensus.cc:2272] T eca28e86e81447cb899fbb41f02a7f03 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.915139   431 tablet_replica.cc:333] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.915740   431 raft_consensus.cc:2243] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.916311   431 raft_consensus.cc:2272] T 5b4c2675dde04ec982395518aa83bfb0 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.918805   431 tablet_replica.cc:333] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.919453   431 raft_consensus.cc:2243] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.920033   431 raft_consensus.cc:2272] T e4bc0b50fa5648f2bd176e361fdf2b55 P 7ecd836bb7a144dc9aa07a046239d6ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.923295   431 tablet_replica.cc:333] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.924006   431 raft_consensus.cc:2243] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.924593   431 raft_consensus.cc:2272] T 6224db6d69b24c2ba435311454b28dc9 P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.926824 10764 consensus_peers.cc:597] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Couldn't send request to peer 7ecd836bb7a144dc9aa07a046239d6ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.927644   431 tablet_replica.cc:333] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce: stopping tablet replica
I20260709 13:05:23.928282   431 raft_consensus.cc:2243] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.929198   431 raft_consensus.cc:2272] T 5a09eebcf668422aa3c84e1950676dcb P 7ecd836bb7a144dc9aa07a046239d6ce [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.964169 10764 consensus_peers.cc:597] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Couldn't send request to peer 7ecd836bb7a144dc9aa07a046239d6ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.986485 10764 consensus_peers.cc:597] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Couldn't send request to peer 7ecd836bb7a144dc9aa07a046239d6ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:24.011698 10764 consensus_peers.cc:597] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Couldn't send request to peer 7ecd836bb7a144dc9aa07a046239d6ce. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:34475: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:24.015751   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:05:24.023152 11940 raft_consensus.cc:493] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:24.023924 11940 raft_consensus.cc:515] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:24.026480 11940 leader_election.cc:290] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
W20260709 13:05:24.031535 10764 leader_election.cc:336] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Network error: Client connection negotiation failed: client connection to 127.0.107.195:34475: connect: Connection refused (error 111)
W20260709 13:05:24.032169 10766 leader_election.cc:336] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:24.032637 10766 leader_election.cc:304] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:24.033824 11940 raft_consensus.cc:2749] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:24.075314 11795 raft_consensus.cc:493] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:24.075989 11795 raft_consensus.cc:515] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } } peers { permanent_uuid: "73db933c4a6848d1b2aeb56813f7bb2c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43755 } }
I20260709 13:05:24.078032 11795 leader_election.cc:290] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795), 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755)
I20260709 13:05:24.080164 11795 raft_consensus.cc:493] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bef3584ebb2c4a2ea619936c71900230)
I20260709 13:05:24.080706 11795 raft_consensus.cc:515] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ecd836bb7a144dc9aa07a046239d6ce" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34475 } } peers { permanent_uuid: "5a12f1c20c674e7f8242df41a9acf5dc" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38707 } } peers { permanent_uuid: "bef3584ebb2c4a2ea619936c71900230" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44795 } }
I20260709 13:05:24.084102 11795 leader_election.cc:290] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475), bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795)
W20260709 13:05:24.084307 10766 leader_election.cc:336] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
W20260709 13:05:24.084998 10767 leader_election.cc:336] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 73db933c4a6848d1b2aeb56813f7bb2c (127.0.107.194:43755): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43755: connect: Connection refused (error 111)
I20260709 13:05:24.085562 10767 leader_election.cc:304] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 73db933c4a6848d1b2aeb56813f7bb2c, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:24.086728 11929 raft_consensus.cc:2749] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:24.088275 10764 leader_election.cc:336] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Network error: Client connection negotiation failed: client connection to 127.0.107.195:34475: connect: Connection refused (error 111)
W20260709 13:05:24.089862 10766 leader_election.cc:336] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bef3584ebb2c4a2ea619936c71900230 (127.0.107.193:44795): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44795: connect: Connection refused (error 111)
I20260709 13:05:24.090333 10766 leader_election.cc:304] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a12f1c20c674e7f8242df41a9acf5dc; no voters: 7ecd836bb7a144dc9aa07a046239d6ce, bef3584ebb2c4a2ea619936c71900230
I20260709 13:05:24.091054 11795 raft_consensus.cc:2749] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:24.116775   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
W20260709 13:05:24.125059 10764 consensus_peers.cc:597] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc -> Peer 7ecd836bb7a144dc9aa07a046239d6ce (127.0.107.195:34475): Couldn't send request to peer 7ecd836bb7a144dc9aa07a046239d6ce. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:34475: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:24.180418   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:24.181391   431 tablet_replica.cc:333] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.182307   431 raft_consensus.cc:2243] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.182955   431 raft_consensus.cc:2272] T 31e856ce94cd487395746d76e17c555a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.185628   431 tablet_replica.cc:333] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.186329   431 raft_consensus.cc:2243] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.186802   431 raft_consensus.cc:2272] T 97eca41f7f434c02976a77643991e556 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.189174   431 tablet_replica.cc:333] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.190075   431 raft_consensus.cc:2243] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.191095   431 raft_consensus.cc:2272] T 315bc88bbbfa4be190f5d7581ace8793 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.194283   431 tablet_replica.cc:333] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.195154   431 raft_consensus.cc:2243] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.196018   431 raft_consensus.cc:2272] T 656245aa555f45a8a6ef044f0da06e46 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.198378   431 tablet_replica.cc:333] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.198964   431 raft_consensus.cc:2243] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.199813   431 raft_consensus.cc:2272] T 2a8082b4eacd414faa49143ba833d6ed P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.202312   431 tablet_replica.cc:333] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.202958   431 raft_consensus.cc:2243] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.203459   431 raft_consensus.cc:2272] T da7141649b494890b1b693fbfc917db0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.205818   431 tablet_replica.cc:333] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.206421   431 raft_consensus.cc:2243] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.207297   431 raft_consensus.cc:2272] T 6d33bf08ed78416ea29aaf59733e35a2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.210026   431 tablet_replica.cc:333] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.210795   431 raft_consensus.cc:2243] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.211309   431 raft_consensus.cc:2272] T 413b57e5acb54f9b86a0741e6f235ad3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.214079   431 tablet_replica.cc:333] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.214799   431 raft_consensus.cc:2243] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.215345   431 raft_consensus.cc:2272] T fb17118da73149cbb340865c5c6959c5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.217869   431 tablet_replica.cc:333] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.218528   431 raft_consensus.cc:2243] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.219012   431 raft_consensus.cc:2272] T 1d4d2b26b86c4f22976db2fc47a3ec1a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.221421   431 tablet_replica.cc:333] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.222180   431 raft_consensus.cc:2243] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.223333   431 raft_consensus.cc:2272] T 3bb134b566b6443f887986585ebc296c P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.226025   431 tablet_replica.cc:333] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.226899   431 raft_consensus.cc:2243] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.227473   431 raft_consensus.cc:2272] T 6e2988fb83d740c6a6b692c5fc8d8105 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.230158   431 tablet_replica.cc:333] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.230863   431 raft_consensus.cc:2243] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.232086   431 raft_consensus.cc:2272] T a48d84eceb174792919f6256ee4fbb21 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.235626   431 tablet_replica.cc:333] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.236845   431 raft_consensus.cc:2243] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.238015   431 raft_consensus.cc:2272] T be849d4cbf0b4f7a99f9935a55611658 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.241011   431 tablet_replica.cc:333] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.241695   431 raft_consensus.cc:2243] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.242290   431 raft_consensus.cc:2272] T dbf062bca1cd416181f4c017fecb396c P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.244889   431 tablet_replica.cc:333] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.245517   431 raft_consensus.cc:2243] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.245937   431 raft_consensus.cc:2272] T 8cfb8e7fc7e448f8b02ff6992503bc35 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.248009   431 tablet_replica.cc:333] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.248539   431 raft_consensus.cc:2243] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.248932   431 raft_consensus.cc:2272] T 69cb859af9704c2596574e50f238ce52 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.250988   431 tablet_replica.cc:333] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.251523   431 raft_consensus.cc:2243] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.251929   431 raft_consensus.cc:2272] T fee952b378da49509e7ced417a0093cc P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.254160   431 tablet_replica.cc:333] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.254684   431 raft_consensus.cc:2243] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.255474   431 raft_consensus.cc:2272] T 33dcc19b71a2421eab2c5d20d5affc10 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.257742   431 tablet_replica.cc:333] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.258339   431 raft_consensus.cc:2243] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.259213   431 raft_consensus.cc:2272] T 46823d8cc8df4f4bafb078e496b9a7d2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.261433   431 tablet_replica.cc:333] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.262140   431 raft_consensus.cc:2243] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.262609   431 raft_consensus.cc:2272] T 9323c8a136794ad692b7dbff074333f3 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.264650   431 tablet_replica.cc:333] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.265254   431 raft_consensus.cc:2243] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.265828   431 raft_consensus.cc:2272] T a5d1771d12424af18f74abf7650fab3a P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.267982   431 tablet_replica.cc:333] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.268586   431 raft_consensus.cc:2243] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.269443   431 raft_consensus.cc:2272] T 95882d5aa57c46ba86be3160648fc85a P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.271517   431 tablet_replica.cc:333] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.272121   431 raft_consensus.cc:2243] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.272588   431 raft_consensus.cc:2272] T add2b1482b6e41b3ba139426c7da8f53 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.275041   431 tablet_replica.cc:333] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.275758   431 raft_consensus.cc:2243] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.276226   431 raft_consensus.cc:2272] T a65b76e958854f86b7ccf675dfeb85db P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.278486   431 tablet_replica.cc:333] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.279069   431 raft_consensus.cc:2243] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.279522   431 raft_consensus.cc:2272] T bad0f5b06cbf47c5a6ae75b972ee23a6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.281622   431 tablet_replica.cc:333] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.282226   431 raft_consensus.cc:2243] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.282790   431 raft_consensus.cc:2272] T 12b99c17dd874ae2bd745e4ed1ba0330 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.284807   431 tablet_replica.cc:333] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.285398   431 raft_consensus.cc:2243] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.286298   431 raft_consensus.cc:2272] T a27c6651a83844a69107fdd6f819ebde P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.288393   431 tablet_replica.cc:333] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.288977   431 raft_consensus.cc:2243] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.289440   431 raft_consensus.cc:2272] T 9f26ffe11b8143728804f33b884ed36a P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.291558   431 tablet_replica.cc:333] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.292137   431 raft_consensus.cc:2243] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.292613   431 raft_consensus.cc:2272] T b5349592564d4f4d863236517a765d59 P 5a12f1c20c674e7f8242df41a9acf5dc [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.294834   431 tablet_replica.cc:333] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.295428   431 raft_consensus.cc:2243] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.296298   431 raft_consensus.cc:2272] T a99af7813c7b40da8a055799f327ac72 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.298321   431 tablet_replica.cc:333] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.298899   431 raft_consensus.cc:2243] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.299324   431 raft_consensus.cc:2272] T 5b14050b5d4a4fb4aa74de480c7d9cdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.301342   431 tablet_replica.cc:333] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.301962   431 raft_consensus.cc:2243] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.302410   431 raft_consensus.cc:2272] T 5b4c2675dde04ec982395518aa83bfb0 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.304504   431 tablet_replica.cc:333] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.305076   431 raft_consensus.cc:2243] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.305552   431 raft_consensus.cc:2272] T e4bc0b50fa5648f2bd176e361fdf2b55 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.307611   431 tablet_replica.cc:333] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.308178   431 raft_consensus.cc:2243] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.308633   431 raft_consensus.cc:2272] T 6224db6d69b24c2ba435311454b28dc9 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.310933   431 tablet_replica.cc:333] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.311658   431 raft_consensus.cc:2243] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.312155   431 raft_consensus.cc:2272] T 46a5186a491b4b87b1ab0c692ba8c7f2 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.314327   431 tablet_replica.cc:333] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.314929   431 raft_consensus.cc:2243] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.315868   431 raft_consensus.cc:2272] T d6ecb24ddae546da9915163c05742a7e P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.318353   431 tablet_replica.cc:333] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.318950   431 raft_consensus.cc:2243] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.319437   431 raft_consensus.cc:2272] T ec71cde09fe64d089f95382f32269db5 P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.321662   431 tablet_replica.cc:333] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.322276   431 raft_consensus.cc:2243] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.322737   431 raft_consensus.cc:2272] T e97c60086a204747ab7fc8b714daa4ea P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.325054   431 tablet_replica.cc:333] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.325661   431 raft_consensus.cc:2243] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.326126   431 raft_consensus.cc:2272] T 756a5f26bcd6404b9b9df57d2c62ee79 P 5a12f1c20c674e7f8242df41a9acf5dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.328333   431 tablet_replica.cc:333] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.328922   431 raft_consensus.cc:2243] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.329910   431 raft_consensus.cc:2272] T 9b930f7fe8fb4ec88f4aa71cd7094c9e P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.332279   431 tablet_replica.cc:333] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.332885   431 raft_consensus.cc:2243] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.333361   431 raft_consensus.cc:2272] T a5f96bbe7d674a5485410f0aa8ec40e6 P 5a12f1c20c674e7f8242df41a9acf5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.335850   431 tablet_replica.cc:333] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.336447   431 raft_consensus.cc:2243] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.337391   431 raft_consensus.cc:2272] T 9887fc1ac0ac404e9f590987c2f0b0be P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.339656   431 tablet_replica.cc:333] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc: stopping tablet replica
I20260709 13:05:24.340265   431 raft_consensus.cc:2243] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:24.340916   431 raft_consensus.cc:2272] T 33e1dc0bb8f54bd2ba02310d2d9d3bdc P 5a12f1c20c674e7f8242df41a9acf5dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.421571   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:05:24.503437   431 master.cc:562] Master@127.0.107.254:34253 shutting down...
I20260709 13:05:24.534272   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:24.534941   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:24.535315   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P c599466512fc46e28facaef651433292: stopping tablet replica
I20260709 13:05:24.580421   431 master.cc:584] Master@127.0.107.254:34253 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (42497 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:05:24.688925   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:38097
I20260709 13:05:24.690722   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:24.698827 11944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:24.698843 11943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:24.699822 11946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:24.701869   431 server_base.cc:1061] running on GCE node
I20260709 13:05:24.703286   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:24.703519   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:24.703743   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602324703702 us; error 0 us; skew 500 ppm
I20260709 13:05:24.704382   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:24.707595   431 webserver.cc:533] Webserver started at http://127.0.107.254:43305/ using document root <none> and password file <none>
I20260709 13:05:24.708194   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:24.708371   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:24.708717   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:24.710194   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/master-0-root/instance:
uuid: "3e3d80e064a8409590edd08fbf125b5d"
format_stamp: "Formatted at 2026-07-09 13:05:24 on dist-test-slave-l8g5"
I20260709 13:05:24.715381   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:05:24.719477 11951 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:24.720291   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.001s
I20260709 13:05:24.720626   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/master-0-root
uuid: "3e3d80e064a8409590edd08fbf125b5d"
format_stamp: "Formatted at 2026-07-09 13:05:24 on dist-test-slave-l8g5"
I20260709 13:05:24.720965   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:24.756368   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:24.758189   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:24.808486   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:38097
I20260709 13:05:24.808591 12002 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:38097 every 8 connection(s)
I20260709 13:05:24.813555 12003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:24.827255 12003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d: Bootstrap starting.
I20260709 13:05:24.833681 12003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:24.839079 12003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d: No bootstrap required, opened a new log
I20260709 13:05:24.841789 12003 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3d80e064a8409590edd08fbf125b5d" member_type: VOTER }
I20260709 13:05:24.842343 12003 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:24.842566 12003 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3d80e064a8409590edd08fbf125b5d, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.843147 12003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3d80e064a8409590edd08fbf125b5d" member_type: VOTER }
I20260709 13:05:24.843582 12003 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:24.843804 12003 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:24.844025 12003 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:24.849431 12003 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3d80e064a8409590edd08fbf125b5d" member_type: VOTER }
I20260709 13:05:24.850055 12003 leader_election.cc:304] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e3d80e064a8409590edd08fbf125b5d; no voters: 
I20260709 13:05:24.851336 12003 leader_election.cc:290] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:24.851912 12006 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:24.853647 12006 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [term 1 LEADER]: Becoming Leader. State: Replica: 3e3d80e064a8409590edd08fbf125b5d, State: Running, Role: LEADER
I20260709 13:05:24.854516 12006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3d80e064a8409590edd08fbf125b5d" member_type: VOTER }
I20260709 13:05:24.854758 12003 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:24.859844 12007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e3d80e064a8409590edd08fbf125b5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3d80e064a8409590edd08fbf125b5d" member_type: VOTER } }
I20260709 13:05:24.860750 12007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:24.860630 12008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e3d80e064a8409590edd08fbf125b5d. Latest consensus state: current_term: 1 leader_uuid: "3e3d80e064a8409590edd08fbf125b5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3d80e064a8409590edd08fbf125b5d" member_type: VOTER } }
I20260709 13:05:24.861671 12008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:24.865366 12015 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:24.871616 12015 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:24.874200   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:24.882454 12015 catalog_manager.cc:1383] Generated new cluster ID: 3e0a6cb4db2f4c0aaf1f3901a3e7d6b7
I20260709 13:05:24.882823 12015 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:24.902835 12015 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:24.904335 12015 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:24.926175 12015 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d: Generated new TSK 0
I20260709 13:05:24.926967 12015 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:24.941864   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:24.949892 12024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:24.951416 12025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:24.954006   431 server_base.cc:1061] running on GCE node
W20260709 13:05:24.953778 12027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:24.955716   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:24.955976   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:24.956108   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602324956096 us; error 0 us; skew 500 ppm
I20260709 13:05:24.956908   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:24.960170   431 webserver.cc:533] Webserver started at http://127.0.107.193:44317/ using document root <none> and password file <none>
I20260709 13:05:24.960724   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:24.960899   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:24.961133   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:24.962370   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-0-root/instance:
uuid: "02778daad9b44c44adc4d26ec80937fe"
format_stamp: "Formatted at 2026-07-09 13:05:24 on dist-test-slave-l8g5"
I20260709 13:05:24.967557   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:05:24.971549 12032 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:24.972647   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:05:24.973146   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-0-root
uuid: "02778daad9b44c44adc4d26ec80937fe"
format_stamp: "Formatted at 2026-07-09 13:05:24 on dist-test-slave-l8g5"
I20260709 13:05:24.973438   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:25.017741   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:25.019483   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:25.021324   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:25.024170   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:25.024456   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:25.024765   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:25.024974   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:25.077756   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:44671
I20260709 13:05:25.077854 12094 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:44671 every 8 connection(s)
I20260709 13:05:25.083684   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:25.092015 12099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:25.093680 12100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:25.098342 12102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:25.100679   431 server_base.cc:1061] running on GCE node
I20260709 13:05:25.102295   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:25.102617   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:25.102843   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602325102821 us; error 0 us; skew 500 ppm
I20260709 13:05:25.103585   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:25.104411 12095 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38097
I20260709 13:05:25.104997 12095 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:25.106185 12095 heartbeater.cc:507] Master 127.0.107.254:38097 requested a full tablet report, sending...
I20260709 13:05:25.107066   431 webserver.cc:533] Webserver started at http://127.0.107.194:33613/ using document root <none> and password file <none>
I20260709 13:05:25.107934   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:25.108208   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:25.108645   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:25.109364 11968 ts_manager.cc:194] Registered new tserver with Master: 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671)
I20260709 13:05:25.110116   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-1-root/instance:
uuid: "5e84ba7b45c74d56a26eab812660f57d"
format_stamp: "Formatted at 2026-07-09 13:05:25 on dist-test-slave-l8g5"
I20260709 13:05:25.111508 11968 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50664
I20260709 13:05:25.116127   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:05:25.120411 12107 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:25.121582   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:05:25.121953   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-1-root
uuid: "5e84ba7b45c74d56a26eab812660f57d"
format_stamp: "Formatted at 2026-07-09 13:05:25 on dist-test-slave-l8g5"
I20260709 13:05:25.122331   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:25.146443   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:25.147843   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:25.149554   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:25.152510   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:25.152714   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:25.153059   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:25.153218   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:25.207775   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:35007
I20260709 13:05:25.207901 12169 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:35007 every 8 connection(s)
I20260709 13:05:25.213533   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:25.222502 12173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:25.223798 12174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:25.226166 12176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:25.228195 12170 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38097
I20260709 13:05:25.228789 12170 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:25.229663   431 server_base.cc:1061] running on GCE node
I20260709 13:05:25.230016 12170 heartbeater.cc:507] Master 127.0.107.254:38097 requested a full tablet report, sending...
I20260709 13:05:25.230835   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:25.231123   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:25.231362   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602325231340 us; error 0 us; skew 500 ppm
I20260709 13:05:25.232127   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:25.232838 11968 ts_manager.cc:194] Registered new tserver with Master: 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:25.235132 11968 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50668
I20260709 13:05:25.235167   431 webserver.cc:533] Webserver started at http://127.0.107.195:41101/ using document root <none> and password file <none>
I20260709 13:05:25.236404   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:25.236775   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:25.237257   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:25.238536   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-2-root/instance:
uuid: "5931067eb88744959a166c3c7ca17be7"
format_stamp: "Formatted at 2026-07-09 13:05:25 on dist-test-slave-l8g5"
I20260709 13:05:25.243674   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:05:25.247882 12181 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:25.249264   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:05:25.249794   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-2-root
uuid: "5931067eb88744959a166c3c7ca17be7"
format_stamp: "Formatted at 2026-07-09 13:05:25 on dist-test-slave-l8g5"
I20260709 13:05:25.250103   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:25.282157   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:25.283699   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:25.285533   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:25.288166   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:25.288434   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:25.288714   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:25.288908   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:25.342308   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:37771
I20260709 13:05:25.342449 12243 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:37771 every 8 connection(s)
I20260709 13:05:25.358770 12244 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38097
I20260709 13:05:25.359308 12244 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:25.360208 12244 heartbeater.cc:507] Master 127.0.107.254:38097 requested a full tablet report, sending...
I20260709 13:05:25.362732 11968 ts_manager.cc:194] Registered new tserver with Master: 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:25.363597   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017525684s
I20260709 13:05:25.364892 11968 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50672
I20260709 13:05:25.393754 11968 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50676:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:25.396862 11968 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:25.478317 12209 tablet_service.cc:1511] Processing CreateTablet for tablet bb97b3caff604bf9922c1f0f93cfce3f (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:05:25.480126 12209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb97b3caff604bf9922c1f0f93cfce3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.484951 12208 tablet_service.cc:1511] Processing CreateTablet for tablet a2079d0db2fd46a1860dfc5f51324ab1 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:05:25.486735 12208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2079d0db2fd46a1860dfc5f51324ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.488989 12060 tablet_service.cc:1511] Processing CreateTablet for tablet bb97b3caff604bf9922c1f0f93cfce3f (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:05:25.489866 12059 tablet_service.cc:1511] Processing CreateTablet for tablet a2079d0db2fd46a1860dfc5f51324ab1 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:05:25.490712 12060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb97b3caff604bf9922c1f0f93cfce3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.499287 12207 tablet_service.cc:1511] Processing CreateTablet for tablet 6fb875cae93c4f4db7949ea8b260f175 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:05:25.501333 12207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fb875cae93c4f4db7949ea8b260f175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.493317 12058 tablet_service.cc:1511] Processing CreateTablet for tablet 6fb875cae93c4f4db7949ea8b260f175 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:05:25.504137 12057 tablet_service.cc:1511] Processing CreateTablet for tablet 28fd3064bea54ab3ba7adad3faddc5ed (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:05:25.505064 12058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fb875cae93c4f4db7949ea8b260f175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.506779 12056 tablet_service.cc:1511] Processing CreateTablet for tablet 3dbdc05a36654d48be71d39e25e90e7d (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:05:25.507638 12135 tablet_service.cc:1511] Processing CreateTablet for tablet bb97b3caff604bf9922c1f0f93cfce3f (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:05:25.508440 12056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dbdc05a36654d48be71d39e25e90e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.509068 12135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb97b3caff604bf9922c1f0f93cfce3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.509398 12059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2079d0db2fd46a1860dfc5f51324ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.512094 12206 tablet_service.cc:1511] Processing CreateTablet for tablet 28fd3064bea54ab3ba7adad3faddc5ed (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:05:25.514807 12205 tablet_service.cc:1511] Processing CreateTablet for tablet 3dbdc05a36654d48be71d39e25e90e7d (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:05:25.516698 12057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28fd3064bea54ab3ba7adad3faddc5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.517894 12206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28fd3064bea54ab3ba7adad3faddc5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.521958 12205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dbdc05a36654d48be71d39e25e90e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.524057 12055 tablet_service.cc:1511] Processing CreateTablet for tablet aa47d580d5d349d4aa183aa2b1c938f4 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:05:25.526758 12133 tablet_service.cc:1511] Processing CreateTablet for tablet 6fb875cae93c4f4db7949ea8b260f175 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:05:25.527199 12131 tablet_service.cc:1511] Processing CreateTablet for tablet 3dbdc05a36654d48be71d39e25e90e7d (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:05:25.528466 12133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fb875cae93c4f4db7949ea8b260f175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.529139 12131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dbdc05a36654d48be71d39e25e90e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.530678 12132 tablet_service.cc:1511] Processing CreateTablet for tablet 28fd3064bea54ab3ba7adad3faddc5ed (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:05:25.532141 12132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28fd3064bea54ab3ba7adad3faddc5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.534628 12134 tablet_service.cc:1511] Processing CreateTablet for tablet a2079d0db2fd46a1860dfc5f51324ab1 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:05:25.536108 12134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2079d0db2fd46a1860dfc5f51324ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.538830 12054 tablet_service.cc:1511] Processing CreateTablet for tablet 712ec9215b1e4546ba0a68b9f52e0eb2 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:05:25.545394 12130 tablet_service.cc:1511] Processing CreateTablet for tablet aa47d580d5d349d4aa183aa2b1c938f4 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:05:25.547040 12130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa47d580d5d349d4aa183aa2b1c938f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.550828 12053 tablet_service.cc:1511] Processing CreateTablet for tablet b50f0c97ddd24bef8b1df9c70c611147 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:05:25.553414 12129 tablet_service.cc:1511] Processing CreateTablet for tablet 712ec9215b1e4546ba0a68b9f52e0eb2 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:05:25.557960 12058 tablet_service.cc:1511] Processing CreateTablet for tablet 7aa91e52f62148258f74d8f19735a8d8 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:05:25.559087 12129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 712ec9215b1e4546ba0a68b9f52e0eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.555610 12203 tablet_service.cc:1511] Processing CreateTablet for tablet 712ec9215b1e4546ba0a68b9f52e0eb2 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:05:25.563066 12203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 712ec9215b1e4546ba0a68b9f52e0eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.564671 12127 tablet_service.cc:1511] Processing CreateTablet for tablet 7aa91e52f62148258f74d8f19735a8d8 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:05:25.566198 12127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aa91e52f62148258f74d8f19735a8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.556900 12128 tablet_service.cc:1511] Processing CreateTablet for tablet b50f0c97ddd24bef8b1df9c70c611147 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:05:25.570868 12052 tablet_service.cc:1511] Processing CreateTablet for tablet e3dfd031362f40daa5a3498b52c0333f (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:05:25.543375 12204 tablet_service.cc:1511] Processing CreateTablet for tablet aa47d580d5d349d4aa183aa2b1c938f4 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:05:25.576527 12128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b50f0c97ddd24bef8b1df9c70c611147. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.578449 12126 tablet_service.cc:1511] Processing CreateTablet for tablet e3dfd031362f40daa5a3498b52c0333f (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:05:25.580668 12126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3dfd031362f40daa5a3498b52c0333f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.589965 12204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa47d580d5d349d4aa183aa2b1c938f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.598938 12055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa47d580d5d349d4aa183aa2b1c938f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.604696 12058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aa91e52f62148258f74d8f19735a8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.621423 12054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 712ec9215b1e4546ba0a68b9f52e0eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.622542 12201 tablet_service.cc:1511] Processing CreateTablet for tablet 7aa91e52f62148258f74d8f19735a8d8 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:05:25.624663 12201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aa91e52f62148258f74d8f19735a8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.626070 12202 tablet_service.cc:1511] Processing CreateTablet for tablet b50f0c97ddd24bef8b1df9c70c611147 (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:05:25.626880 12053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b50f0c97ddd24bef8b1df9c70c611147. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.628140 12202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b50f0c97ddd24bef8b1df9c70c611147. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.633013 12052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3dfd031362f40daa5a3498b52c0333f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.689576 12260 tablet_bootstrap.cc:492] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d: Bootstrap starting.
I20260709 13:05:25.702042 12259 tablet_bootstrap.cc:492] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:25.740103 12259 tablet_bootstrap.cc:654] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.762617 12200 tablet_service.cc:1511] Processing CreateTablet for tablet e3dfd031362f40daa5a3498b52c0333f (DEFAULT_TABLE table=test-workload [id=60e348b374634b469fa050472878adc5]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:05:25.764562 12200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3dfd031362f40daa5a3498b52c0333f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:25.780076 12260 tablet_bootstrap.cc:654] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.840724 12261 tablet_bootstrap.cc:492] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7: Bootstrap starting.
I20260709 13:05:25.842849 12259 tablet_bootstrap.cc:492] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe: No bootstrap required, opened a new log
I20260709 13:05:25.843528 12259 ts_tablet_manager.cc:1403] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.142s	user 0.016s	sys 0.010s
I20260709 13:05:25.855598 12259 raft_consensus.cc:359] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:25.856565 12259 raft_consensus.cc:385] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.856881 12259 raft_consensus.cc:740] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.867498 12260 tablet_bootstrap.cc:492] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d: No bootstrap required, opened a new log
W20260709 13:05:25.871214 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:25.859578 12259 consensus_queue.cc:260] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:25.883530 12261 tablet_bootstrap.cc:654] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.878077 12260 ts_tablet_manager.cc:1403] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d: Time spent bootstrapping tablet: real 0.189s	user 0.019s	sys 0.010s
I20260709 13:05:25.908780 12095 heartbeater.cc:499] Master 127.0.107.254:38097 was elected leader, sending a full tablet report...
I20260709 13:05:25.914047 12260 raft_consensus.cc:359] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:25.915066 12260 raft_consensus.cc:385] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.915421 12260 raft_consensus.cc:740] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.915562 12259 ts_tablet_manager.cc:1434] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.072s	user 0.013s	sys 0.020s
I20260709 13:05:25.917068 12259 tablet_bootstrap.cc:492] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:25.916668 12260 consensus_queue.cc:260] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:25.917290 12261 tablet_bootstrap.cc:492] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7: No bootstrap required, opened a new log
I20260709 13:05:25.918042 12261 ts_tablet_manager.cc:1403] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7: Time spent bootstrapping tablet: real 0.078s	user 0.013s	sys 0.009s
I20260709 13:05:25.921577 12261 raft_consensus.cc:359] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:25.922340 12261 raft_consensus.cc:385] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.922647 12261 raft_consensus.cc:740] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.923647 12261 consensus_queue.cc:260] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:25.926369 12170 heartbeater.cc:499] Master 127.0.107.254:38097 was elected leader, sending a full tablet report...
I20260709 13:05:25.927054 12260 ts_tablet_manager.cc:1434] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.006s
I20260709 13:05:25.928416 12260 tablet_bootstrap.cc:492] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d: Bootstrap starting.
I20260709 13:05:25.928416 12259 tablet_bootstrap.cc:654] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.933059 12244 heartbeater.cc:499] Master 127.0.107.254:38097 was elected leader, sending a full tablet report...
I20260709 13:05:25.934549 12261 ts_tablet_manager.cc:1434] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.002s
I20260709 13:05:25.936084 12261 tablet_bootstrap.cc:492] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7: Bootstrap starting.
I20260709 13:05:25.940985 12260 tablet_bootstrap.cc:654] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.944561 12261 tablet_bootstrap.cc:654] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.955624 12259 tablet_bootstrap.cc:492] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe: No bootstrap required, opened a new log
I20260709 13:05:25.956133 12260 tablet_bootstrap.cc:492] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d: No bootstrap required, opened a new log
I20260709 13:05:25.956401 12259 ts_tablet_manager.cc:1403] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.008s
I20260709 13:05:25.956957 12260 ts_tablet_manager.cc:1403] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.001s
I20260709 13:05:25.959532 12261 tablet_bootstrap.cc:492] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7: No bootstrap required, opened a new log
I20260709 13:05:25.960148 12261 ts_tablet_manager.cc:1403] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.003s
I20260709 13:05:25.960357 12259 raft_consensus.cc:359] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:25.960543 12260 raft_consensus.cc:359] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:25.961246 12259 raft_consensus.cc:385] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.961283 12260 raft_consensus.cc:385] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.961745 12259 raft_consensus.cc:740] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.961776 12260 raft_consensus.cc:740] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Initialized, Role: FOLLOWER
W20260709 13:05:25.962971 12171 tablet.cc:2406] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:05:25.963234 12259 consensus_queue.cc:260] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:25.962963 12260 consensus_queue.cc:260] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:25.966421 12259 ts_tablet_manager.cc:1434] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.004s
I20260709 13:05:25.967450 12260 ts_tablet_manager.cc:1434] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 13:05:25.967682 12259 tablet_bootstrap.cc:492] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:25.968724 12260 tablet_bootstrap.cc:492] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d: Bootstrap starting.
I20260709 13:05:25.963606 12261 raft_consensus.cc:359] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:25.976753 12259 tablet_bootstrap.cc:654] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.981083 12260 tablet_bootstrap.cc:654] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.972558 12261 raft_consensus.cc:385] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.983356 12261 raft_consensus.cc:740] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.984572 12261 consensus_queue.cc:260] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:25.986985 12261 ts_tablet_manager.cc:1434] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:05:25.987483 12260 tablet_bootstrap.cc:492] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d: No bootstrap required, opened a new log
I20260709 13:05:25.988106 12261 tablet_bootstrap.cc:492] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7: Bootstrap starting.
I20260709 13:05:25.988130 12260 ts_tablet_manager.cc:1403] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.000s
I20260709 13:05:25.993299 12260 raft_consensus.cc:359] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:25.994228 12260 raft_consensus.cc:385] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.994575 12260 raft_consensus.cc:740] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.995664 12259 tablet_bootstrap.cc:492] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe: No bootstrap required, opened a new log
I20260709 13:05:25.995589 12260 consensus_queue.cc:260] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:25.996317 12259 ts_tablet_manager.cc:1403] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.002s
I20260709 13:05:25.996555 12261 tablet_bootstrap.cc:654] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.998647 12260 ts_tablet_manager.cc:1434] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:05:25.999915 12260 tablet_bootstrap.cc:492] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d: Bootstrap starting.
I20260709 13:05:26.000384 12259 raft_consensus.cc:359] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.001832 12259 raft_consensus.cc:385] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.002347 12259 raft_consensus.cc:740] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.003888 12259 consensus_queue.cc:260] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.006552 12259 ts_tablet_manager.cc:1434] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:05:26.007730 12259 tablet_bootstrap.cc:492] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:26.008378 12260 tablet_bootstrap.cc:654] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.014832 12261 tablet_bootstrap.cc:492] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7: No bootstrap required, opened a new log
I20260709 13:05:26.015527 12259 tablet_bootstrap.cc:654] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.015571 12261 ts_tablet_manager.cc:1403] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.006s
I20260709 13:05:26.019807 12261 raft_consensus.cc:359] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.021034 12261 raft_consensus.cc:385] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.021116 12260 tablet_bootstrap.cc:492] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d: No bootstrap required, opened a new log
I20260709 13:05:26.021718 12261 raft_consensus.cc:740] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.022017 12260 ts_tablet_manager.cc:1403] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 13:05:26.023231 12261 consensus_queue.cc:260] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.026381 12261 ts_tablet_manager.cc:1434] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:05:26.026481 12260 raft_consensus.cc:359] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.027455 12260 raft_consensus.cc:385] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.027824 12261 tablet_bootstrap.cc:492] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7: Bootstrap starting.
I20260709 13:05:26.027801 12260 raft_consensus.cc:740] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.029438 12260 consensus_queue.cc:260] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.031073 12259 tablet_bootstrap.cc:492] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: No bootstrap required, opened a new log
I20260709 13:05:26.031704 12259 ts_tablet_manager.cc:1403] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.005s
I20260709 13:05:26.032217 12260 ts_tablet_manager.cc:1434] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:05:26.033669 12260 tablet_bootstrap.cc:492] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d: Bootstrap starting.
I20260709 13:05:26.035037 12259 raft_consensus.cc:359] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.036144 12259 raft_consensus.cc:385] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.036521 12259 raft_consensus.cc:740] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.037554 12259 consensus_queue.cc:260] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.040055 12259 ts_tablet_manager.cc:1434] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:26.041244 12259 tablet_bootstrap.cc:492] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:26.042189 12260 tablet_bootstrap.cc:654] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.046900 12261 tablet_bootstrap.cc:654] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.048800 12259 tablet_bootstrap.cc:654] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.054486 12260 tablet_bootstrap.cc:492] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d: No bootstrap required, opened a new log
I20260709 13:05:26.055085 12260 ts_tablet_manager.cc:1403] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 13:05:26.057361 12266 raft_consensus.cc:493] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.058316 12266 raft_consensus.cc:515] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:26.064899 12266 leader_election.cc:290] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771), 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671)
I20260709 13:05:26.068370 12266 raft_consensus.cc:493] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.074061 12266 raft_consensus.cc:515] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.068039 12260 raft_consensus.cc:359] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.078039 12260 raft_consensus.cc:385] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.078497 12260 raft_consensus.cc:740] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.078709 12261 tablet_bootstrap.cc:492] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7: No bootstrap required, opened a new log
I20260709 13:05:26.079685 12261 ts_tablet_manager.cc:1403] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7: Time spent bootstrapping tablet: real 0.052s	user 0.018s	sys 0.002s
I20260709 13:05:26.079684 12260 consensus_queue.cc:260] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.077369 12266 leader_election.cc:290] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.086176 12261 raft_consensus.cc:359] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.094247 12261 raft_consensus.cc:385] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.092482 12260 ts_tablet_manager.cc:1434] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.005s
I20260709 13:05:26.094765 12261 raft_consensus.cc:740] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.096540 12260 tablet_bootstrap.cc:492] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d: Bootstrap starting.
I20260709 13:05:26.096196 12261 consensus_queue.cc:260] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.100183 12261 ts_tablet_manager.cc:1434] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 13:05:26.101877 12261 tablet_bootstrap.cc:492] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7: Bootstrap starting.
I20260709 13:05:26.109256 12259 tablet_bootstrap.cc:492] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe: No bootstrap required, opened a new log
I20260709 13:05:26.110441 12259 ts_tablet_manager.cc:1403] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.069s	user 0.022s	sys 0.030s
I20260709 13:05:26.111151 12261 tablet_bootstrap.cc:654] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.113006 12264 raft_consensus.cc:493] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.113626 12264 raft_consensus.cc:515] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.127852 12284 raft_consensus.cc:493] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.128710 12284 raft_consensus.cc:515] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.130697 12264 leader_election.cc:290] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.131686 12284 leader_election.cc:290] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.133853 12259 raft_consensus.cc:359] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.134431 12260 tablet_bootstrap.cc:654] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.134794 12259 raft_consensus.cc:385] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.135151 12259 raft_consensus.cc:740] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.136049 12259 consensus_queue.cc:260] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.145525 12284 raft_consensus.cc:493] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.146039 12259 ts_tablet_manager.cc:1434] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.004s
I20260709 13:05:26.146207 12284 raft_consensus.cc:515] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.147462 12259 tablet_bootstrap.cc:492] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:26.148667 12284 leader_election.cc:290] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.161214 12261 tablet_bootstrap.cc:492] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7: No bootstrap required, opened a new log
I20260709 13:05:26.164196 12261 ts_tablet_manager.cc:1403] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7: Time spent bootstrapping tablet: real 0.063s	user 0.029s	sys 0.011s
I20260709 13:05:26.168634 12259 tablet_bootstrap.cc:654] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.177340 12261 raft_consensus.cc:359] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.179601 12261 raft_consensus.cc:385] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.180792 12261 raft_consensus.cc:740] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.183207 12261 consensus_queue.cc:260] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.188057 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
I20260709 13:05:26.189356 12070 raft_consensus.cc:2468] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 0.
I20260709 13:05:26.189998 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.191794 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.191447 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
W20260709 13:05:26.193854 12109 leader_election.cc:343] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:26.196246 12109 leader_election.cc:343] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.197557 12110 leader_election.cc:304] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:26.197619 12217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.198102 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.198688 12217 raft_consensus.cc:2468] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 0.
W20260709 13:05:26.199398 12110 leader_election.cc:343] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.199627 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.199952 12110 leader_election.cc:304] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:26.200484 12219 raft_consensus.cc:2468] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 0.
I20260709 13:05:26.201011 12266 raft_consensus.cc:2749] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.201380 12267 raft_consensus.cc:493] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.202093 12267 raft_consensus.cc:515] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.203423 12261 ts_tablet_manager.cc:1434] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7: Time spent starting tablet: real 0.035s	user 0.012s	sys 0.000s
I20260709 13:05:26.177343 12260 tablet_bootstrap.cc:492] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d: No bootstrap required, opened a new log
I20260709 13:05:26.204658 12261 tablet_bootstrap.cc:492] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7: Bootstrap starting.
I20260709 13:05:26.204799 12260 ts_tablet_manager.cc:1403] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d: Time spent bootstrapping tablet: real 0.108s	user 0.017s	sys 0.004s
I20260709 13:05:26.198717 12279 raft_consensus.cc:2804] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.206077 12279 raft_consensus.cc:493] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.206489 12279 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.210206 12260 raft_consensus.cc:359] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.211423 12260 raft_consensus.cc:385] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.211840 12260 raft_consensus.cc:740] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.213447 12261 tablet_bootstrap.cc:654] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.218026 12259 tablet_bootstrap.cc:492] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe: No bootstrap required, opened a new log
I20260709 13:05:26.219105 12259 ts_tablet_manager.cc:1403] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.072s	user 0.014s	sys 0.020s
I20260709 13:05:26.214709 12260 consensus_queue.cc:260] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.198787 12218 raft_consensus.cc:2468] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 0.
I20260709 13:05:26.205235 12267 leader_election.cc:290] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:26.230525 12259 raft_consensus.cc:359] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.231559 12259 raft_consensus.cc:385] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.231967 12259 raft_consensus.cc:740] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.232787 12259 consensus_queue.cc:260] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.237514 12267 raft_consensus.cc:493] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.238864 12267 raft_consensus.cc:515] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.241572 12267 leader_election.cc:290] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:26.237421 12260 ts_tablet_manager.cc:1434] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d: Time spent starting tablet: real 0.032s	user 0.012s	sys 0.000s
I20260709 13:05:26.241464 12034 leader_election.cc:304] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7; no voters: 
I20260709 13:05:26.249266 12034 leader_election.cc:304] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7; no voters: 
I20260709 13:05:26.249869 12284 raft_consensus.cc:2804] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.251158 12284 raft_consensus.cc:493] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.251688 12284 raft_consensus.cc:3060] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.252159 12259 ts_tablet_manager.cc:1434] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:05:26.253545 12259 tablet_bootstrap.cc:492] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:26.263695 12266 raft_consensus.cc:493] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.264720 12266 raft_consensus.cc:515] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.267416 12259 tablet_bootstrap.cc:654] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.269021 12266 leader_election.cc:290] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.271332 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
I20260709 13:05:26.272464 12070 raft_consensus.cc:2468] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 0.
I20260709 13:05:26.274137 12264 raft_consensus.cc:2804] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.275009 12264 raft_consensus.cc:493] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.275888 12264 raft_consensus.cc:3060] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.274425 12110 leader_election.cc:304] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:26.264742 12260 tablet_bootstrap.cc:492] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d: Bootstrap starting.
I20260709 13:05:26.282021 12266 raft_consensus.cc:2804] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.282843 12266 raft_consensus.cc:493] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.283377 12266 raft_consensus.cc:3060] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:26.293704 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37771, user_credentials={real_user=slave}} blocked reactor thread for 44579us
I20260709 13:05:26.297294 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:26.300928 12279 raft_consensus.cc:515] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:26.304706 12261 tablet_bootstrap.cc:492] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7: No bootstrap required, opened a new log
I20260709 13:05:26.305706 12261 ts_tablet_manager.cc:1403] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7: Time spent bootstrapping tablet: real 0.101s	user 0.063s	sys 0.031s
I20260709 13:05:26.308772 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:26.309723 12070 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.311488 12284 raft_consensus.cc:515] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.317989 12279 leader_election.cc:290] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 election: Requested vote from peers 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771), 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671)
I20260709 13:05:26.319061 12279 raft_consensus.cc:493] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.319566 12279 raft_consensus.cc:515] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.322917 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.324147 12218 raft_consensus.cc:2468] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 0.
I20260709 13:05:26.326341 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:26.326617 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
I20260709 13:05:26.327446 12069 raft_consensus.cc:2468] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 0.
I20260709 13:05:26.327672 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:26.328635 12260 tablet_bootstrap.cc:654] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.328653 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.329053 12110 leader_election.cc:304] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:26.330185 12296 raft_consensus.cc:2804] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.330639 12296 raft_consensus.cc:493] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.331013 12296 raft_consensus.cc:3060] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.331511 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:26.333313 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:26.333557 12261 raft_consensus.cc:359] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:26.334456 12261 raft_consensus.cc:385] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.334805 12261 raft_consensus.cc:740] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Initialized, Role: FOLLOWER
W20260709 13:05:26.334897 12109 leader_election.cc:343] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.333930 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:26.335860 12261 consensus_queue.cc:260] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:26.321671 12284 leader_election.cc:290] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Requested vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
W20260709 13:05:26.342242 12109 leader_election.cc:343] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.338706 12259 tablet_bootstrap.cc:492] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: No bootstrap required, opened a new log
I20260709 13:05:26.343004 12284 raft_consensus.cc:493] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.344187 12259 ts_tablet_manager.cc:1403] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.091s	user 0.031s	sys 0.041s
I20260709 13:05:26.344071 12284 raft_consensus.cc:515] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.333139 12070 raft_consensus.cc:2468] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 1.
I20260709 13:05:26.351086 12110 leader_election.cc:304] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:26.351725 12284 leader_election.cc:290] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.352592 12261 ts_tablet_manager.cc:1434] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7: Time spent starting tablet: real 0.046s	user 0.012s	sys 0.000s
I20260709 13:05:26.342689 12218 raft_consensus.cc:3060] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.354131 12261 tablet_bootstrap.cc:492] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7: Bootstrap starting.
I20260709 13:05:26.347470 12266 raft_consensus.cc:515] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.333675 12279 leader_election.cc:290] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.368945 12279 raft_consensus.cc:2804] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.371273 12145 raft_consensus.cc:2393] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02778daad9b44c44adc4d26ec80937fe in current term 1: Already voted for candidate 5e84ba7b45c74d56a26eab812660f57d in this term.
I20260709 13:05:26.373257 12279 raft_consensus.cc:697] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Becoming Leader. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:26.377574 12264 raft_consensus.cc:515] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.386894 12266 leader_election.cc:290] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.388361 12264 leader_election.cc:290] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Requested vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.388382 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:26.389412 12219 raft_consensus.cc:3060] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.375825 12279 consensus_queue.cc:237] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:26.391032 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:26.386693 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "712ec9215b1e4546ba0a68b9f52e0eb2" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
I20260709 13:05:26.392170 12068 raft_consensus.cc:2393] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e84ba7b45c74d56a26eab812660f57d in current term 1: Already voted for candidate 02778daad9b44c44adc4d26ec80937fe in this term.
I20260709 13:05:26.393160 12260 tablet_bootstrap.cc:492] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d: No bootstrap required, opened a new log
W20260709 13:05:26.393589 12184 leader_election.cc:343] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.396150 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "712ec9215b1e4546ba0a68b9f52e0eb2" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:26.396888 12260 ts_tablet_manager.cc:1403] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d: Time spent bootstrapping tablet: real 0.133s	user 0.031s	sys 0.004s
I20260709 13:05:26.397365 12218 raft_consensus.cc:2468] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 1.
I20260709 13:05:26.394645 12261 tablet_bootstrap.cc:654] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.395077 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa47d580d5d349d4aa183aa2b1c938f4" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:26.398000 12034 leader_election.cc:304] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7; no voters: 
I20260709 13:05:26.386642 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa47d580d5d349d4aa183aa2b1c938f4" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
I20260709 13:05:26.396854 12143 raft_consensus.cc:2468] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 0.
I20260709 13:05:26.378278 12259 raft_consensus.cc:359] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.399789 12259 raft_consensus.cc:385] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.399067 12070 raft_consensus.cc:2468] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 0.
I20260709 13:05:26.399935 12183 leader_election.cc:304] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe
I20260709 13:05:26.400130 12259 raft_consensus.cc:740] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.402724 12267 raft_consensus.cc:2804] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.403255 12267 raft_consensus.cc:493] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.403661 12267 raft_consensus.cc:3060] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.403530 12184 leader_election.cc:304] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7; no voters: 
I20260709 13:05:26.393844 12296 raft_consensus.cc:515] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.405500 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa47d580d5d349d4aa183aa2b1c938f4" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:26.408584 12260 raft_consensus.cc:359] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.409552 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:26.400818 12264 raft_consensus.cc:2804] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:26.410820 12183 leader_election.cc:343] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.410617 12296 leader_election.cc:290] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
W20260709 13:05:26.413070 12034 leader_election.cc:343] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.415956 12264 raft_consensus.cc:493] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.416653 12264 raft_consensus.cc:3060] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.416957 12217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa47d580d5d349d4aa183aa2b1c938f4" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.417753 12260 raft_consensus.cc:385] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.414733 12070 raft_consensus.cc:3060] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.420483 12216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:26.403352 12259 consensus_queue.cc:260] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.422981 12215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:26.414964 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:26.424543 12259 ts_tablet_manager.cc:1434] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.079s	user 0.005s	sys 0.005s
I20260709 13:05:26.417779 12217 raft_consensus.cc:2468] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 0.
W20260709 13:05:26.424984 12109 leader_election.cc:343] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.424988 12145 raft_consensus.cc:2393] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 02778daad9b44c44adc4d26ec80937fe in current term 1: Already voted for candidate 5e84ba7b45c74d56a26eab812660f57d in this term.
I20260709 13:05:26.426697 12259 tablet_bootstrap.cc:492] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
W20260709 13:05:26.418251 12034 leader_election.cc:343] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.429682 12260 raft_consensus.cc:740] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Initialized, Role: FOLLOWER
W20260709 13:05:26.430770 12034 leader_election.cc:343] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:26.432240 12034 leader_election.cc:343] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.432044 12260 consensus_queue.cc:260] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.435614 12267 raft_consensus.cc:515] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.448107 12261 tablet_bootstrap.cc:492] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7: No bootstrap required, opened a new log
I20260709 13:05:26.449224 12261 ts_tablet_manager.cc:1403] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7: Time spent bootstrapping tablet: real 0.095s	user 0.012s	sys 0.009s
I20260709 13:05:26.448107 11967 catalog_manager.cc:5697] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5e84ba7b45c74d56a26eab812660f57d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.450537 12260 ts_tablet_manager.cc:1434] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.001s
I20260709 13:05:26.436480 12301 raft_consensus.cc:2804] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.433173 12034 leader_election.cc:304] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7; no voters: 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:26.452152 12260 tablet_bootstrap.cc:492] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d: Bootstrap starting.
I20260709 13:05:26.454077 12297 raft_consensus.cc:2804] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.454967 12297 raft_consensus.cc:697] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 1 LEADER]: Becoming Leader. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Running, Role: LEADER
I20260709 13:05:26.455688 12301 raft_consensus.cc:493] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.456177 12301 raft_consensus.cc:3060] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.458019 12279 raft_consensus.cc:493] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.441319 12264 raft_consensus.cc:515] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.458626 12279 raft_consensus.cc:515] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.457463 12070 raft_consensus.cc:2468] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 1.
I20260709 13:05:26.453213 12034 leader_election.cc:304] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7; no voters: 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:26.456281 12297 consensus_queue.cc:237] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.458068 12267 leader_election.cc:290] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:26.461194 12264 leader_election.cc:290] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Requested vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.462524 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbdc05a36654d48be71d39e25e90e7d" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
I20260709 13:05:26.466403 12068 raft_consensus.cc:2468] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 0.
I20260709 13:05:26.466512 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "712ec9215b1e4546ba0a68b9f52e0eb2" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:26.414162 12219 raft_consensus.cc:2468] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 1.
I20260709 13:05:26.447744 12259 tablet_bootstrap.cc:654] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.462949 12110 leader_election.cc:304] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:26.468261 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbdc05a36654d48be71d39e25e90e7d" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.464195 12284 raft_consensus.cc:2804] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.469429 12296 raft_consensus.cc:2804] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.469794 12284 raft_consensus.cc:493] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.461032 12279 leader_election.cc:290] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.470942 12109 leader_election.cc:304] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe
I20260709 13:05:26.470371 12284 raft_consensus.cc:3060] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.471812 12110 leader_election.cc:304] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:26.472967 12266 raft_consensus.cc:2804] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.473493 12266 raft_consensus.cc:697] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Becoming Leader. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:26.473758 12299 raft_consensus.cc:2804] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.474177 12299 raft_consensus.cc:493] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.474570 12299 raft_consensus.cc:3060] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.474434 12266 consensus_queue.cc:237] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.474916 12034 leader_election.cc:304] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe; no voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:26.475816 12264 raft_consensus.cc:2749] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.467058 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "712ec9215b1e4546ba0a68b9f52e0eb2" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
W20260709 13:05:26.481348 12109 leader_election.cc:343] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.470386 12296 raft_consensus.cc:697] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Becoming Leader. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:26.477205 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
W20260709 13:05:26.516415 12034 leader_election.cc:343] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:26.481098 12184 leader_election.cc:343] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.489156 12267 raft_consensus.cc:493] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.510293 12260 tablet_bootstrap.cc:654] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.519454 12279 raft_consensus.cc:493] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.519553 12296 consensus_queue.cc:237] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.481129 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:26.510438 12145 raft_consensus.cc:3060] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.478861 12261 raft_consensus.cc:359] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.525934 12261 raft_consensus.cc:385] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.524191 12279 raft_consensus.cc:515] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.526278 12261 raft_consensus.cc:740] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.525434 12218 raft_consensus.cc:3060] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.527127 12261 consensus_queue.cc:260] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.532528 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
W20260709 13:05:26.535130 12110 leader_election.cc:343] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.527966 12279 leader_election.cc:290] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.524125 12267 raft_consensus.cc:515] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:26.544673 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
I20260709 13:05:26.546066 12068 raft_consensus.cc:2393] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5931067eb88744959a166c3c7ca17be7 in current term 1: Already voted for candidate 5e84ba7b45c74d56a26eab812660f57d in this term.
I20260709 13:05:26.546700 12218 raft_consensus.cc:2468] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 1.
I20260709 13:05:26.547107 12284 raft_consensus.cc:515] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.548246 12034 leader_election.cc:304] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7; no voters: 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:26.548991 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:26.549676 12297 raft_consensus.cc:2804] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.550213 12297 raft_consensus.cc:697] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 1 LEADER]: Becoming Leader. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Running, Role: LEADER
I20260709 13:05:26.550526 12299 raft_consensus.cc:515] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.551709 12267 leader_election.cc:290] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671)
I20260709 13:05:26.553542 12297 consensus_queue.cc:237] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.556680 12261 ts_tablet_manager.cc:1434] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7: Time spent starting tablet: real 0.107s	user 0.017s	sys 0.024s
I20260709 13:05:26.557894 12261 tablet_bootstrap.cc:492] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7: Bootstrap starting.
I20260709 13:05:26.560680 12260 tablet_bootstrap.cc:492] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d: No bootstrap required, opened a new log
I20260709 13:05:26.554363 12183 leader_election.cc:304] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7; no voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:26.521811 11966 catalog_manager.cc:5697] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 02778daad9b44c44adc4d26ec80937fe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "02778daad9b44c44adc4d26ec80937fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.561892 12260 ts_tablet_manager.cc:1403] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d: Time spent bootstrapping tablet: real 0.110s	user 0.012s	sys 0.017s
I20260709 13:05:26.564818 12261 tablet_bootstrap.cc:654] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.565130 12299 leader_election.cc:290] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.554236 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.567462 12260 raft_consensus.cc:359] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.549350 12284 leader_election.cc:290] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Requested vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.568506 12260 raft_consensus.cc:385] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.560462 12145 raft_consensus.cc:2468] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 1.
I20260709 13:05:26.568933 12260 raft_consensus.cc:740] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.571645 12260 consensus_queue.cc:260] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.574720 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbdc05a36654d48be71d39e25e90e7d" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:26.557098 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbdc05a36654d48be71d39e25e90e7d" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:26.576617 12068 raft_consensus.cc:3060] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:26.577132 12109 leader_election.cc:343] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.571768 12267 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.582207 12260 ts_tablet_manager.cc:1434] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 13:05:26.588033 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa47d580d5d349d4aa183aa2b1c938f4" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:26.588899 12143 raft_consensus.cc:3060] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.593132 12267 raft_consensus.cc:2749] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.595122 11966 catalog_manager.cc:5697] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 02778daad9b44c44adc4d26ec80937fe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "02778daad9b44c44adc4d26ec80937fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:26.598299 12109 leader_election.cc:343] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.598934 12109 leader_election.cc:304] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:26.599506 12260 tablet_bootstrap.cc:492] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d: Bootstrap starting.
I20260709 13:05:26.603129 11967 catalog_manager.cc:5697] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5e84ba7b45c74d56a26eab812660f57d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.605686 12299 raft_consensus.cc:2749] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.611747 12259 tablet_bootstrap.cc:492] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe: No bootstrap required, opened a new log
I20260709 13:05:26.612995 12259 ts_tablet_manager.cc:1403] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.187s	user 0.021s	sys 0.013s
I20260709 13:05:26.616420 12259 raft_consensus.cc:359] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.617813 12259 raft_consensus.cc:385] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.618216 12259 raft_consensus.cc:740] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.619549 12259 consensus_queue.cc:260] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.623432 12259 ts_tablet_manager.cc:1434] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:05:26.625053 12259 tablet_bootstrap.cc:492] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
W20260709 13:05:26.628504 12183 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:35007, user_credentials={real_user=slave}} blocked reactor thread for 75884.9us
I20260709 13:05:26.605381 11967 catalog_manager.cc:5697] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5e84ba7b45c74d56a26eab812660f57d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.630366 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa47d580d5d349d4aa183aa2b1c938f4" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
W20260709 13:05:26.630246 12312 log.cc:927] Time spent T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:05:26.631256 12301 raft_consensus.cc:515] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.632848 12218 raft_consensus.cc:2393] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 02778daad9b44c44adc4d26ec80937fe in current term 1: Already voted for candidate 5931067eb88744959a166c3c7ca17be7 in this term.
I20260709 13:05:26.634363 12068 raft_consensus.cc:2468] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 1.
I20260709 13:05:26.634902 12183 leader_election.cc:304] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe
I20260709 13:05:26.635908 12110 leader_election.cc:304] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:26.636389 12260 tablet_bootstrap.cc:654] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.637446 12299 raft_consensus.cc:2804] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.638101 12299 raft_consensus.cc:697] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Becoming Leader. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:26.638962 12299 consensus_queue.cc:237] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.640157 12143 raft_consensus.cc:2468] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 1.
I20260709 13:05:26.641651 12034 leader_election.cc:304] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:26.643025 12284 raft_consensus.cc:2804] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.643586 12284 raft_consensus.cc:697] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 1 LEADER]: Becoming Leader. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Running, Role: LEADER
I20260709 13:05:26.643618 12314 raft_consensus.cc:2804] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.644620 12314 raft_consensus.cc:697] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 1 LEADER]: Becoming Leader. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:26.644531 12284 consensus_queue.cc:237] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.646744 12314 consensus_queue.cc:237] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.656682 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa47d580d5d349d4aa183aa2b1c938f4" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:26.659516 12259 tablet_bootstrap.cc:654] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.661659 12260 tablet_bootstrap.cc:492] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d: No bootstrap required, opened a new log
I20260709 13:05:26.662298 12260 ts_tablet_manager.cc:1403] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d: Time spent bootstrapping tablet: real 0.063s	user 0.020s	sys 0.001s
I20260709 13:05:26.667282 12260 raft_consensus.cc:359] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.668226 12260 raft_consensus.cc:385] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.668503 12260 raft_consensus.cc:740] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.671639 12261 tablet_bootstrap.cc:492] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7: No bootstrap required, opened a new log
I20260709 13:05:26.672294 12261 ts_tablet_manager.cc:1403] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7: Time spent bootstrapping tablet: real 0.115s	user 0.015s	sys 0.029s
I20260709 13:05:26.673556 12301 leader_election.cc:290] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:26.669216 12260 consensus_queue.cc:260] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.675571 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa47d580d5d349d4aa183aa2b1c938f4" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:26.676908 12143 raft_consensus.cc:2393] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5931067eb88744959a166c3c7ca17be7 in current term 1: Already voted for candidate 02778daad9b44c44adc4d26ec80937fe in this term.
I20260709 13:05:26.677991 12260 ts_tablet_manager.cc:1434] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 13:05:26.677784 11965 catalog_manager.cc:5697] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5931067eb88744959a166c3c7ca17be7 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "5931067eb88744959a166c3c7ca17be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:26.679198 12260 tablet_bootstrap.cc:492] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d: Bootstrap starting.
I20260709 13:05:26.683281 12183 leader_election.cc:304] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7; no voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:26.684751 12314 raft_consensus.cc:2749] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.686079 12261 raft_consensus.cc:359] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.687049 12261 raft_consensus.cc:385] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.687423 12261 raft_consensus.cc:740] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.687685 12260 tablet_bootstrap.cc:654] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.688318 12261 consensus_queue.cc:260] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.685235 11966 catalog_manager.cc:5697] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5e84ba7b45c74d56a26eab812660f57d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.690765 12259 tablet_bootstrap.cc:492] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: No bootstrap required, opened a new log
I20260709 13:05:26.691387 12259 ts_tablet_manager.cc:1403] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.000s
I20260709 13:05:26.694880 12259 raft_consensus.cc:359] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.695636 12259 raft_consensus.cc:385] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.695907 12259 raft_consensus.cc:740] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.696941 12261 ts_tablet_manager.cc:1434] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.001s
I20260709 13:05:26.700006 12261 tablet_bootstrap.cc:492] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7: Bootstrap starting.
I20260709 13:05:26.703956 12260 tablet_bootstrap.cc:492] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d: No bootstrap required, opened a new log
I20260709 13:05:26.704723 12260 ts_tablet_manager.cc:1403] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.004s
I20260709 13:05:26.697865 12259 consensus_queue.cc:260] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.706959 12259 ts_tablet_manager.cc:1434] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:05:26.707763 12261 tablet_bootstrap.cc:654] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.708052 12259 tablet_bootstrap.cc:492] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:26.708473 12260 raft_consensus.cc:359] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.709374 12260 raft_consensus.cc:385] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.709765 12260 raft_consensus.cc:740] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.710728 12260 consensus_queue.cc:260] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.713762 12260 ts_tablet_manager.cc:1434] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:05:26.726742 11967 catalog_manager.cc:5697] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 02778daad9b44c44adc4d26ec80937fe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "02778daad9b44c44adc4d26ec80937fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.738399 12261 tablet_bootstrap.cc:492] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7: No bootstrap required, opened a new log
I20260709 13:05:26.739084 12261 ts_tablet_manager.cc:1403] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7: Time spent bootstrapping tablet: real 0.039s	user 0.017s	sys 0.016s
I20260709 13:05:26.739925 12259 tablet_bootstrap.cc:654] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.742408 12261 raft_consensus.cc:359] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.743382 12261 raft_consensus.cc:385] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.743637 12261 raft_consensus.cc:740] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.744403 12261 consensus_queue.cc:260] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.745582 12259 tablet_bootstrap.cc:492] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe: No bootstrap required, opened a new log
I20260709 13:05:26.746129 12259 ts_tablet_manager.cc:1403] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.000s
I20260709 13:05:26.747084 12261 ts_tablet_manager.cc:1434] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:26.748327 12261 tablet_bootstrap.cc:492] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7: Bootstrap starting.
I20260709 13:05:26.749207 12259 raft_consensus.cc:359] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.750007 12259 raft_consensus.cc:385] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.750291 12259 raft_consensus.cc:740] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.750972 12259 consensus_queue.cc:260] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.753445 12259 ts_tablet_manager.cc:1434] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:26.755587 12261 tablet_bootstrap.cc:654] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.763278 12261 tablet_bootstrap.cc:492] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7: No bootstrap required, opened a new log
I20260709 13:05:26.763988 12261 ts_tablet_manager.cc:1403] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:05:26.767699 12261 raft_consensus.cc:359] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.768558 12261 raft_consensus.cc:385] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.768853 12261 raft_consensus.cc:740] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.769981 12261 consensus_queue.cc:260] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.771713 12261 ts_tablet_manager.cc:1434] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:05:26.847215 12299 consensus_queue.cc:1048] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:26.861804 12299 consensus_queue.cc:1048] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.874185 12299 raft_consensus.cc:493] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.874795 12299 raft_consensus.cc:515] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
W20260709 13:05:26.877110 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:26.877137 12299 leader_election.cc:290] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.878445 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
I20260709 13:05:26.879024 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.879853 12218 raft_consensus.cc:2393] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e84ba7b45c74d56a26eab812660f57d in current term 1: Already voted for candidate 02778daad9b44c44adc4d26ec80937fe in this term.
I20260709 13:05:26.880963 12109 leader_election.cc:304] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:26.881860 12299 raft_consensus.cc:3060] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.883530 12284 raft_consensus.cc:493] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.884292 12284 raft_consensus.cc:515] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.886612 12284 leader_election.cc:290] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.889222 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:26.889217 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.890278 12143 raft_consensus.cc:2468] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 0.
I20260709 13:05:26.890528 12218 raft_consensus.cc:2468] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 0.
I20260709 13:05:26.891587 12299 raft_consensus.cc:2749] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.891933 12034 leader_election.cc:304] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:26.892763 12284 raft_consensus.cc:2804] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.893143 12284 raft_consensus.cc:493] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.893517 12284 raft_consensus.cc:3060] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.895041 12299 raft_consensus.cc:493] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.895610 12299 raft_consensus.cc:515] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.898031 12299 leader_election.cc:290] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.899114 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
I20260709 13:05:26.899365 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.900110 12218 raft_consensus.cc:2393] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e84ba7b45c74d56a26eab812660f57d in current term 1: Already voted for candidate 02778daad9b44c44adc4d26ec80937fe in this term.
I20260709 13:05:26.901110 12109 leader_election.cc:304] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:26.901945 12299 raft_consensus.cc:3060] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.902079 12284 raft_consensus.cc:515] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.904145 12284 leader_election.cc:290] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Requested vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.905064 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:26.905287 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:26.905853 12143 raft_consensus.cc:3060] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.905972 12218 raft_consensus.cc:3060] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.910758 12299 raft_consensus.cc:2749] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.914362 12218 raft_consensus.cc:2468] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 1.
I20260709 13:05:26.915957 12034 leader_election.cc:304] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7; no voters: 
I20260709 13:05:26.916813 12143 raft_consensus.cc:2468] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 1.
I20260709 13:05:26.917224 12284 raft_consensus.cc:2804] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.917886 12284 raft_consensus.cc:697] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 1 LEADER]: Becoming Leader. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Running, Role: LEADER
I20260709 13:05:26.918838 12284 consensus_queue.cc:237] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.929392 12299 consensus_queue.cc:1048] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.929345 11965 catalog_manager.cc:5697] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 02778daad9b44c44adc4d26ec80937fe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "02778daad9b44c44adc4d26ec80937fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.946943 12299 consensus_queue.cc:1048] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.955169 12284 raft_consensus.cc:493] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.955808 12284 raft_consensus.cc:515] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.959771 12284 leader_election.cc:290] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.960291 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:26.960976 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:26.961138 12143 raft_consensus.cc:2468] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 0.
I20260709 13:05:26.961695 12218 raft_consensus.cc:2468] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 0.
I20260709 13:05:26.962649 12034 leader_election.cc:304] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:26.963615 12284 raft_consensus.cc:2804] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.964023 12284 raft_consensus.cc:493] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.964377 12284 raft_consensus.cc:3060] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.972700 12284 raft_consensus.cc:515] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.974768 12284 leader_election.cc:290] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Requested vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:26.976068 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:26.976511 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147" candidate_uuid: "02778daad9b44c44adc4d26ec80937fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:26.976913 12143 raft_consensus.cc:3060] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.977460 12218 raft_consensus.cc:3060] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.984222 12301 raft_consensus.cc:493] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.985383 12301 raft_consensus.cc:515] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.987689 12284 consensus_queue.cc:1048] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.988179 12143 raft_consensus.cc:2468] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 1.
I20260709 13:05:26.989116 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
I20260709 13:05:26.990106 12068 raft_consensus.cc:2393] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5931067eb88744959a166c3c7ca17be7 in current term 1: Already voted for candidate 5e84ba7b45c74d56a26eab812660f57d in this term.
I20260709 13:05:26.990005 12034 leader_election.cc:304] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:26.990475 12301 leader_election.cc:290] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:26.991178 12297 raft_consensus.cc:2804] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.991828 12297 raft_consensus.cc:697] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 1 LEADER]: Becoming Leader. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Running, Role: LEADER
I20260709 13:05:26.992681 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:26.992833 12297 consensus_queue.cc:237] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:26.993697 12218 raft_consensus.cc:2468] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02778daad9b44c44adc4d26ec80937fe in term 1.
I20260709 13:05:26.994717 12183 leader_election.cc:304] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7; no voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:26.996512 12301 raft_consensus.cc:3060] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.010118 12297 consensus_queue.cc:1048] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:27.012892 11967 catalog_manager.cc:5697] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 02778daad9b44c44adc4d26ec80937fe (127.0.107.193). New cstate: current_term: 1 leader_uuid: "02778daad9b44c44adc4d26ec80937fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.030673 12301 raft_consensus.cc:2749] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:27.037710 12299 consensus_queue.cc:1048] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:27.040225 12068 raft_consensus.cc:3060] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.051878 12301 consensus_queue.cc:1048] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:27.053403 12314 consensus_queue.cc:1048] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:27.055188 12264 consensus_queue.cc:1048] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:27.072719 12279 consensus_queue.cc:1048] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:27.100214 12264 consensus_queue.cc:1048] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:27.116091 12314 raft_consensus.cc:493] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.116753 12314 raft_consensus.cc:515] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:27.125654 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbdc05a36654d48be71d39e25e90e7d" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02778daad9b44c44adc4d26ec80937fe" is_pre_election: true
I20260709 13:05:27.125923 12297 consensus_queue.cc:1048] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:27.126641 12070 raft_consensus.cc:2393] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5931067eb88744959a166c3c7ca17be7 in current term 1: Already voted for candidate 5e84ba7b45c74d56a26eab812660f57d in this term.
I20260709 13:05:27.127727 12297 consensus_queue.cc:1048] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:27.139492 12314 leader_election.cc:290] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:27.141404 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbdc05a36654d48be71d39e25e90e7d" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:27.143564 12183 leader_election.cc:304] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7; no voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:27.144985 12314 raft_consensus.cc:3060] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.165750 12314 raft_consensus.cc:2749] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:27.250262 12296 consensus_queue.cc:1048] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:27.258338   431 ts_manager.cc:295] Set tserver state for 02778daad9b44c44adc4d26ec80937fe to MAINTENANCE_MODE
I20260709 13:05:27.259598   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:05:27.266465 12110 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 180 similar messages]
I20260709 13:05:27.268157 12296 consensus_queue.cc:1048] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:27.298420 12110 consensus_peers.cc:597] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.305091 12110 consensus_peers.cc:597] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.334461   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:27.334292 12297 consensus_queue.cc:1048] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:27.338699   431 tablet_replica.cc:333] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: stopping tablet replica
I20260709 13:05:27.339871   431 raft_consensus.cc:2243] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.340839   431 pending_rounds.cc:70] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Trying to abort 1 pending ops.
I20260709 13:05:27.341112   431 pending_rounds.cc:77] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635131388432384 op_type: NO_OP noop_request { }
I20260709 13:05:27.341642   431 raft_consensus.cc:2889] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:27.342005   431 raft_consensus.cc:2272] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.345755   431 tablet_replica.cc:333] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe: stopping tablet replica
I20260709 13:05:27.346470   431 raft_consensus.cc:2243] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.347910   431 raft_consensus.cc:2272] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.352175   431 tablet_replica.cc:333] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: stopping tablet replica
I20260709 13:05:27.353201   431 raft_consensus.cc:2243] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.353710   431 pending_rounds.cc:70] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: Trying to abort 1 pending ops.
I20260709 13:05:27.353914   431 pending_rounds.cc:77] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635129925582848 op_type: NO_OP noop_request { }
I20260709 13:05:27.354223   431 raft_consensus.cc:2889] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:27.354595   431 raft_consensus.cc:2272] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.358379   431 tablet_replica.cc:333] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe: stopping tablet replica
I20260709 13:05:27.359161   431 raft_consensus.cc:2243] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.359714   431 raft_consensus.cc:2272] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.363399   431 tablet_replica.cc:333] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe: stopping tablet replica
I20260709 13:05:27.364099   431 raft_consensus.cc:2243] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.365134   431 raft_consensus.cc:2272] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.368027   431 tablet_replica.cc:333] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: stopping tablet replica
I20260709 13:05:27.368686   431 raft_consensus.cc:2243] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.369948   431 raft_consensus.cc:2272] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.373391   431 tablet_replica.cc:333] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe: stopping tablet replica
I20260709 13:05:27.374255   431 raft_consensus.cc:2243] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:27.375506   431 raft_consensus.cc:2272] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.378289   431 tablet_replica.cc:333] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe: stopping tablet replica
I20260709 13:05:27.378973   431 raft_consensus.cc:2243] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.379509   431 raft_consensus.cc:2272] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.382777   431 tablet_replica.cc:333] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe: stopping tablet replica
I20260709 13:05:27.383443   431 raft_consensus.cc:2243] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.384066   431 raft_consensus.cc:2272] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.387544   431 tablet_replica.cc:333] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe: stopping tablet replica
I20260709 13:05:27.388144   431 raft_consensus.cc:2243] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.388645   431 raft_consensus.cc:2272] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.425781   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:05:27.505154 12110 consensus_peers.cc:597] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.555435 12110 consensus_peers.cc:597] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.687899 12184 consensus_peers.cc:597] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.878705 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:28.508396 12354 raft_consensus.cc:493] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
I20260709 13:05:28.509125 12354 raft_consensus.cc:515] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.511507 12354 leader_election.cc:290] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:28.512827 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:28.514815 12279 raft_consensus.cc:493] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
I20260709 13:05:28.515532 12279 raft_consensus.cc:515] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
W20260709 13:05:28.516530 12184 leader_election.cc:336] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.517103 12184 leader_election.cc:304] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7; no voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:28.517860 12354 raft_consensus.cc:2749] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:28.518419 12279 leader_election.cc:290] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:28.519797 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:28.520539 12218 raft_consensus.cc:2468] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 1.
I20260709 13:05:28.522146 12109 leader_election.cc:304] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:28.523124 12279 raft_consensus.cc:2804] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:28.523764 12279 raft_consensus.cc:493] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
W20260709 13:05:28.523818 12110 leader_election.cc:336] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.524287 12279 raft_consensus.cc:3060] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:28.530920 12279 raft_consensus.cc:515] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.532948 12279 leader_election.cc:290] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:28.534299 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:28.534950 12218 raft_consensus.cc:3060] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:28.537124 12110 leader_election.cc:336] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.541791 12218 raft_consensus.cc:2468] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 2.
I20260709 13:05:28.543794 12109 leader_election.cc:304] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe
I20260709 13:05:28.545048 12279 raft_consensus.cc:2804] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:28.545632 12279 raft_consensus.cc:697] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 2 LEADER]: Becoming Leader. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:28.546567 12279 consensus_queue.cc:237] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.556283 11965 catalog_manager.cc:5697] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d reported cstate change: term changed from 1 to 2, leader changed from 02778daad9b44c44adc4d26ec80937fe (127.0.107.193) to 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "5e84ba7b45c74d56a26eab812660f57d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.610710 12279 raft_consensus.cc:493] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
I20260709 13:05:28.611337 12279 raft_consensus.cc:515] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.613444 12279 leader_election.cc:290] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:28.614796 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa47d580d5d349d4aa183aa2b1c938f4" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
W20260709 13:05:28.618126 12110 leader_election.cc:336] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.618677 12110 leader_election.cc:304] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:28.619647 12279 raft_consensus.cc:2749] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:28.671366 12314 raft_consensus.cc:493] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
I20260709 13:05:28.674968 12279 raft_consensus.cc:493] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
I20260709 13:05:28.674811 12314 raft_consensus.cc:515] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.676442 12279 raft_consensus.cc:515] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.677424 12354 raft_consensus.cc:493] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
I20260709 13:05:28.678786 12354 raft_consensus.cc:515] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.684038 12279 leader_election.cc:290] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:28.684669 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7" is_pre_election: true
I20260709 13:05:28.686187 12354 leader_election.cc:290] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:28.686812 12218 raft_consensus.cc:2468] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 1.
I20260709 13:05:28.686911 12314 leader_election.cc:290] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:28.689636 12109 leader_election.cc:304] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:28.689713 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:28.690568 12144 raft_consensus.cc:2468] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 1.
I20260709 13:05:28.691555 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa47d580d5d349d4aa183aa2b1c938f4" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:28.692560 12183 leader_election.cc:304] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:28.692677 12145 raft_consensus.cc:2468] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 1.
W20260709 13:05:28.693783 12110 leader_election.cc:336] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.693799 12279 raft_consensus.cc:2804] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:05:28.694156 12184 leader_election.cc:336] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.693904 12314 raft_consensus.cc:2804] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:28.695313 12314 raft_consensus.cc:493] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
I20260709 13:05:28.695694 12183 leader_election.cc:304] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:28.695840 12314 raft_consensus.cc:3060] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:28.695894 12279 raft_consensus.cc:493] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
I20260709 13:05:28.696597 12279 raft_consensus.cc:3060] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:28.696767 12354 raft_consensus.cc:2804] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:05:28.696791 12184 leader_election.cc:336] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.697495 12354 raft_consensus.cc:493] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
I20260709 13:05:28.698004 12354 raft_consensus.cc:3060] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:28.709054 12279 raft_consensus.cc:515] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.711501 12354 raft_consensus.cc:515] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.711438 12314 raft_consensus.cc:515] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.712086 12279 leader_election.cc:290] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:28.713726 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:28.714139 12314 leader_election.cc:290] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:28.714691 12218 raft_consensus.cc:2393] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e84ba7b45c74d56a26eab812660f57d in current term 2: Already voted for candidate 5931067eb88744959a166c3c7ca17be7 in this term.
I20260709 13:05:28.716158 12354 leader_election.cc:290] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:28.715987 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa47d580d5d349d4aa183aa2b1c938f4" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:28.716818 12145 raft_consensus.cc:3060] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:28.716928 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:28.717937 12144 raft_consensus.cc:2393] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5931067eb88744959a166c3c7ca17be7 in current term 2: Already voted for candidate 5e84ba7b45c74d56a26eab812660f57d in this term.
W20260709 13:05:28.719048 12110 leader_election.cc:336] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.719640 12109 leader_election.cc:304] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:28.720769 12279 raft_consensus.cc:2749] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:05:28.723040 12184 leader_election.cc:336] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
W20260709 13:05:28.723650 12184 leader_election.cc:336] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.724148 12184 leader_election.cc:304] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7; no voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:28.724996 12354 raft_consensus.cc:2749] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:28.728659 12145 raft_consensus.cc:2468] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 2.
I20260709 13:05:28.730232 12183 leader_election.cc:304] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe
I20260709 13:05:28.731099 12354 raft_consensus.cc:2804] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:28.731598 12354 raft_consensus.cc:697] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Becoming Leader. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:28.733037 12354 consensus_queue.cc:237] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.743547 11965 catalog_manager.cc:5697] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 reported cstate change: term changed from 1 to 2, leader changed from 02778daad9b44c44adc4d26ec80937fe (127.0.107.193) to 5931067eb88744959a166c3c7ca17be7 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "5931067eb88744959a166c3c7ca17be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.745826 12354 raft_consensus.cc:493] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.746551 12354 raft_consensus.cc:515] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.748870 12354 leader_election.cc:290] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:28.750510 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:28.751703 12145 raft_consensus.cc:2468] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 1.
I20260709 13:05:28.753235 12183 leader_election.cc:304] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:28.754184 12354 raft_consensus.cc:2804] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:05:28.754508 12184 leader_election.cc:336] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.754653 12354 raft_consensus.cc:493] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.755290 12354 raft_consensus.cc:3060] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:28.763199 12354 raft_consensus.cc:515] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.765095 12354 leader_election.cc:290] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:28.766701 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:28.767452 12145 raft_consensus.cc:3060] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:28.769573 12184 leader_election.cc:336] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.774931 12145 raft_consensus.cc:2468] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 2.
I20260709 13:05:28.776587 12183 leader_election.cc:304] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe
I20260709 13:05:28.777572 12354 raft_consensus.cc:2804] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:28.778209 12354 raft_consensus.cc:697] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Becoming Leader. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:28.779285 12354 consensus_queue.cc:237] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.788578 11965 catalog_manager.cc:5697] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 reported cstate change: term changed from 1 to 2, leader changed from 02778daad9b44c44adc4d26ec80937fe (127.0.107.193) to 5931067eb88744959a166c3c7ca17be7 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "5931067eb88744959a166c3c7ca17be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.850687 12354 raft_consensus.cc:493] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
I20260709 13:05:28.851432 12354 raft_consensus.cc:515] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.853767 12354 leader_election.cc:290] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:28.855005 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:28.856137 12145 raft_consensus.cc:2468] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 1.
I20260709 13:05:28.857475 12183 leader_election.cc:304] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:28.858510 12354 raft_consensus.cc:2804] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:05:28.858528 12184 leader_election.cc:336] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.859056 12354 raft_consensus.cc:493] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 02778daad9b44c44adc4d26ec80937fe)
I20260709 13:05:28.859501 12354 raft_consensus.cc:3060] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:28.866326 12354 raft_consensus.cc:515] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.868239 12354 leader_election.cc:290] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:28.869750 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:28.870574 12145 raft_consensus.cc:3060] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:28.872884 12184 leader_election.cc:336] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:28.878285 12145 raft_consensus.cc:2468] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 2.
I20260709 13:05:28.879904 12183 leader_election.cc:304] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe
W20260709 13:05:28.880565 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:28.881197 12354 raft_consensus.cc:2804] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:28.881752 12354 raft_consensus.cc:697] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Becoming Leader. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:28.882884 12354 consensus_queue.cc:237] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:28.892822 11965 catalog_manager.cc:5697] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 reported cstate change: term changed from 1 to 2, leader changed from 02778daad9b44c44adc4d26ec80937fe (127.0.107.193) to 5931067eb88744959a166c3c7ca17be7 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "5931067eb88744959a166c3c7ca17be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.938854 12218 raft_consensus.cc:1275] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Refusing update from remote peer 5e84ba7b45c74d56a26eab812660f57d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:28.940544 12279 consensus_queue.cc:1048] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:28.961064 12110 consensus_peers.cc:597] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:29.207370 12145 raft_consensus.cc:1275] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:29.208967 12354 consensus_queue.cc:1048] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:29.226438 12184 consensus_peers.cc:597] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:29.285774 12145 raft_consensus.cc:1275] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.287765 12370 consensus_queue.cc:1048] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:29.297073 12184 consensus_peers.cc:597] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:29.307093 12184 consensus_peers.cc:597] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:29.346911 12145 raft_consensus.cc:1275] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:29.348487 12354 consensus_queue.cc:1048] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:29.716568 12110 consensus_peers.cc:597] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:29.811546 12110 consensus_peers.cc:597] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:29.881924 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:05:30.015484 12110 consensus_peers.cc:597] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:30.104672 12184 consensus_peers.cc:597] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:30.184602 12110 consensus_peers.cc:597] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:30.245052 12370 raft_consensus.cc:493] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.245927 12370 raft_consensus.cc:515] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:30.251075 12370 leader_election.cc:290] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
W20260709 13:05:30.256378 12184 leader_election.cc:336] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:30.274806 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d" is_pre_election: true
I20260709 13:05:30.275866 12144 raft_consensus.cc:2468] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 2.
I20260709 13:05:30.277619 12183 leader_election.cc:304] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe
I20260709 13:05:30.278574 12370 raft_consensus.cc:2804] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:30.278982 12370 raft_consensus.cc:493] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.279366 12370 raft_consensus.cc:3060] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:30.295554 12370 raft_consensus.cc:515] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:30.299919 12370 leader_election.cc:290] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 3 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
W20260709 13:05:30.355434 12184 leader_election.cc:336] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111)
I20260709 13:05:30.361578 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:30.362702 12145 raft_consensus.cc:3060] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:30.371852 12145 raft_consensus.cc:2468] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 3.
I20260709 13:05:30.377902 12183 leader_election.cc:304] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 02778daad9b44c44adc4d26ec80937fe
I20260709 13:05:30.379192 12370 raft_consensus.cc:2804] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:30.379774 12370 raft_consensus.cc:697] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 3 LEADER]: Becoming Leader. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:30.380918 12370 consensus_queue.cc:237] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:30.392107 11966 catalog_manager.cc:5697] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 reported cstate change: term changed from 1 to 3, leader changed from 02778daad9b44c44adc4d26ec80937fe (127.0.107.193) to 5931067eb88744959a166c3c7ca17be7 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "5931067eb88744959a166c3c7ca17be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:30.795975 12145 raft_consensus.cc:1275] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:30.797631 12370 consensus_queue.cc:1048] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:30.817889 12184 consensus_peers.cc:597] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:30.883275 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:05:31.314971 12166 debug-util.cc:398] Leaking SignalData structure 0x7b08002582e0 after lost signal to thread 434
W20260709 13:05:31.316720 12166 debug-util.cc:398] Leaking SignalData structure 0x7b080029f940 after lost signal to thread 12002
W20260709 13:05:31.318099 12166 debug-util.cc:398] Leaking SignalData structure 0x7b08000feca0 after lost signal to thread 12169
W20260709 13:05:31.319322 12166 debug-util.cc:398] Leaking SignalData structure 0x7b08002eb1e0 after lost signal to thread 12243
W20260709 13:05:31.660696 12184 consensus_peers.cc:597] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:31.742250 12110 consensus_peers.cc:597] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:31.870632 12184 consensus_peers.cc:597] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:31.884538 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:05:31.963230 12184 consensus_peers.cc:597] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:32.207456 12110 consensus_peers.cc:597] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:32.297154 12110 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111) [suppressed 94 similar messages]
W20260709 13:05:32.442451 12110 consensus_peers.cc:597] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:32.500517 12110 consensus_peers.cc:597] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:32.522667   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:32.533954 12385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:32.534293 12384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:32.541428 12387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:32.542305   431 server_base.cc:1061] running on GCE node
I20260709 13:05:32.544912   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:32.545230   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:32.545498   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602332545458 us; error 0 us; skew 500 ppm
I20260709 13:05:32.546406   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:32.551862   431 webserver.cc:533] Webserver started at http://127.0.107.193:44317/ using document root <none> and password file <none>
I20260709 13:05:32.552765   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:32.553038   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:05:32.558003   431 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:05:32.562951 12392 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:32.563939   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 13:05:32.564280   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-0-root
uuid: "02778daad9b44c44adc4d26ec80937fe"
format_stamp: "Formatted at 2026-07-09 13:05:24 on dist-test-slave-l8g5"
I20260709 13:05:32.564694   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601924925497-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:32.583356   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:32.584985   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:32.587217   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:32.592228 12399 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:05:32.658762   431 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:05:32.659117   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.068s	user 0.002s	sys 0.001s
I20260709 13:05:32.659474   431 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:05:32.668545 12399 tablet_bootstrap.cc:492] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
W20260709 13:05:32.676445 12110 consensus_peers.cc:597] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:32.688731 12399 tablet_bootstrap.cc:492] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:32.689931 12399 tablet_bootstrap.cc:492] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe: Bootstrap complete.
I20260709 13:05:32.690677 12399 ts_tablet_manager.cc:1403] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.003s
I20260709 13:05:32.693900 12399 raft_consensus.cc:359] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.694686 12399 raft_consensus.cc:740] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.695677 12399 consensus_queue.cc:260] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.701085 12399 ts_tablet_manager.cc:1434] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.004s
I20260709 13:05:32.702028 12399 tablet_bootstrap.cc:492] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:32.721431 12399 tablet_bootstrap.cc:492] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:32.722929 12399 tablet_bootstrap.cc:492] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe: Bootstrap complete.
I20260709 13:05:32.723790 12399 ts_tablet_manager.cc:1403] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.012s
I20260709 13:05:32.726930 12399 raft_consensus.cc:359] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
W20260709 13:05:32.727432 12184 consensus_peers.cc:597] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:32.727792 12399 raft_consensus.cc:740] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.728682 12399 consensus_queue.cc:260] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:32.730695 12399 ts_tablet_manager.cc:1434] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:32.731654 12399 tablet_bootstrap.cc:492] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:32.740396   431 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:05:32.740993   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.082s	user 0.065s	sys 0.012s
I20260709 13:05:32.762434 12399 tablet_bootstrap.cc:492] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:32.763605 12399 tablet_bootstrap.cc:492] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap complete.
I20260709 13:05:32.764441 12399 ts_tablet_manager.cc:1403] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.008s
I20260709 13:05:32.768887 12399 raft_consensus.cc:359] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.769809 12399 raft_consensus.cc:740] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.770826 12399 consensus_queue.cc:260] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.773007 12399 ts_tablet_manager.cc:1434] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:05:32.773908 12399 tablet_bootstrap.cc:492] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:32.797866 12399 tablet_bootstrap.cc:492] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:32.799242 12399 tablet_bootstrap.cc:492] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap complete.
I20260709 13:05:32.800078 12399 ts_tablet_manager.cc:1403] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.026s	user 0.021s	sys 0.001s
I20260709 13:05:32.803277 12399 raft_consensus.cc:359] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.804111 12399 raft_consensus.cc:740] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.804893 12399 consensus_queue.cc:260] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.808135 12399 ts_tablet_manager.cc:1434] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:05:32.809079 12399 tablet_bootstrap.cc:492] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:32.831624 12399 tablet_bootstrap.cc:492] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:32.833284 12399 tablet_bootstrap.cc:492] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap complete.
I20260709 13:05:32.834991 12399 ts_tablet_manager.cc:1403] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.004s
I20260709 13:05:32.839321 12399 raft_consensus.cc:359] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.840479 12399 raft_consensus.cc:740] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.841854 12399 consensus_queue.cc:260] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.848675 12399 ts_tablet_manager.cc:1434] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:05:32.850135 12399 tablet_bootstrap.cc:492] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:32.879483 12399 tablet_bootstrap.cc:492] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:05:32.880785 12399 tablet_bootstrap.cc:492] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap complete.
I20260709 13:05:32.881680 12399 ts_tablet_manager.cc:1403] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.008s
I20260709 13:05:32.883337 12464 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:44671 every 8 connection(s)
I20260709 13:05:32.884126   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:44671
I20260709 13:05:32.885185 12399 raft_consensus.cc:359] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.886307 12399 raft_consensus.cc:740] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
W20260709 13:05:32.886402 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:32.887326 12399 consensus_queue.cc:260] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.889773 12399 ts_tablet_manager.cc:1434] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:32.890758 12399 tablet_bootstrap.cc:492] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:32.916946   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:32.927664 12399 tablet_bootstrap.cc:492] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:32.929010 12399 tablet_bootstrap.cc:492] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap complete.
I20260709 13:05:32.929793 12399 ts_tablet_manager.cc:1403] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.039s	user 0.028s	sys 0.009s
I20260709 13:05:32.933463 12399 raft_consensus.cc:359] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.934444 12399 raft_consensus.cc:740] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.935753 12399 consensus_queue.cc:260] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:32.947935 12399 ts_tablet_manager.cc:1434] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 13:05:32.954645 12399 tablet_bootstrap.cc:492] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:32.958576 12465 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38097
I20260709 13:05:32.959354 12465 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:32.960482 12465 heartbeater.cc:507] Master 127.0.107.254:38097 requested a full tablet report, sending...
I20260709 13:05:32.969789 11965 ts_manager.cc:194] Re-registered known tserver with Master: 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671)
I20260709 13:05:32.987747 11965 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51688
I20260709 13:05:32.993794 12465 heartbeater.cc:499] Master 127.0.107.254:38097 was elected leader, sending a full tablet report...
I20260709 13:05:32.994839 12399 tablet_bootstrap.cc:492] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:32.996090 12399 tablet_bootstrap.cc:492] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe: Bootstrap complete.
I20260709 13:05:32.997598 12399 ts_tablet_manager.cc:1403] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.005s
I20260709 13:05:33.001101 12437 raft_consensus.cc:3060] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:33.001629 12438 raft_consensus.cc:3060] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.002669 12399 raft_consensus.cc:359] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.003469 12399 raft_consensus.cc:740] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.004312 12399 consensus_queue.cc:260] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.006735 12399 ts_tablet_manager.cc:1434] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:05:33.007679 12399 tablet_bootstrap.cc:492] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:33.019821 12219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147"
dest_uuid: "5931067eb88744959a166c3c7ca17be7"
mode: GRACEFUL
new_leader_uuid: "5e84ba7b45c74d56a26eab812660f57d"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:05:33.020727 12219 raft_consensus.cc:606] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Received request to transfer leadership to TS 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:33.024436 12436 raft_consensus.cc:3060] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:33.039290 12436 pending_rounds.cc:85] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Aborting all ops after (but not including) 0
I20260709 13:05:33.039738 12436 pending_rounds.cc:107] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:05:33.040228 12436 raft_consensus.cc:2889] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:33.058419 12399 tablet_bootstrap.cc:492] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:05:33.064356 12437 raft_consensus.cc:3060] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:33.064917 12399 tablet_bootstrap.cc:492] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: Bootstrap complete.
I20260709 13:05:33.068176 12399 ts_tablet_manager.cc:1403] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.061s	user 0.020s	sys 0.004s
I20260709 13:05:33.073562 12399 raft_consensus.cc:359] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.084506 12145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f"
dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
mode: GRACEFUL
new_leader_uuid: "5931067eb88744959a166c3c7ca17be7"
 from {username='slave'} at 127.0.0.1:33588
I20260709 13:05:33.085656 12145 raft_consensus.cc:606] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Received request to transfer leadership to TS 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:33.087350 12399 raft_consensus.cc:740] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.088209   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:33.088336 12399 consensus_queue.cc:260] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.090000   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:33.093066 12399 ts_tablet_manager.cc:1434] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 13:05:33.094125   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:33.094269 12399 tablet_bootstrap.cc:492] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe: Bootstrap starting.
I20260709 13:05:33.118789 12399 tablet_bootstrap.cc:492] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:33.119689 12399 tablet_bootstrap.cc:492] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe: Bootstrap complete.
I20260709 13:05:33.120323 12399 ts_tablet_manager.cc:1403] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.004s
I20260709 13:05:33.122519 12399 raft_consensus.cc:359] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.123418 12399 raft_consensus.cc:740] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02778daad9b44c44adc4d26ec80937fe, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.124325 12399 consensus_queue.cc:260] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.126332 12399 ts_tablet_manager.cc:1434] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:05:33.268100 12400 raft_consensus.cc:993] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d: : Instructing follower 5931067eb88744959a166c3c7ca17be7 to start an election
I20260709 13:05:33.268821 12483 raft_consensus.cc:1081] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Signalling peer 5931067eb88744959a166c3c7ca17be7 to start an election
I20260709 13:05:33.270851 12219 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f"
dest_uuid: "5931067eb88744959a166c3c7ca17be7"
 from {username='slave'} at 127.0.0.1:45660
I20260709 13:05:33.271647 12219 raft_consensus.cc:493] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:33.272109 12219 raft_consensus.cc:3060] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:33.282842 12219 raft_consensus.cc:515] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.285454 12219 leader_election.cc:290] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:33.286523 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:33.286405 12438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:33.287358 12145 raft_consensus.cc:3055] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:33.287382 12438 raft_consensus.cc:3060] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:33.287844 12145 raft_consensus.cc:740] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:33.288851 12145 consensus_queue.cc:260] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.290097 12145 raft_consensus.cc:3060] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:33.296452 12145 raft_consensus.cc:2468] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 2.
I20260709 13:05:33.296962 12438 raft_consensus.cc:2468] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 2.
I20260709 13:05:33.298296 12183 leader_election.cc:304] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:33.299974 12382 raft_consensus.cc:2804] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:33.301844 12382 raft_consensus.cc:697] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Becoming Leader. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:33.304086 12382 consensus_queue.cc:237] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.317126 11965 catalog_manager.cc:5697] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 reported cstate change: term changed from 1 to 2, leader changed from 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194) to 5931067eb88744959a166c3c7ca17be7 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "5931067eb88744959a166c3c7ca17be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:33.367563 12438 raft_consensus.cc:3060] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:33.411142 12382 raft_consensus.cc:993] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7: : Instructing follower 5e84ba7b45c74d56a26eab812660f57d to start an election
I20260709 13:05:33.411993 12404 raft_consensus.cc:1081] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Signalling peer 5e84ba7b45c74d56a26eab812660f57d to start an election
I20260709 13:05:33.414008 12145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147"
dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
 from {username='slave'} at 127.0.0.1:60408
I20260709 13:05:33.414674 12145 raft_consensus.cc:493] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:33.415109 12145 raft_consensus.cc:3060] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.428093 12145 raft_consensus.cc:515] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.431613 12145 leader_election.cc:290] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 3 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:33.431910 12438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:33.432739 12438 raft_consensus.cc:3060] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.433072 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50f0c97ddd24bef8b1df9c70c611147" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:33.433748 12219 raft_consensus.cc:3055] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:33.434084 12219 raft_consensus.cc:740] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:33.434965 12219 consensus_queue.cc:260] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.439812 12219 raft_consensus.cc:3060] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.457543 12219 raft_consensus.cc:2468] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 3.
I20260709 13:05:33.463863 12438 raft_consensus.cc:2468] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 3.
I20260709 13:05:33.466176 12110 leader_election.cc:304] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:33.468691 12483 raft_consensus.cc:2804] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:33.469280 12483 raft_consensus.cc:697] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 3 LEADER]: Becoming Leader. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:33.471338 12483 consensus_queue.cc:237] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:33.626897 11965 catalog_manager.cc:5697] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d reported cstate change: term changed from 2 to 3, leader changed from 5931067eb88744959a166c3c7ca17be7 (127.0.107.195) to 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194). New cstate: current_term: 3 leader_uuid: "5e84ba7b45c74d56a26eab812660f57d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:33.835767 12145 raft_consensus.cc:1275] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:33.837180 12484 consensus_queue.cc:1048] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:33.853561 12438 raft_consensus.cc:1275] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:33.856339 12382 consensus_queue.cc:1048] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:33.887596 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:33.901430 12219 raft_consensus.cc:1275] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Refusing update from remote peer 5e84ba7b45c74d56a26eab812660f57d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:33.903119 12493 consensus_queue.cc:1048] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:33.912977 12438 raft_consensus.cc:1275] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Refusing update from remote peer 5e84ba7b45c74d56a26eab812660f57d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:33.915132 12483 consensus_queue.cc:1048] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:34.126886   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:34.139676 12145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1"
dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
mode: GRACEFUL
new_leader_uuid: "5931067eb88744959a166c3c7ca17be7"
 from {username='slave'} at 127.0.0.1:33588
I20260709 13:05:34.140599 12145 raft_consensus.cc:606] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 2 LEADER]: Received request to transfer leadership to TS 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:34.144054 12219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f"
dest_uuid: "5931067eb88744959a166c3c7ca17be7"
mode: GRACEFUL
new_leader_uuid: "5e84ba7b45c74d56a26eab812660f57d"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:05:34.144868 12219 raft_consensus.cc:606] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Received request to transfer leadership to TS 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:34.146636   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:34.147766   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:34.148653   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:34.369642 12400 raft_consensus.cc:993] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d: : Instructing follower 5931067eb88744959a166c3c7ca17be7 to start an election
I20260709 13:05:34.370388 12483 raft_consensus.cc:1081] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 2 LEADER]: Signalling peer 5931067eb88744959a166c3c7ca17be7 to start an election
I20260709 13:05:34.372215 12219 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1"
dest_uuid: "5931067eb88744959a166c3c7ca17be7"
 from {username='slave'} at 127.0.0.1:45660
I20260709 13:05:34.372977 12219 raft_consensus.cc:493] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:34.373396 12219 raft_consensus.cc:3060] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:34.383702 12219 raft_consensus.cc:515] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:34.386221 12219 leader_election.cc:290] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 3 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:34.387372 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:34.388019 12145 raft_consensus.cc:3055] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:34.388309 12145 raft_consensus.cc:740] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:34.388372 12438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2079d0db2fd46a1860dfc5f51324ab1" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:34.389117 12438 raft_consensus.cc:3060] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:34.389109 12145 consensus_queue.cc:260] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:34.390591 12145 raft_consensus.cc:3060] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:34.395818 12145 raft_consensus.cc:2468] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 3.
I20260709 13:05:34.397064 12183 leader_election.cc:304] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:34.397914 12484 raft_consensus.cc:2804] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:34.398386 12484 raft_consensus.cc:697] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 3 LEADER]: Becoming Leader. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:34.398974 12438 raft_consensus.cc:2468] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 3.
I20260709 13:05:34.399353 12484 consensus_queue.cc:237] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:34.405273 12499 raft_consensus.cc:993] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7: : Instructing follower 5e84ba7b45c74d56a26eab812660f57d to start an election
I20260709 13:05:34.407997 12382 raft_consensus.cc:1081] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Signalling peer 5e84ba7b45c74d56a26eab812660f57d to start an election
I20260709 13:05:34.410895 12145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f"
dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
 from {username='slave'} at 127.0.0.1:60408
I20260709 13:05:34.412199 12145 raft_consensus.cc:493] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:34.413115 12145 raft_consensus.cc:3060] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:34.426152 12145 raft_consensus.cc:515] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:34.424921 11965 catalog_manager.cc:5697] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 reported cstate change: term changed from 2 to 3, leader changed from 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194) to 5931067eb88744959a166c3c7ca17be7 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "5931067eb88744959a166c3c7ca17be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.428205 12145 leader_election.cc:290] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 3 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:34.429240 12438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:34.429512 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dfd031362f40daa5a3498b52c0333f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:34.430054 12438 raft_consensus.cc:3060] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:34.430456 12219 raft_consensus.cc:3055] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:34.430845 12219 raft_consensus.cc:740] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:34.431834 12219 consensus_queue.cc:260] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:34.433566 12219 raft_consensus.cc:3060] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:34.442404 12438 raft_consensus.cc:2468] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 3.
I20260709 13:05:34.444038 12110 leader_election.cc:304] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:34.444973 12483 raft_consensus.cc:2804] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:34.445508 12483 raft_consensus.cc:697] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 3 LEADER]: Becoming Leader. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:34.446458 12483 consensus_queue.cc:237] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:34.450708 12219 raft_consensus.cc:2468] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 3.
I20260709 13:05:34.459463 11966 catalog_manager.cc:5697] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d reported cstate change: term changed from 2 to 3, leader changed from 5931067eb88744959a166c3c7ca17be7 (127.0.107.195) to 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194). New cstate: current_term: 3 leader_uuid: "5e84ba7b45c74d56a26eab812660f57d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:34.589774 12460 debug-util.cc:398] Leaking SignalData structure 0x7b08003c8c40 after lost signal to thread 434
W20260709 13:05:34.591168 12460 debug-util.cc:398] Leaking SignalData structure 0x7b0800083120 after lost signal to thread 12002
W20260709 13:05:34.592619 12460 debug-util.cc:398] Leaking SignalData structure 0x7b080009cbc0 after lost signal to thread 12169
W20260709 13:05:34.594260 12460 debug-util.cc:398] Leaking SignalData structure 0x7b08002853e0 after lost signal to thread 12243
W20260709 13:05:34.596210 12460 debug-util.cc:398] Leaking SignalData structure 0x7b0800199ce0 after lost signal to thread 12464
I20260709 13:05:34.817632 12145 raft_consensus.cc:1275] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:34.819711 12500 consensus_queue.cc:1048] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:34.837586 12438 raft_consensus.cc:1275] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:34.839780 12484 consensus_queue.cc:1048] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:34.880983 12219 raft_consensus.cc:1275] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Refusing update from remote peer 5e84ba7b45c74d56a26eab812660f57d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:34.881003 12438 raft_consensus.cc:1275] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Refusing update from remote peer 5e84ba7b45c74d56a26eab812660f57d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:34.884143 12505 consensus_queue.cc:1048] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:34.885619 12400 consensus_queue.cc:1048] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:05:34.888824 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:35.160406   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:35.169872 12218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8"
dest_uuid: "5931067eb88744959a166c3c7ca17be7"
mode: GRACEFUL
new_leader_uuid: "5e84ba7b45c74d56a26eab812660f57d"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:05:35.170682 12218 raft_consensus.cc:606] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 3 LEADER]: Received request to transfer leadership to TS 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:35.173645 12145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f"
dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
mode: GRACEFUL
new_leader_uuid: "5931067eb88744959a166c3c7ca17be7"
 from {username='slave'} at 127.0.0.1:33588
I20260709 13:05:35.174173 12145 raft_consensus.cc:606] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Received request to transfer leadership to TS 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:35.175607   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:35.176188   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:35.176960   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:35.288576 12484 raft_consensus.cc:993] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7: : Instructing follower 5e84ba7b45c74d56a26eab812660f57d to start an election
I20260709 13:05:35.289239 12500 raft_consensus.cc:1081] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 3 LEADER]: Signalling peer 5e84ba7b45c74d56a26eab812660f57d to start an election
I20260709 13:05:35.291007 12145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8"
dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
 from {username='slave'} at 127.0.0.1:60408
I20260709 13:05:35.291555 12145 raft_consensus.cc:493] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:35.291872 12145 raft_consensus.cc:3060] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:35.299183 12145 raft_consensus.cc:515] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:35.301419 12145 leader_election.cc:290] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 4 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:35.302870 12218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:35.303148 12436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa91e52f62148258f74d8f19735a8d8" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:35.303732 12218 raft_consensus.cc:3055] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:35.303933 12436 raft_consensus.cc:3060] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:35.304167 12218 raft_consensus.cc:740] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:35.305442 12218 consensus_queue.cc:260] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:35.306759 12218 raft_consensus.cc:3060] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:35.315236 12218 raft_consensus.cc:2468] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 4.
I20260709 13:05:35.315232 12436 raft_consensus.cc:2468] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 4.
I20260709 13:05:35.316648 12109 leader_election.cc:304] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:35.317665 12400 raft_consensus.cc:2804] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:35.318212 12400 raft_consensus.cc:697] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 4 LEADER]: Becoming Leader. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:35.319281 12400 consensus_queue.cc:237] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:35.328995 11966 catalog_manager.cc:5697] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d reported cstate change: term changed from 3 to 4, leader changed from 5931067eb88744959a166c3c7ca17be7 (127.0.107.195) to 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194). New cstate: current_term: 4 leader_uuid: "5e84ba7b45c74d56a26eab812660f57d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.518538 12505 raft_consensus.cc:993] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d: : Instructing follower 5931067eb88744959a166c3c7ca17be7 to start an election
I20260709 13:05:35.519100 12400 raft_consensus.cc:1081] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Signalling peer 5931067eb88744959a166c3c7ca17be7 to start an election
I20260709 13:05:35.520977 12218 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f"
dest_uuid: "5931067eb88744959a166c3c7ca17be7"
 from {username='slave'} at 127.0.0.1:45660
I20260709 13:05:35.521629 12218 raft_consensus.cc:493] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:35.521936 12218 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:35.528198 12218 raft_consensus.cc:515] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:35.530215 12218 leader_election.cc:290] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Requested vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671)
I20260709 13:05:35.530965 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:35.531634 12145 raft_consensus.cc:3055] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:35.531953 12145 raft_consensus.cc:740] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:35.532841 12436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:35.532755 12145 consensus_queue.cc:260] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:35.533689 12436 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:35.534344 12145 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:35.540264 12145 raft_consensus.cc:2468] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 2.
I20260709 13:05:35.541618 12183 leader_election.cc:304] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5931067eb88744959a166c3c7ca17be7, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:35.542461 12500 raft_consensus.cc:2804] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:35.542944 12500 raft_consensus.cc:697] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Becoming Leader. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:35.543236 12436 raft_consensus.cc:2468] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 2.
I20260709 13:05:35.543977 12500 consensus_queue.cc:237] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:35.555526 11966 catalog_manager.cc:5697] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 reported cstate change: term changed from 1 to 2, leader changed from 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194) to 5931067eb88744959a166c3c7ca17be7 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "5931067eb88744959a166c3c7ca17be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.741590 12219 raft_consensus.cc:1275] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 4 FOLLOWER]: Refusing update from remote peer 5e84ba7b45c74d56a26eab812660f57d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:35.743211 12400 consensus_queue.cc:1048] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:35.779740 12436 raft_consensus.cc:1275] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 4 FOLLOWER]: Refusing update from remote peer 5e84ba7b45c74d56a26eab812660f57d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:35.782083 12400 consensus_queue.cc:1048] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:05:35.890230 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:35.928849 12438 raft_consensus.cc:1275] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:35.930415 12500 consensus_queue.cc:1048] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:35.948154 12145 raft_consensus.cc:1275] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:35.949965 12500 consensus_queue.cc:1048] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:36.185058   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:36.197274 12145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175"
dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
mode: GRACEFUL
new_leader_uuid: "5931067eb88744959a166c3c7ca17be7"
 from {username='slave'} at 127.0.0.1:33588
I20260709 13:05:36.198562 12145 raft_consensus.cc:606] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Received request to transfer leadership to TS 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:36.203778 12219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f"
dest_uuid: "5931067eb88744959a166c3c7ca17be7"
mode: GRACEFUL
new_leader_uuid: "5e84ba7b45c74d56a26eab812660f57d"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:05:36.204725 12219 raft_consensus.cc:606] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Received request to transfer leadership to TS 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:36.207261   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:36.207998   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:36.208968   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:36.346354 12500 raft_consensus.cc:993] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7: : Instructing follower 5e84ba7b45c74d56a26eab812660f57d to start an election
I20260709 13:05:36.346951 12500 raft_consensus.cc:1081] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Signalling peer 5e84ba7b45c74d56a26eab812660f57d to start an election
I20260709 13:05:36.349254 12144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f"
dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
 from {username='slave'} at 127.0.0.1:60408
I20260709 13:05:36.350118 12144 raft_consensus.cc:493] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:36.350535 12144 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.362360 12144 raft_consensus.cc:515] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:36.365607 12144 leader_election.cc:290] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 3 election: Requested vote from peers 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771), 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671)
I20260709 13:05:36.366936 12438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:36.366945 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:36.368342 12438 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.368745 12219 raft_consensus.cc:3055] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:36.369187 12219 raft_consensus.cc:740] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:36.370783 12219 consensus_queue.cc:260] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:36.373410 12219 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.381052 12438 raft_consensus.cc:2468] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 3.
I20260709 13:05:36.383100 12110 leader_election.cc:304] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:36.384088 12219 raft_consensus.cc:2468] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 3.
I20260709 13:05:36.384109 12528 raft_consensus.cc:2804] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:36.385026 12528 raft_consensus.cc:697] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 3 LEADER]: Becoming Leader. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:36.386360 12528 consensus_queue.cc:237] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:36.396780 11966 catalog_manager.cc:5697] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d reported cstate change: term changed from 2 to 3, leader changed from 5931067eb88744959a166c3c7ca17be7 (127.0.107.195) to 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194). New cstate: current_term: 3 leader_uuid: "5e84ba7b45c74d56a26eab812660f57d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.548300 12527 raft_consensus.cc:993] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d: : Instructing follower 5931067eb88744959a166c3c7ca17be7 to start an election
I20260709 13:05:36.548897 12528 raft_consensus.cc:1081] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Signalling peer 5931067eb88744959a166c3c7ca17be7 to start an election
I20260709 13:05:36.550987 12219 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175"
dest_uuid: "5931067eb88744959a166c3c7ca17be7"
 from {username='slave'} at 127.0.0.1:45660
I20260709 13:05:36.551748 12219 raft_consensus.cc:493] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:36.552210 12219 raft_consensus.cc:3060] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.559229 12219 raft_consensus.cc:515] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:36.561990 12219 leader_election.cc:290] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007)
I20260709 13:05:36.563040 12438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:36.563548 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fb875cae93c4f4db7949ea8b260f175" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:36.563865 12438 raft_consensus.cc:3060] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.564311 12144 raft_consensus.cc:3055] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:36.564756 12144 raft_consensus.cc:740] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:36.565840 12144 consensus_queue.cc:260] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:36.567584 12144 raft_consensus.cc:3060] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.572263 12438 raft_consensus.cc:2468] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 2.
I20260709 13:05:36.573876 12184 leader_election.cc:304] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7; no voters: 
I20260709 13:05:36.574762 12500 raft_consensus.cc:2804] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:36.575309 12500 raft_consensus.cc:697] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Becoming Leader. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:36.576303 12500 consensus_queue.cc:237] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:36.578100 12144 raft_consensus.cc:2468] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 2.
I20260709 13:05:36.587203 11966 catalog_manager.cc:5697] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 reported cstate change: term changed from 1 to 2, leader changed from 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194) to 5931067eb88744959a166c3c7ca17be7 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "5931067eb88744959a166c3c7ca17be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.779094 12219 raft_consensus.cc:1275] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Refusing update from remote peer 5e84ba7b45c74d56a26eab812660f57d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:36.780736 12528 consensus_queue.cc:1048] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:36.797910 12438 raft_consensus.cc:1275] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Refusing update from remote peer 5e84ba7b45c74d56a26eab812660f57d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:36.800405 12505 consensus_queue.cc:1048] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:05:36.891811 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:37.061956 12144 raft_consensus.cc:1275] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:37.063952 12500 consensus_queue.cc:1048] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:37.078981 12438 raft_consensus.cc:1275] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:37.081696 12540 consensus_queue.cc:1048] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:37.216384   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:37.225208 12219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed"
dest_uuid: "5931067eb88744959a166c3c7ca17be7"
mode: GRACEFUL
new_leader_uuid: "5e84ba7b45c74d56a26eab812660f57d"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:05:37.226154 12219 raft_consensus.cc:606] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Received request to transfer leadership to TS 5e84ba7b45c74d56a26eab812660f57d
I20260709 13:05:37.230199 12144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f"
dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
mode: GRACEFUL
new_leader_uuid: "5931067eb88744959a166c3c7ca17be7"
 from {username='slave'} at 127.0.0.1:33588
I20260709 13:05:37.231068 12144 raft_consensus.cc:606] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 3 LEADER]: Received request to transfer leadership to TS 5931067eb88744959a166c3c7ca17be7
I20260709 13:05:37.232678   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:37.233781   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:37.234810   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:37.241977 12505 raft_consensus.cc:993] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d: : Instructing follower 5931067eb88744959a166c3c7ca17be7 to start an election
I20260709 13:05:37.242650 12505 raft_consensus.cc:1081] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 3 LEADER]: Signalling peer 5931067eb88744959a166c3c7ca17be7 to start an election
I20260709 13:05:37.244398 12219 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f"
dest_uuid: "5931067eb88744959a166c3c7ca17be7"
 from {username='slave'} at 127.0.0.1:45660
I20260709 13:05:37.244961 12219 raft_consensus.cc:493] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:37.245328 12219 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.252079 12219 raft_consensus.cc:515] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:37.254199 12219 leader_election.cc:290] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 4 election: Requested vote from peers 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007), 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671)
I20260709 13:05:37.255980 12438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:37.256237 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb97b3caff604bf9922c1f0f93cfce3f" candidate_uuid: "5931067eb88744959a166c3c7ca17be7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
I20260709 13:05:37.256641 12438 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.256887 12144 raft_consensus.cc:3055] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:37.257313 12144 raft_consensus.cc:740] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:37.258249 12144 consensus_queue.cc:260] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:37.259675 12144 raft_consensus.cc:3060] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.265744 12438 raft_consensus.cc:2468] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 4.
I20260709 13:05:37.267365 12184 leader_election.cc:304] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5931067eb88744959a166c3c7ca17be7; no voters: 
I20260709 13:05:37.268286 12540 raft_consensus.cc:2804] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:37.268847 12540 raft_consensus.cc:697] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 4 LEADER]: Becoming Leader. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:37.268652 12144 raft_consensus.cc:2468] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5931067eb88744959a166c3c7ca17be7 in term 4.
I20260709 13:05:37.270285 12540 consensus_queue.cc:237] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } }
I20260709 13:05:37.279448 11965 catalog_manager.cc:5697] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 reported cstate change: term changed from 3 to 4, leader changed from 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194) to 5931067eb88744959a166c3c7ca17be7 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "5931067eb88744959a166c3c7ca17be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.459929 12529 raft_consensus.cc:993] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7: : Instructing follower 5e84ba7b45c74d56a26eab812660f57d to start an election
I20260709 13:05:37.460556 12540 raft_consensus.cc:1081] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Signalling peer 5e84ba7b45c74d56a26eab812660f57d to start an election
I20260709 13:05:37.462236 12144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed"
dest_uuid: "5e84ba7b45c74d56a26eab812660f57d"
 from {username='slave'} at 127.0.0.1:60408
I20260709 13:05:37.462780 12144 raft_consensus.cc:493] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:37.463132 12144 raft_consensus.cc:3060] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:37.472102 12144 raft_consensus.cc:515] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:37.474123 12144 leader_election.cc:290] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 3 election: Requested vote from peers 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671), 5931067eb88744959a166c3c7ca17be7 (127.0.107.195:37771)
I20260709 13:05:37.475222 12438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "02778daad9b44c44adc4d26ec80937fe"
I20260709 13:05:37.475301 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28fd3064bea54ab3ba7adad3faddc5ed" candidate_uuid: "5e84ba7b45c74d56a26eab812660f57d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5931067eb88744959a166c3c7ca17be7"
I20260709 13:05:37.476118 12438 raft_consensus.cc:3060] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:37.476187 12219 raft_consensus.cc:3055] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:37.476653 12219 raft_consensus.cc:740] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5931067eb88744959a166c3c7ca17be7, State: Running, Role: LEADER
I20260709 13:05:37.478557 12219 consensus_queue.cc:260] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:37.479911 12219 raft_consensus.cc:3060] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:37.486171 12438 raft_consensus.cc:2468] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 3.
I20260709 13:05:37.487629 12219 raft_consensus.cc:2468] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e84ba7b45c74d56a26eab812660f57d in term 3.
I20260709 13:05:37.487617 12110 leader_election.cc:304] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02778daad9b44c44adc4d26ec80937fe, 5e84ba7b45c74d56a26eab812660f57d; no voters: 
I20260709 13:05:37.488683 12505 raft_consensus.cc:2804] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:37.489248 12505 raft_consensus.cc:697] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 3 LEADER]: Becoming Leader. State: Replica: 5e84ba7b45c74d56a26eab812660f57d, State: Running, Role: LEADER
I20260709 13:05:37.490989 12505 consensus_queue.cc:237] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } }
I20260709 13:05:37.500748 11965 catalog_manager.cc:5697] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d reported cstate change: term changed from 2 to 3, leader changed from 5931067eb88744959a166c3c7ca17be7 (127.0.107.195) to 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194). New cstate: current_term: 3 leader_uuid: "5e84ba7b45c74d56a26eab812660f57d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.716786 12438 raft_consensus.cc:1275] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 4 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:37.718809 12529 consensus_queue.cc:1048] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:37.735510 12144 raft_consensus.cc:1275] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 4 FOLLOWER]: Refusing update from remote peer 5931067eb88744959a166c3c7ca17be7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:37.737442 12529 consensus_queue.cc:1048] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e84ba7b45c74d56a26eab812660f57d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:37.874346 12219 raft_consensus.cc:1275] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Refusing update from remote peer 5e84ba7b45c74d56a26eab812660f57d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:37.875860 12528 consensus_queue.cc:1048] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5931067eb88744959a166c3c7ca17be7" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:37.891142 12438 raft_consensus.cc:1275] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Refusing update from remote peer 5e84ba7b45c74d56a26eab812660f57d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:05:37.893196 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:37.894800 12528 consensus_queue.cc:1048] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [LEADER]: Connected to new peer: Peer: permanent_uuid: "02778daad9b44c44adc4d26ec80937fe" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:38.248122   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:38.289371   431 tablet_server.cc:179] TabletServer@127.0.107.193:44671 shutting down...
I20260709 13:05:38.338327   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:38.339483   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:38.340243   431 raft_consensus.cc:2243] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.340852   431 raft_consensus.cc:2272] T e3dfd031362f40daa5a3498b52c0333f P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.343623   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:38.344694   431 raft_consensus.cc:2243] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.345221   431 raft_consensus.cc:2272] T 3dbdc05a36654d48be71d39e25e90e7d P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.347880   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:38.348582   431 raft_consensus.cc:2243] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.349112   431 raft_consensus.cc:2272] T aa47d580d5d349d4aa183aa2b1c938f4 P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.351445   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:38.352169   431 raft_consensus.cc:2243] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.352632   431 raft_consensus.cc:2272] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 02778daad9b44c44adc4d26ec80937fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.355792   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:05:38.356752 12110 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
I20260709 13:05:38.356871   431 raft_consensus.cc:2243] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.357729   431 raft_consensus.cc:2272] T 6fb875cae93c4f4db7949ea8b260f175 P 02778daad9b44c44adc4d26ec80937fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.360630   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:05:38.361073 12110 consensus_peers.cc:597] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.361578   431 raft_consensus.cc:2243] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.362149   431 raft_consensus.cc:2272] T 7aa91e52f62148258f74d8f19735a8d8 P 02778daad9b44c44adc4d26ec80937fe [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.365119   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:38.365796   431 raft_consensus.cc:2243] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.366253   431 raft_consensus.cc:2272] T a2079d0db2fd46a1860dfc5f51324ab1 P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.370021   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:38.370805   431 raft_consensus.cc:2243] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.371465   431 raft_consensus.cc:2272] T bb97b3caff604bf9922c1f0f93cfce3f P 02778daad9b44c44adc4d26ec80937fe [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.374485   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:38.375136   431 raft_consensus.cc:2243] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.375834   431 raft_consensus.cc:2272] T b50f0c97ddd24bef8b1df9c70c611147 P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.378293   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:38.378850   431 raft_consensus.cc:2243] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.379433   431 raft_consensus.cc:2272] T 28fd3064bea54ab3ba7adad3faddc5ed P 02778daad9b44c44adc4d26ec80937fe [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.384410 12110 consensus_peers.cc:597] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.438876   431 tablet_server.cc:196] TabletServer@127.0.107.193:44671 shutdown complete.
I20260709 13:05:38.463816   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:05:38.474303 12184 consensus_peers.cc:597] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:38.490567 12110 consensus_peers.cc:597] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.506033   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:38.507016   431 tablet_replica.cc:333] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d: stopping tablet replica
I20260709 13:05:38.507865   431 raft_consensus.cc:2243] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.508410   431 raft_consensus.cc:2272] T a2079d0db2fd46a1860dfc5f51324ab1 P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.511445   431 tablet_replica.cc:333] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d: stopping tablet replica
I20260709 13:05:38.512178   431 raft_consensus.cc:2243] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.513098   431 raft_consensus.cc:2272] T 7aa91e52f62148258f74d8f19735a8d8 P 5e84ba7b45c74d56a26eab812660f57d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.515501   431 tablet_replica.cc:333] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d: stopping tablet replica
I20260709 13:05:38.516116   431 raft_consensus.cc:2243] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.516624   431 raft_consensus.cc:2272] T aa47d580d5d349d4aa183aa2b1c938f4 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.519004   431 tablet_replica.cc:333] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d: stopping tablet replica
I20260709 13:05:38.519613   431 raft_consensus.cc:2243] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.520610   431 raft_consensus.cc:2272] T 3dbdc05a36654d48be71d39e25e90e7d P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.523043   431 tablet_replica.cc:333] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d: stopping tablet replica
I20260709 13:05:38.523790   431 raft_consensus.cc:2243] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.524466   431 raft_consensus.cc:2272] T bb97b3caff604bf9922c1f0f93cfce3f P 5e84ba7b45c74d56a26eab812660f57d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.528151   431 tablet_replica.cc:333] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d: stopping tablet replica
I20260709 13:05:38.528836   431 raft_consensus.cc:2243] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.529829   431 raft_consensus.cc:2272] T 28fd3064bea54ab3ba7adad3faddc5ed P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.532722   431 tablet_replica.cc:333] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d: stopping tablet replica
I20260709 13:05:38.533402   431 raft_consensus.cc:2243] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.534343   431 raft_consensus.cc:2272] T b50f0c97ddd24bef8b1df9c70c611147 P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.536747   431 tablet_replica.cc:333] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d: stopping tablet replica
I20260709 13:05:38.537739   431 raft_consensus.cc:2243] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.538470   431 raft_consensus.cc:2272] T 6fb875cae93c4f4db7949ea8b260f175 P 5e84ba7b45c74d56a26eab812660f57d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.540944   431 tablet_replica.cc:333] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d: stopping tablet replica
I20260709 13:05:38.541553   431 raft_consensus.cc:2243] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.542057   431 raft_consensus.cc:2272] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5e84ba7b45c74d56a26eab812660f57d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.544549   431 tablet_replica.cc:333] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d: stopping tablet replica
I20260709 13:05:38.545089   431 raft_consensus.cc:2243] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.546038   431 raft_consensus.cc:2272] T e3dfd031362f40daa5a3498b52c0333f P 5e84ba7b45c74d56a26eab812660f57d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.585667 12184 consensus_peers.cc:597] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.603333   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:05:38.625284 12183 consensus_peers.cc:597] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 -> Peer 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007): Couldn't send request to peer 5e84ba7b45c74d56a26eab812660f57d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:35007: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.641464   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:05:38.680548   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:38.681520   431 tablet_replica.cc:333] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7: stopping tablet replica
I20260709 13:05:38.682389   431 raft_consensus.cc:2243] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.683027   431 raft_consensus.cc:2272] T 3dbdc05a36654d48be71d39e25e90e7d P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.686180   431 tablet_replica.cc:333] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7: stopping tablet replica
I20260709 13:05:38.686918   431 raft_consensus.cc:2243] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.687462   431 raft_consensus.cc:2272] T e3dfd031362f40daa5a3498b52c0333f P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.721918   431 tablet_replica.cc:333] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7: stopping tablet replica
I20260709 13:05:38.725449   431 raft_consensus.cc:2243] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.727435   431 raft_consensus.cc:2272] T 28fd3064bea54ab3ba7adad3faddc5ed P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.728030 12184 consensus_peers.cc:597] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:38.730094 12183 consensus_peers.cc:597] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 -> Peer 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007): Couldn't send request to peer 5e84ba7b45c74d56a26eab812660f57d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:35007: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:38.730921 12183 consensus_peers.cc:597] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 -> Peer 5e84ba7b45c74d56a26eab812660f57d (127.0.107.194:35007): Couldn't send request to peer 5e84ba7b45c74d56a26eab812660f57d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:35007: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.734567   431 tablet_replica.cc:333] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7: stopping tablet replica
W20260709 13:05:38.736301 12184 consensus_peers.cc:597] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 -> Peer 02778daad9b44c44adc4d26ec80937fe (127.0.107.193:44671): Couldn't send request to peer 02778daad9b44c44adc4d26ec80937fe. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.736506   431 raft_consensus.cc:2243] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.737607   431 raft_consensus.cc:2272] T b50f0c97ddd24bef8b1df9c70c611147 P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.740957   431 tablet_replica.cc:333] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7: stopping tablet replica
I20260709 13:05:38.741902   431 raft_consensus.cc:2243] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.749392   431 raft_consensus.cc:2272] T bb97b3caff604bf9922c1f0f93cfce3f P 5931067eb88744959a166c3c7ca17be7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.753289   431 tablet_replica.cc:333] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7: stopping tablet replica
I20260709 13:05:38.754110   431 raft_consensus.cc:2243] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.755879   431 raft_consensus.cc:2272] T 6fb875cae93c4f4db7949ea8b260f175 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.759706   431 tablet_replica.cc:333] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7: stopping tablet replica
I20260709 13:05:38.760629   431 raft_consensus.cc:2243] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.762310   431 raft_consensus.cc:2272] T 712ec9215b1e4546ba0a68b9f52e0eb2 P 5931067eb88744959a166c3c7ca17be7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.765877   431 tablet_replica.cc:333] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7: stopping tablet replica
I20260709 13:05:38.766681   431 raft_consensus.cc:2243] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.768287   431 raft_consensus.cc:2272] T aa47d580d5d349d4aa183aa2b1c938f4 P 5931067eb88744959a166c3c7ca17be7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.771661   431 tablet_replica.cc:333] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7: stopping tablet replica
I20260709 13:05:38.772683   431 raft_consensus.cc:2243] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.773661   431 raft_consensus.cc:2272] T 7aa91e52f62148258f74d8f19735a8d8 P 5931067eb88744959a166c3c7ca17be7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.776741   431 tablet_replica.cc:333] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7: stopping tablet replica
I20260709 13:05:38.777592   431 raft_consensus.cc:2243] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.779067   431 raft_consensus.cc:2272] T a2079d0db2fd46a1860dfc5f51324ab1 P 5931067eb88744959a166c3c7ca17be7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.826148   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:05:38.867774   431 master.cc:562] Master@127.0.107.254:38097 shutting down...
W20260709 13:05:38.895812 12016 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:38.906755   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.907963   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.908707   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3e3d80e064a8409590edd08fbf125b5d: stopping tablet replica
I20260709 13:05:38.935590   431 master.cc:584] Master@127.0.107.254:38097 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (14308 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:05:38.997764   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:34071
I20260709 13:05:38.999141   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:39.005921 12564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:39.006242 12563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:39.007388 12566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:39.009361   431 server_base.cc:1061] running on GCE node
I20260709 13:05:39.010591   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:39.010782   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:39.010975   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602339010955 us; error 0 us; skew 500 ppm
I20260709 13:05:39.011579   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:39.014796   431 webserver.cc:533] Webserver started at http://127.0.107.254:39647/ using document root <none> and password file <none>
I20260709 13:05:39.015990   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:39.016253   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:39.016538   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:39.018042   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/master-0-root/instance:
uuid: "6131031cfd3a47a6a64754fd57610e37"
format_stamp: "Formatted at 2026-07-09 13:05:39 on dist-test-slave-l8g5"
I20260709 13:05:39.023324   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:05:39.027912 12571 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:39.028956   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:05:39.029331   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/master-0-root
uuid: "6131031cfd3a47a6a64754fd57610e37"
format_stamp: "Formatted at 2026-07-09 13:05:39 on dist-test-slave-l8g5"
I20260709 13:05:39.029716   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:39.040577   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:39.042212   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:39.093407   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:34071
I20260709 13:05:39.093932 12622 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:34071 every 8 connection(s)
I20260709 13:05:39.099141 12623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.118464 12623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37: Bootstrap starting.
I20260709 13:05:39.125116 12623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.130928 12623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37: No bootstrap required, opened a new log
I20260709 13:05:39.133870 12623 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6131031cfd3a47a6a64754fd57610e37" member_type: VOTER }
I20260709 13:05:39.134407 12623 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.134601 12623 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6131031cfd3a47a6a64754fd57610e37, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.135178 12623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6131031cfd3a47a6a64754fd57610e37" member_type: VOTER }
I20260709 13:05:39.135654 12623 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:39.135872 12623 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:39.136101 12623 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.142204 12623 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6131031cfd3a47a6a64754fd57610e37" member_type: VOTER }
I20260709 13:05:39.142769 12623 leader_election.cc:304] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6131031cfd3a47a6a64754fd57610e37; no voters: 
I20260709 13:05:39.144043 12623 leader_election.cc:290] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:39.144632 12626 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.146278 12626 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [term 1 LEADER]: Becoming Leader. State: Replica: 6131031cfd3a47a6a64754fd57610e37, State: Running, Role: LEADER
I20260709 13:05:39.147119 12626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6131031cfd3a47a6a64754fd57610e37" member_type: VOTER }
I20260709 13:05:39.147894 12623 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:39.152704 12627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6131031cfd3a47a6a64754fd57610e37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6131031cfd3a47a6a64754fd57610e37" member_type: VOTER } }
I20260709 13:05:39.154913 12627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:39.152880 12628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6131031cfd3a47a6a64754fd57610e37. Latest consensus state: current_term: 1 leader_uuid: "6131031cfd3a47a6a64754fd57610e37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6131031cfd3a47a6a64754fd57610e37" member_type: VOTER } }
I20260709 13:05:39.160534 12628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:39.166699 12633 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:39.167951   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:39.172416 12633 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:39.183630 12633 catalog_manager.cc:1383] Generated new cluster ID: 113caae1bee5432da7ce32d4a4678606
I20260709 13:05:39.184010 12633 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:39.204418 12633 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:39.206043 12633 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:39.228737 12633 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37: Generated new TSK 0
I20260709 13:05:39.229796 12633 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:39.235208   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:39.242820 12644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:39.244280 12645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:39.246977   431 server_base.cc:1061] running on GCE node
W20260709 13:05:39.247789 12647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:39.248950   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:39.249192   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:39.249368   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602339249348 us; error 0 us; skew 500 ppm
I20260709 13:05:39.250142   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:39.253028   431 webserver.cc:533] Webserver started at http://127.0.107.193:34573/ using document root <none> and password file <none>
I20260709 13:05:39.253674   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:39.253911   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:39.254202   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:39.255666   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-0-root/instance:
uuid: "cfeaf86cdbd24416a2768138d5e7488c"
format_stamp: "Formatted at 2026-07-09 13:05:39 on dist-test-slave-l8g5"
I20260709 13:05:39.261996   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:05:39.267208 12652 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:39.268597   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 13:05:39.269058   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-0-root
uuid: "cfeaf86cdbd24416a2768138d5e7488c"
format_stamp: "Formatted at 2026-07-09 13:05:39 on dist-test-slave-l8g5"
I20260709 13:05:39.269538   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:39.290530   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:39.292307   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:39.294301   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:39.297403   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:39.297734   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:05:39.298166   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:39.298410   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:39.361125   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:42407
I20260709 13:05:39.361326 12714 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:42407 every 8 connection(s)
I20260709 13:05:39.367970   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:39.378149 12720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:39.379556 12719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:39.380250 12722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:39.382989   431 server_base.cc:1061] running on GCE node
I20260709 13:05:39.385103   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:39.385418   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:39.385689   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602339385666 us; error 0 us; skew 500 ppm
I20260709 13:05:39.385936 12715 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34071
I20260709 13:05:39.386525   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:39.386548 12715 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:39.387705 12715 heartbeater.cc:507] Master 127.0.107.254:34071 requested a full tablet report, sending...
I20260709 13:05:39.389909   431 webserver.cc:533] Webserver started at http://127.0.107.194:42897/ using document root <none> and password file <none>
I20260709 13:05:39.390595   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:39.390829   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:39.390952 12588 ts_manager.cc:194] Registered new tserver with Master: cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193:42407)
I20260709 13:05:39.391116   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:39.392789   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-1-root/instance:
uuid: "aa55094a82634ea3970f61b1897a2850"
format_stamp: "Formatted at 2026-07-09 13:05:39 on dist-test-slave-l8g5"
I20260709 13:05:39.393380 12588 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33856
I20260709 13:05:39.398398   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20260709 13:05:39.402977 12727 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:39.403950   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:05:39.404317   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-1-root
uuid: "aa55094a82634ea3970f61b1897a2850"
format_stamp: "Formatted at 2026-07-09 13:05:39 on dist-test-slave-l8g5"
I20260709 13:05:39.404680   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:39.436136   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:39.437732   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:39.439425   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:39.442137   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:39.442399   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:39.442668   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:39.442862   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:39.493460   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:43587
I20260709 13:05:39.493636 12789 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:43587 every 8 connection(s)
I20260709 13:05:39.499106   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:39.508585 12793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:39.509052 12794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:39.513547 12796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:39.513595   431 server_base.cc:1061] running on GCE node
I20260709 13:05:39.514511 12790 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34071
I20260709 13:05:39.515161 12790 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:39.515669   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:39.516026   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:39.516263   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602339516246 us; error 0 us; skew 500 ppm
I20260709 13:05:39.516335 12790 heartbeater.cc:507] Master 127.0.107.254:34071 requested a full tablet report, sending...
I20260709 13:05:39.517961   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:39.520183 12588 ts_manager.cc:194] Registered new tserver with Master: aa55094a82634ea3970f61b1897a2850 (127.0.107.194:43587)
I20260709 13:05:39.521555   431 webserver.cc:533] Webserver started at http://127.0.107.195:35273/ using document root <none> and password file <none>
I20260709 13:05:39.522372   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:39.522656   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:39.522444 12588 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33860
I20260709 13:05:39.523072   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:39.525130   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-2-root/instance:
uuid: "65655c7d03b04947adcc4cf326691e89"
format_stamp: "Formatted at 2026-07-09 13:05:39 on dist-test-slave-l8g5"
I20260709 13:05:39.530699   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:39.535394 12801 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:39.536511   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:05:39.536908   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-2-root
uuid: "65655c7d03b04947adcc4cf326691e89"
format_stamp: "Formatted at 2026-07-09 13:05:39 on dist-test-slave-l8g5"
I20260709 13:05:39.537317   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601924925497-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:39.569921   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:39.571601   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:39.573572   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:39.576354   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:39.576633   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:39.576932   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:39.577139   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:39.631837   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:36113
I20260709 13:05:39.631990 12863 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:36113 every 8 connection(s)
I20260709 13:05:39.650113 12864 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34071
I20260709 13:05:39.650770 12864 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:39.651760 12864 heartbeater.cc:507] Master 127.0.107.254:34071 requested a full tablet report, sending...
I20260709 13:05:39.654103 12588 ts_manager.cc:194] Registered new tserver with Master: 65655c7d03b04947adcc4cf326691e89 (127.0.107.195:36113)
I20260709 13:05:39.654973   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01887994s
I20260709 13:05:39.655690 12588 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33870
I20260709 13:05:39.689777 12588 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33874:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:39.692751 12588 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:39.756974 12680 tablet_service.cc:1511] Processing CreateTablet for tablet 7f990fb0d5e841eeabe2927a4beab3ca (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1665c8dce9a74f718206e946c558bd1a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:05:39.758747 12680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f990fb0d5e841eeabe2927a4beab3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.759569 12679 tablet_service.cc:1511] Processing CreateTablet for tablet 28a44d80e37848518b2ea81687fc5320 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1665c8dce9a74f718206e946c558bd1a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:05:39.761116 12679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a44d80e37848518b2ea81687fc5320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.765772 12754 tablet_service.cc:1511] Processing CreateTablet for tablet 28a44d80e37848518b2ea81687fc5320 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1665c8dce9a74f718206e946c558bd1a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:05:39.765772 12755 tablet_service.cc:1511] Processing CreateTablet for tablet 7f990fb0d5e841eeabe2927a4beab3ca (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1665c8dce9a74f718206e946c558bd1a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:05:39.767503 12754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a44d80e37848518b2ea81687fc5320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.767429 12829 tablet_service.cc:1511] Processing CreateTablet for tablet 7f990fb0d5e841eeabe2927a4beab3ca (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1665c8dce9a74f718206e946c558bd1a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:05:39.768356 12755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f990fb0d5e841eeabe2927a4beab3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.768968 12829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f990fb0d5e841eeabe2927a4beab3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.779435 12828 tablet_service.cc:1511] Processing CreateTablet for tablet 28a44d80e37848518b2ea81687fc5320 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1665c8dce9a74f718206e946c558bd1a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:05:39.781157 12828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a44d80e37848518b2ea81687fc5320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:39.797932 12880 tablet_bootstrap.cc:492] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c: Bootstrap starting.
I20260709 13:05:39.806262 12880 tablet_bootstrap.cc:654] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.812290 12879 tablet_bootstrap.cc:492] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850: Bootstrap starting.
I20260709 13:05:39.818842 12882 tablet_bootstrap.cc:492] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89: Bootstrap starting.
I20260709 13:05:39.821820 12879 tablet_bootstrap.cc:654] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.822027 12880 tablet_bootstrap.cc:492] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c: No bootstrap required, opened a new log
I20260709 13:05:39.822711 12880 ts_tablet_manager.cc:1403] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.004s
I20260709 13:05:39.827031 12880 raft_consensus.cc:359] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.828083 12880 raft_consensus.cc:385] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.828425 12880 raft_consensus.cc:740] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfeaf86cdbd24416a2768138d5e7488c, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.829211 12882 tablet_bootstrap.cc:654] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.829284 12880 consensus_queue.cc:260] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.830286 12879 tablet_bootstrap.cc:492] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850: No bootstrap required, opened a new log
I20260709 13:05:39.830938 12879 ts_tablet_manager.cc:1403] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 13:05:39.832777 12715 heartbeater.cc:499] Master 127.0.107.254:34071 was elected leader, sending a full tablet report...
I20260709 13:05:39.834916 12880 ts_tablet_manager.cc:1434] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:05:39.835062 12879 raft_consensus.cc:359] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.836097 12879 raft_consensus.cc:385] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.836462 12879 raft_consensus.cc:740] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa55094a82634ea3970f61b1897a2850, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.836861 12880 tablet_bootstrap.cc:492] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c: Bootstrap starting.
I20260709 13:05:39.837268 12879 consensus_queue.cc:260] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.838245 12882 tablet_bootstrap.cc:492] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89: No bootstrap required, opened a new log
I20260709 13:05:39.839057 12882 ts_tablet_manager.cc:1403] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 13:05:39.843732 12882 raft_consensus.cc:359] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.845055 12882 raft_consensus.cc:385] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.845599 12882 raft_consensus.cc:740] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65655c7d03b04947adcc4cf326691e89, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.846238 12880 tablet_bootstrap.cc:654] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.846884 12882 consensus_queue.cc:260] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.852448 12879 ts_tablet_manager.cc:1434] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.012s
I20260709 13:05:39.854854 12879 tablet_bootstrap.cc:492] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850: Bootstrap starting.
I20260709 13:05:39.855803 12880 tablet_bootstrap.cc:492] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c: No bootstrap required, opened a new log
I20260709 13:05:39.855772 12790 heartbeater.cc:499] Master 127.0.107.254:34071 was elected leader, sending a full tablet report...
I20260709 13:05:39.858771 12864 heartbeater.cc:499] Master 127.0.107.254:34071 was elected leader, sending a full tablet report...
I20260709 13:05:39.859426 12880 ts_tablet_manager.cc:1403] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.009s
I20260709 13:05:39.859462 12882 ts_tablet_manager.cc:1434] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89: Time spent starting tablet: real 0.020s	user 0.011s	sys 0.008s
I20260709 13:05:39.861048 12882 tablet_bootstrap.cc:492] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89: Bootstrap starting.
I20260709 13:05:39.863971 12880 raft_consensus.cc:359] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.864888 12880 raft_consensus.cc:385] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.866134 12880 raft_consensus.cc:740] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfeaf86cdbd24416a2768138d5e7488c, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.867487 12879 tablet_bootstrap.cc:654] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.867205 12880 consensus_queue.cc:260] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.870594 12882 tablet_bootstrap.cc:654] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:39.875551 12880 ts_tablet_manager.cc:1434] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 13:05:39.879994 12882 tablet_bootstrap.cc:492] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89: No bootstrap required, opened a new log
I20260709 13:05:39.880717 12882 ts_tablet_manager.cc:1403] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.008s
I20260709 13:05:39.883991 12879 tablet_bootstrap.cc:492] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850: No bootstrap required, opened a new log
I20260709 13:05:39.884284 12882 raft_consensus.cc:359] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.885258 12882 raft_consensus.cc:385] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.885454 12879 ts_tablet_manager.cc:1403] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.005s
I20260709 13:05:39.885605 12882 raft_consensus.cc:740] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65655c7d03b04947adcc4cf326691e89, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.886845 12882 consensus_queue.cc:260] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.888886 12882 ts_tablet_manager.cc:1434] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:39.889377 12879 raft_consensus.cc:359] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.890280 12879 raft_consensus.cc:385] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:39.890596 12879 raft_consensus.cc:740] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa55094a82634ea3970f61b1897a2850, State: Initialized, Role: FOLLOWER
I20260709 13:05:39.891472 12879 consensus_queue.cc:260] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.893891 12879 ts_tablet_manager.cc:1434] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:39.935680 12884 raft_consensus.cc:493] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.936329 12884 raft_consensus.cc:515] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.939278 12884 leader_election.cc:290] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa55094a82634ea3970f61b1897a2850 (127.0.107.194:43587), 65655c7d03b04947adcc4cf326691e89 (127.0.107.195:36113)
I20260709 13:05:39.939436 12887 raft_consensus.cc:493] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.940258 12887 raft_consensus.cc:515] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.946444 12887 leader_election.cc:290] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193:42407), aa55094a82634ea3970f61b1897a2850 (127.0.107.194:43587)
I20260709 13:05:39.962512 12765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f990fb0d5e841eeabe2927a4beab3ca" candidate_uuid: "cfeaf86cdbd24416a2768138d5e7488c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa55094a82634ea3970f61b1897a2850" is_pre_election: true
I20260709 13:05:39.963799 12765 raft_consensus.cc:2468] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfeaf86cdbd24416a2768138d5e7488c in term 0.
I20260709 13:05:39.964931 12839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f990fb0d5e841eeabe2927a4beab3ca" candidate_uuid: "cfeaf86cdbd24416a2768138d5e7488c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65655c7d03b04947adcc4cf326691e89" is_pre_election: true
I20260709 13:05:39.965865 12690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a44d80e37848518b2ea81687fc5320" candidate_uuid: "65655c7d03b04947adcc4cf326691e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfeaf86cdbd24416a2768138d5e7488c" is_pre_election: true
I20260709 13:05:39.966317 12839 raft_consensus.cc:2468] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfeaf86cdbd24416a2768138d5e7488c in term 0.
I20260709 13:05:39.966250 12656 leader_election.cc:304] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa55094a82634ea3970f61b1897a2850, cfeaf86cdbd24416a2768138d5e7488c; no voters: 
I20260709 13:05:39.967175 12690 raft_consensus.cc:2468] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65655c7d03b04947adcc4cf326691e89 in term 0.
I20260709 13:05:39.967841 12884 raft_consensus.cc:2804] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.968299 12884 raft_consensus.cc:493] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.968643 12884 raft_consensus.cc:3060] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.969779 12804 leader_election.cc:304] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65655c7d03b04947adcc4cf326691e89, cfeaf86cdbd24416a2768138d5e7488c; no voters: 
I20260709 13:05:39.970952 12887 raft_consensus.cc:2804] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:39.971397 12887 raft_consensus.cc:493] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.971767 12887 raft_consensus.cc:3060] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.974406 12765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a44d80e37848518b2ea81687fc5320" candidate_uuid: "65655c7d03b04947adcc4cf326691e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa55094a82634ea3970f61b1897a2850" is_pre_election: true
I20260709 13:05:39.975131 12765 raft_consensus.cc:2468] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65655c7d03b04947adcc4cf326691e89 in term 0.
I20260709 13:05:39.979306 12884 raft_consensus.cc:515] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.982287 12884 leader_election.cc:290] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [CANDIDATE]: Term 1 election: Requested vote from peers aa55094a82634ea3970f61b1897a2850 (127.0.107.194:43587), 65655c7d03b04947adcc4cf326691e89 (127.0.107.195:36113)
I20260709 13:05:39.982285 12887 raft_consensus.cc:515] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:39.983273 12765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f990fb0d5e841eeabe2927a4beab3ca" candidate_uuid: "cfeaf86cdbd24416a2768138d5e7488c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa55094a82634ea3970f61b1897a2850"
I20260709 13:05:39.983613 12839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f990fb0d5e841eeabe2927a4beab3ca" candidate_uuid: "cfeaf86cdbd24416a2768138d5e7488c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65655c7d03b04947adcc4cf326691e89"
I20260709 13:05:39.984421 12765 raft_consensus.cc:3060] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.984452 12839 raft_consensus.cc:3060] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.985992 12887 leader_election.cc:290] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [CANDIDATE]: Term 1 election: Requested vote from peers cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193:42407), aa55094a82634ea3970f61b1897a2850 (127.0.107.194:43587)
I20260709 13:05:39.986214 12690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a44d80e37848518b2ea81687fc5320" candidate_uuid: "65655c7d03b04947adcc4cf326691e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfeaf86cdbd24416a2768138d5e7488c"
I20260709 13:05:39.986863 12690 raft_consensus.cc:3060] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.987882 12764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a44d80e37848518b2ea81687fc5320" candidate_uuid: "65655c7d03b04947adcc4cf326691e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa55094a82634ea3970f61b1897a2850"
I20260709 13:05:39.988623 12764 raft_consensus.cc:3060] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:39.995333 12839 raft_consensus.cc:2468] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfeaf86cdbd24416a2768138d5e7488c in term 1.
I20260709 13:05:39.995337 12765 raft_consensus.cc:2468] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfeaf86cdbd24416a2768138d5e7488c in term 1.
I20260709 13:05:39.997195 12653 leader_election.cc:304] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65655c7d03b04947adcc4cf326691e89, cfeaf86cdbd24416a2768138d5e7488c; no voters: 
I20260709 13:05:39.998229 12884 raft_consensus.cc:2804] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:39.998808 12764 raft_consensus.cc:2468] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65655c7d03b04947adcc4cf326691e89 in term 1.
I20260709 13:05:39.998996 12690 raft_consensus.cc:2468] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65655c7d03b04947adcc4cf326691e89 in term 1.
I20260709 13:05:40.000607 12804 leader_election.cc:304] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65655c7d03b04947adcc4cf326691e89, cfeaf86cdbd24416a2768138d5e7488c; no voters: 
I20260709 13:05:40.000398 12884 raft_consensus.cc:697] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 1 LEADER]: Becoming Leader. State: Replica: cfeaf86cdbd24416a2768138d5e7488c, State: Running, Role: LEADER
I20260709 13:05:40.001735 12887 raft_consensus.cc:2804] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.002305 12884 consensus_queue.cc:237] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.003444 12887 raft_consensus.cc:697] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 1 LEADER]: Becoming Leader. State: Replica: 65655c7d03b04947adcc4cf326691e89, State: Running, Role: LEADER
I20260709 13:05:40.004455 12887 consensus_queue.cc:237] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.017262 12586 catalog_manager.cc:5697] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c reported cstate change: term changed from 0 to 1, leader changed from <none> to cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193). New cstate: current_term: 1 leader_uuid: "cfeaf86cdbd24416a2768138d5e7488c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.017568 12587 catalog_manager.cc:5697] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65655c7d03b04947adcc4cf326691e89 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "65655c7d03b04947adcc4cf326691e89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.100862 12587 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33884:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:40.103565 12587 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:40.132620 12680 tablet_service.cc:1511] Processing CreateTablet for tablet 722311fc8c7841b0ac863ccf37a51cfa (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=92f60385b1994286b318722046a351ff]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:05:40.134155 12680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 722311fc8c7841b0ac863ccf37a51cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.134057 12754 tablet_service.cc:1511] Processing CreateTablet for tablet 722311fc8c7841b0ac863ccf37a51cfa (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=92f60385b1994286b318722046a351ff]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:05:40.135695 12754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 722311fc8c7841b0ac863ccf37a51cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.135825 12829 tablet_service.cc:1511] Processing CreateTablet for tablet 722311fc8c7841b0ac863ccf37a51cfa (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=92f60385b1994286b318722046a351ff]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:05:40.137207 12829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 722311fc8c7841b0ac863ccf37a51cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.138536 12679 tablet_service.cc:1511] Processing CreateTablet for tablet 8e07cb6d61ca43fab89d6e9d3d47cef9 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=92f60385b1994286b318722046a351ff]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:05:40.139892 12679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e07cb6d61ca43fab89d6e9d3d47cef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.142586 12755 tablet_service.cc:1511] Processing CreateTablet for tablet 8e07cb6d61ca43fab89d6e9d3d47cef9 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=92f60385b1994286b318722046a351ff]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:05:40.143368 12828 tablet_service.cc:1511] Processing CreateTablet for tablet 8e07cb6d61ca43fab89d6e9d3d47cef9 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=92f60385b1994286b318722046a351ff]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:05:40.143998 12755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e07cb6d61ca43fab89d6e9d3d47cef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.145926 12828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e07cb6d61ca43fab89d6e9d3d47cef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:40.156044 12879 tablet_bootstrap.cc:492] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850: Bootstrap starting.
W20260709 13:05:40.161742 12636 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:40.162317 12880 tablet_bootstrap.cc:492] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c: Bootstrap starting.
I20260709 13:05:40.163776 12879 tablet_bootstrap.cc:654] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.171324 12880 tablet_bootstrap.cc:654] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.177106 12879 tablet_bootstrap.cc:492] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850: No bootstrap required, opened a new log
I20260709 13:05:40.177827 12879 ts_tablet_manager.cc:1403] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.010s
I20260709 13:05:40.182168 12879 raft_consensus.cc:359] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.183606 12879 raft_consensus.cc:385] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.184096 12879 raft_consensus.cc:740] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa55094a82634ea3970f61b1897a2850, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.185225 12880 tablet_bootstrap.cc:492] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c: No bootstrap required, opened a new log
I20260709 13:05:40.185922 12880 ts_tablet_manager.cc:1403] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.015s
I20260709 13:05:40.185461 12879 consensus_queue.cc:260] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.190096 12882 tablet_bootstrap.cc:492] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89: Bootstrap starting.
I20260709 13:05:40.189733 12880 raft_consensus.cc:359] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.190621 12880 raft_consensus.cc:385] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.190987 12880 raft_consensus.cc:740] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfeaf86cdbd24416a2768138d5e7488c, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.191946 12880 consensus_queue.cc:260] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.194837 12880 ts_tablet_manager.cc:1434] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:05:40.200300 12879 ts_tablet_manager.cc:1434] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:05:40.202965 12882 tablet_bootstrap.cc:654] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.207535 12880 tablet_bootstrap.cc:492] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c: Bootstrap starting.
I20260709 13:05:40.216586 12879 tablet_bootstrap.cc:492] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850: Bootstrap starting.
I20260709 13:05:40.216907 12880 tablet_bootstrap.cc:654] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.225222 12882 tablet_bootstrap.cc:492] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89: No bootstrap required, opened a new log
I20260709 13:05:40.225831 12880 tablet_bootstrap.cc:492] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c: No bootstrap required, opened a new log
I20260709 13:05:40.225870 12882 ts_tablet_manager.cc:1403] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89: Time spent bootstrapping tablet: real 0.036s	user 0.021s	sys 0.009s
I20260709 13:05:40.226487 12879 tablet_bootstrap.cc:654] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.226528 12880 ts_tablet_manager.cc:1403] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 13:05:40.229945 12882 raft_consensus.cc:359] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.230166 12880 raft_consensus.cc:359] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.231089 12882 raft_consensus.cc:385] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.231595 12880 raft_consensus.cc:385] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.231633 12882 raft_consensus.cc:740] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65655c7d03b04947adcc4cf326691e89, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.232091 12880 raft_consensus.cc:740] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfeaf86cdbd24416a2768138d5e7488c, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.233198 12880 consensus_queue.cc:260] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.233089 12882 consensus_queue.cc:260] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.235505 12879 tablet_bootstrap.cc:492] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850: No bootstrap required, opened a new log
I20260709 13:05:40.235661 12880 ts_tablet_manager.cc:1434] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:40.236186 12879 ts_tablet_manager.cc:1403] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.009s
I20260709 13:05:40.239590 12882 ts_tablet_manager.cc:1434] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 13:05:40.240269 12879 raft_consensus.cc:359] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.240970 12882 tablet_bootstrap.cc:492] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89: Bootstrap starting.
I20260709 13:05:40.240995 12879 raft_consensus.cc:385] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.241575 12879 raft_consensus.cc:740] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa55094a82634ea3970f61b1897a2850, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.242547 12879 consensus_queue.cc:260] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.248212 12879 ts_tablet_manager.cc:1434] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.009s
I20260709 13:05:40.250633 12882 tablet_bootstrap.cc:654] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:40.251345 12886 raft_consensus.cc:493] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.252097 12886 raft_consensus.cc:515] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.256951 12886 leader_election.cc:290] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193:42407), 65655c7d03b04947adcc4cf326691e89 (127.0.107.195:36113)
I20260709 13:05:40.263128 12882 tablet_bootstrap.cc:492] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89: No bootstrap required, opened a new log
I20260709 13:05:40.264111 12882 ts_tablet_manager.cc:1403] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.007s
I20260709 13:05:40.270027 12882 raft_consensus.cc:359] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.271683 12882 raft_consensus.cc:385] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:40.272032 12882 raft_consensus.cc:740] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65655c7d03b04947adcc4cf326691e89, State: Initialized, Role: FOLLOWER
I20260709 13:05:40.273201 12882 consensus_queue.cc:260] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.276077 12882 ts_tablet_manager.cc:1434] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:05:40.278198 12690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722311fc8c7841b0ac863ccf37a51cfa" candidate_uuid: "aa55094a82634ea3970f61b1897a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfeaf86cdbd24416a2768138d5e7488c" is_pre_election: true
I20260709 13:05:40.279104 12690 raft_consensus.cc:2468] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa55094a82634ea3970f61b1897a2850 in term 0.
I20260709 13:05:40.280563 12730 leader_election.cc:304] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa55094a82634ea3970f61b1897a2850, cfeaf86cdbd24416a2768138d5e7488c; no voters: 
I20260709 13:05:40.281589 12886 raft_consensus.cc:2804] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.282039 12886 raft_consensus.cc:493] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.282430 12886 raft_consensus.cc:3060] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.286365 12839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722311fc8c7841b0ac863ccf37a51cfa" candidate_uuid: "aa55094a82634ea3970f61b1897a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65655c7d03b04947adcc4cf326691e89" is_pre_election: true
I20260709 13:05:40.287329 12839 raft_consensus.cc:2468] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa55094a82634ea3970f61b1897a2850 in term 0.
I20260709 13:05:40.292279 12886 raft_consensus.cc:515] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.294569 12886 leader_election.cc:290] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [CANDIDATE]: Term 1 election: Requested vote from peers cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193:42407), 65655c7d03b04947adcc4cf326691e89 (127.0.107.195:36113)
I20260709 13:05:40.295743 12690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722311fc8c7841b0ac863ccf37a51cfa" candidate_uuid: "aa55094a82634ea3970f61b1897a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfeaf86cdbd24416a2768138d5e7488c"
I20260709 13:05:40.296607 12690 raft_consensus.cc:3060] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.296833 12839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722311fc8c7841b0ac863ccf37a51cfa" candidate_uuid: "aa55094a82634ea3970f61b1897a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65655c7d03b04947adcc4cf326691e89"
I20260709 13:05:40.297640 12839 raft_consensus.cc:3060] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.304177 12886 raft_consensus.cc:493] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.304263 12839 raft_consensus.cc:2468] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa55094a82634ea3970f61b1897a2850 in term 1.
I20260709 13:05:40.304641 12690 raft_consensus.cc:2468] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa55094a82634ea3970f61b1897a2850 in term 1.
I20260709 13:05:40.305136 12886 raft_consensus.cc:515] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.306020 12730 leader_election.cc:304] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa55094a82634ea3970f61b1897a2850, cfeaf86cdbd24416a2768138d5e7488c; no voters: 
I20260709 13:05:40.307514 12886 leader_election.cc:290] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193:42407), 65655c7d03b04947adcc4cf326691e89 (127.0.107.195:36113)
I20260709 13:05:40.308058 12913 raft_consensus.cc:2804] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.308717 12913 raft_consensus.cc:697] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 1 LEADER]: Becoming Leader. State: Replica: aa55094a82634ea3970f61b1897a2850, State: Running, Role: LEADER
I20260709 13:05:40.308969 12690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9" candidate_uuid: "aa55094a82634ea3970f61b1897a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfeaf86cdbd24416a2768138d5e7488c" is_pre_election: true
I20260709 13:05:40.308917 12839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9" candidate_uuid: "aa55094a82634ea3970f61b1897a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65655c7d03b04947adcc4cf326691e89" is_pre_election: true
I20260709 13:05:40.309827 12690 raft_consensus.cc:2468] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa55094a82634ea3970f61b1897a2850 in term 0.
I20260709 13:05:40.309963 12839 raft_consensus.cc:2468] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa55094a82634ea3970f61b1897a2850 in term 0.
I20260709 13:05:40.309731 12913 consensus_queue.cc:237] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.311080 12730 leader_election.cc:304] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa55094a82634ea3970f61b1897a2850, cfeaf86cdbd24416a2768138d5e7488c; no voters: 
I20260709 13:05:40.312583 12886 raft_consensus.cc:2804] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:40.312956 12886 raft_consensus.cc:493] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.313314 12886 raft_consensus.cc:3060] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.320916 12886 raft_consensus.cc:515] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.321159 12587 catalog_manager.cc:5697] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa55094a82634ea3970f61b1897a2850 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "aa55094a82634ea3970f61b1897a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.324379 12886 leader_election.cc:290] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [CANDIDATE]: Term 1 election: Requested vote from peers cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193:42407), 65655c7d03b04947adcc4cf326691e89 (127.0.107.195:36113)
I20260709 13:05:40.324635 12839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9" candidate_uuid: "aa55094a82634ea3970f61b1897a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65655c7d03b04947adcc4cf326691e89"
I20260709 13:05:40.324957 12690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9" candidate_uuid: "aa55094a82634ea3970f61b1897a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfeaf86cdbd24416a2768138d5e7488c"
I20260709 13:05:40.325412 12839 raft_consensus.cc:3060] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.325558 12690 raft_consensus.cc:3060] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:40.337219 12839 raft_consensus.cc:2468] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa55094a82634ea3970f61b1897a2850 in term 1.
I20260709 13:05:40.337435 12690 raft_consensus.cc:2468] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa55094a82634ea3970f61b1897a2850 in term 1.
I20260709 13:05:40.338902 12728 leader_election.cc:304] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65655c7d03b04947adcc4cf326691e89, aa55094a82634ea3970f61b1897a2850; no voters: 
I20260709 13:05:40.339761 12886 raft_consensus.cc:2804] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:40.340279 12886 raft_consensus.cc:697] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 1 LEADER]: Becoming Leader. State: Replica: aa55094a82634ea3970f61b1897a2850, State: Running, Role: LEADER
I20260709 13:05:40.341125 12886 consensus_queue.cc:237] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.349383 12587 catalog_manager.cc:5697] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa55094a82634ea3970f61b1897a2850 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "aa55094a82634ea3970f61b1897a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.418298 12764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "722311fc8c7841b0ac863ccf37a51cfa"
dest_uuid: "aa55094a82634ea3970f61b1897a2850"
mode: GRACEFUL
new_leader_uuid: "cfeaf86cdbd24416a2768138d5e7488c"
 from {username='slave'} at 127.0.0.1:50230
I20260709 13:05:40.419139 12764 raft_consensus.cc:606] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 1 LEADER]: Received request to transfer leadership to TS cfeaf86cdbd24416a2768138d5e7488c
I20260709 13:05:40.425356 12764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9"
dest_uuid: "aa55094a82634ea3970f61b1897a2850"
mode: GRACEFUL
new_leader_uuid: "cfeaf86cdbd24416a2768138d5e7488c"
 from {username='slave'} at 127.0.0.1:50230
I20260709 13:05:40.426093 12764 raft_consensus.cc:606] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 1 LEADER]: Received request to transfer leadership to TS cfeaf86cdbd24416a2768138d5e7488c
I20260709 13:05:40.431162 12884 consensus_queue.cc:1048] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:40.439889 12887 consensus_queue.cc:1048] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:40.469751 12884 consensus_queue.cc:1048] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [LEADER]: Connected to new peer: Peer: permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:40.494663 12925 consensus_queue.cc:1048] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:40.728899 12886 consensus_queue.cc:1048] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:40.736785 12913 raft_consensus.cc:993] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850: : Instructing follower cfeaf86cdbd24416a2768138d5e7488c to start an election
I20260709 13:05:40.737753 12886 raft_consensus.cc:1081] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 1 LEADER]: Signalling peer cfeaf86cdbd24416a2768138d5e7488c to start an election
I20260709 13:05:40.740056 12689 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9"
dest_uuid: "cfeaf86cdbd24416a2768138d5e7488c"
 from {username='slave'} at 127.0.0.1:44182
I20260709 13:05:40.741935 12689 raft_consensus.cc:493] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:40.742429 12689 raft_consensus.cc:3060] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.745414 12913 consensus_queue.cc:1048] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:40.752203 12689 raft_consensus.cc:515] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.758356 12765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9" candidate_uuid: "cfeaf86cdbd24416a2768138d5e7488c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa55094a82634ea3970f61b1897a2850"
I20260709 13:05:40.758515 12689 leader_election.cc:290] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [CANDIDATE]: Term 2 election: Requested vote from peers aa55094a82634ea3970f61b1897a2850 (127.0.107.194:43587), 65655c7d03b04947adcc4cf326691e89 (127.0.107.195:36113)
I20260709 13:05:40.759157 12765 raft_consensus.cc:3055] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:40.759627 12765 raft_consensus.cc:740] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aa55094a82634ea3970f61b1897a2850, State: Running, Role: LEADER
I20260709 13:05:40.760679 12765 consensus_queue.cc:260] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.762136 12838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9" candidate_uuid: "cfeaf86cdbd24416a2768138d5e7488c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "65655c7d03b04947adcc4cf326691e89"
I20260709 13:05:40.762830 12838 raft_consensus.cc:3060] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.762214 12765 raft_consensus.cc:3060] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.772419 12838 raft_consensus.cc:2468] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfeaf86cdbd24416a2768138d5e7488c in term 2.
I20260709 13:05:40.772707 12765 raft_consensus.cc:2468] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfeaf86cdbd24416a2768138d5e7488c in term 2.
I20260709 13:05:40.773972 12653 leader_election.cc:304] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65655c7d03b04947adcc4cf326691e89, cfeaf86cdbd24416a2768138d5e7488c; no voters: 
I20260709 13:05:40.775378 12895 raft_consensus.cc:2804] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.775877 12895 raft_consensus.cc:697] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 2 LEADER]: Becoming Leader. State: Replica: cfeaf86cdbd24416a2768138d5e7488c, State: Running, Role: LEADER
I20260709 13:05:40.776099 12913 consensus_queue.cc:1048] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:40.776940 12895 consensus_queue.cc:237] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.793190 12886 consensus_queue.cc:1048] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:40.792227 12584 catalog_manager.cc:5697] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c reported cstate change: term changed from 1 to 2, leader changed from aa55094a82634ea3970f61b1897a2850 (127.0.107.194) to cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193). New cstate: current_term: 2 leader_uuid: "cfeaf86cdbd24416a2768138d5e7488c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.806584 12886 raft_consensus.cc:993] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850: : Instructing follower cfeaf86cdbd24416a2768138d5e7488c to start an election
I20260709 13:05:40.807195 12886 raft_consensus.cc:1081] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 1 LEADER]: Signalling peer cfeaf86cdbd24416a2768138d5e7488c to start an election
I20260709 13:05:40.809055 12689 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "722311fc8c7841b0ac863ccf37a51cfa"
dest_uuid: "cfeaf86cdbd24416a2768138d5e7488c"
 from {username='slave'} at 127.0.0.1:44182
I20260709 13:05:40.809753 12689 raft_consensus.cc:493] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:40.810122 12689 raft_consensus.cc:3060] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.816969 12689 raft_consensus.cc:515] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.819243 12689 leader_election.cc:290] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [CANDIDATE]: Term 2 election: Requested vote from peers aa55094a82634ea3970f61b1897a2850 (127.0.107.194:43587), 65655c7d03b04947adcc4cf326691e89 (127.0.107.195:36113)
I20260709 13:05:40.820345 12765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722311fc8c7841b0ac863ccf37a51cfa" candidate_uuid: "cfeaf86cdbd24416a2768138d5e7488c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa55094a82634ea3970f61b1897a2850"
I20260709 13:05:40.820638 12838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722311fc8c7841b0ac863ccf37a51cfa" candidate_uuid: "cfeaf86cdbd24416a2768138d5e7488c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "65655c7d03b04947adcc4cf326691e89"
I20260709 13:05:40.821344 12765 raft_consensus.cc:3055] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:40.821347 12838 raft_consensus.cc:3060] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.821970 12765 raft_consensus.cc:740] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aa55094a82634ea3970f61b1897a2850, State: Running, Role: LEADER
I20260709 13:05:40.822907 12765 consensus_queue.cc:260] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.824033 12765 raft_consensus.cc:3060] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.829205 12838 raft_consensus.cc:2468] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfeaf86cdbd24416a2768138d5e7488c in term 2.
I20260709 13:05:40.830080 12765 raft_consensus.cc:2468] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfeaf86cdbd24416a2768138d5e7488c in term 2.
I20260709 13:05:40.830703 12653 leader_election.cc:304] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65655c7d03b04947adcc4cf326691e89, cfeaf86cdbd24416a2768138d5e7488c; no voters: 
I20260709 13:05:40.831653 12895 raft_consensus.cc:2804] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.832212 12895 raft_consensus.cc:697] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 2 LEADER]: Becoming Leader. State: Replica: cfeaf86cdbd24416a2768138d5e7488c, State: Running, Role: LEADER
I20260709 13:05:40.833057 12895 consensus_queue.cc:237] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:40.840155 12584 catalog_manager.cc:5697] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c reported cstate change: term changed from 1 to 2, leader changed from aa55094a82634ea3970f61b1897a2850 (127.0.107.194) to cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193). New cstate: current_term: 2 leader_uuid: "cfeaf86cdbd24416a2768138d5e7488c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.959513   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:05:40.961687   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:05:40.962179   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:05:40.976792 12689 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9"
dest_uuid: "cfeaf86cdbd24416a2768138d5e7488c"
mode: GRACEFUL
new_leader_uuid: "aa55094a82634ea3970f61b1897a2850"
 from {username='slave'} at 127.0.0.1:44188
I20260709 13:05:40.977548 12689 raft_consensus.cc:606] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 2 LEADER]: Received request to transfer leadership to TS aa55094a82634ea3970f61b1897a2850
I20260709 13:05:40.978996   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:05:41.227072 12765 raft_consensus.cc:1275] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 2 FOLLOWER]: Refusing update from remote peer cfeaf86cdbd24416a2768138d5e7488c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.228591 12935 consensus_queue.cc:1048] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.237744 12838 raft_consensus.cc:1275] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [term 2 FOLLOWER]: Refusing update from remote peer cfeaf86cdbd24416a2768138d5e7488c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.240154 12935 consensus_queue.cc:1048] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [LEADER]: Connected to new peer: Peer: permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.317586 12838 raft_consensus.cc:1275] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 2 FOLLOWER]: Refusing update from remote peer cfeaf86cdbd24416a2768138d5e7488c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.319109 12935 consensus_queue.cc:1048] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [LEADER]: Connected to new peer: Peer: permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.328491 12765 raft_consensus.cc:1275] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 2 FOLLOWER]: Refusing update from remote peer cfeaf86cdbd24416a2768138d5e7488c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.331228 12935 consensus_queue.cc:1048] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:41.343103 12935 raft_consensus.cc:993] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c: : Instructing follower aa55094a82634ea3970f61b1897a2850 to start an election
I20260709 13:05:41.343657 12922 raft_consensus.cc:1081] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 2 LEADER]: Signalling peer aa55094a82634ea3970f61b1897a2850 to start an election
I20260709 13:05:41.345952 12765 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9"
dest_uuid: "aa55094a82634ea3970f61b1897a2850"
 from {username='slave'} at 127.0.0.1:50218
I20260709 13:05:41.346947 12765 raft_consensus.cc:493] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:41.347395 12765 raft_consensus.cc:3060] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.357369 12765 raft_consensus.cc:515] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:41.360039 12765 leader_election.cc:290] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [CANDIDATE]: Term 3 election: Requested vote from peers cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193:42407), 65655c7d03b04947adcc4cf326691e89 (127.0.107.195:36113)
I20260709 13:05:41.361032 12689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9" candidate_uuid: "aa55094a82634ea3970f61b1897a2850" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cfeaf86cdbd24416a2768138d5e7488c"
I20260709 13:05:41.361222 12838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e07cb6d61ca43fab89d6e9d3d47cef9" candidate_uuid: "aa55094a82634ea3970f61b1897a2850" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "65655c7d03b04947adcc4cf326691e89"
I20260709 13:05:41.361910 12689 raft_consensus.cc:3055] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:41.362012 12838 raft_consensus.cc:3060] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.362231 12689 raft_consensus.cc:740] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cfeaf86cdbd24416a2768138d5e7488c, State: Running, Role: LEADER
I20260709 13:05:41.363318 12689 consensus_queue.cc:260] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:41.365077 12689 raft_consensus.cc:3060] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.370075 12838 raft_consensus.cc:2468] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa55094a82634ea3970f61b1897a2850 in term 3.
I20260709 13:05:41.371868 12728 leader_election.cc:304] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65655c7d03b04947adcc4cf326691e89, aa55094a82634ea3970f61b1897a2850; no voters: 
I20260709 13:05:41.372234 12689 raft_consensus.cc:2468] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa55094a82634ea3970f61b1897a2850 in term 3.
I20260709 13:05:41.372798 12886 raft_consensus.cc:2804] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:41.374051 12886 raft_consensus.cc:697] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 3 LEADER]: Becoming Leader. State: Replica: aa55094a82634ea3970f61b1897a2850, State: Running, Role: LEADER
I20260709 13:05:41.375114 12886 consensus_queue.cc:237] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } }
I20260709 13:05:41.383387 12585 catalog_manager.cc:5697] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 reported cstate change: term changed from 2 to 3, leader changed from cfeaf86cdbd24416a2768138d5e7488c (127.0.107.193) to aa55094a82634ea3970f61b1897a2850 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "aa55094a82634ea3970f61b1897a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfeaf86cdbd24416a2768138d5e7488c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa55094a82634ea3970f61b1897a2850" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65655c7d03b04947adcc4cf326691e89" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.625392   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:05:41.648375   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:41.649569   431 tablet_replica.cc:333] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c: stopping tablet replica
I20260709 13:05:41.650399   431 raft_consensus.cc:2243] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:41.651460   431 raft_consensus.cc:2272] T 722311fc8c7841b0ac863ccf37a51cfa P cfeaf86cdbd24416a2768138d5e7488c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.654294   431 tablet_replica.cc:333] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c: stopping tablet replica
I20260709 13:05:41.654954   431 raft_consensus.cc:2243] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:41.655445   431 raft_consensus.cc:2272] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P cfeaf86cdbd24416a2768138d5e7488c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.658387   431 tablet_replica.cc:333] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c: stopping tablet replica
I20260709 13:05:41.659099   431 raft_consensus.cc:2243] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:41.660027   431 raft_consensus.cc:2272] T 7f990fb0d5e841eeabe2927a4beab3ca P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.662416   431 tablet_replica.cc:333] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c: stopping tablet replica
I20260709 13:05:41.662989   431 raft_consensus.cc:2243] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:41.663482   431 raft_consensus.cc:2272] T 28a44d80e37848518b2ea81687fc5320 P cfeaf86cdbd24416a2768138d5e7488c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.689038   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:05:41.705742   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:05:41.726048   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:41.726935   431 tablet_replica.cc:333] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850: stopping tablet replica
I20260709 13:05:41.727574   431 raft_consensus.cc:2243] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:41.728135   431 raft_consensus.cc:2272] T 722311fc8c7841b0ac863ccf37a51cfa P aa55094a82634ea3970f61b1897a2850 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.730660   431 tablet_replica.cc:333] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850: stopping tablet replica
I20260709 13:05:41.731350   431 raft_consensus.cc:2243] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:41.732175   431 pending_rounds.cc:70] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850: Trying to abort 1 pending ops.
I20260709 13:05:41.732342   431 pending_rounds.cc:77] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635190281428992 op_type: NO_OP noop_request { }
I20260709 13:05:41.732695   431 raft_consensus.cc:2889] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:41.732990   431 raft_consensus.cc:2272] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P aa55094a82634ea3970f61b1897a2850 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.735505   431 tablet_replica.cc:333] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850: stopping tablet replica
I20260709 13:05:41.736094   431 raft_consensus.cc:2243] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:41.736564   431 raft_consensus.cc:2272] T 7f990fb0d5e841eeabe2927a4beab3ca P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.738692   431 tablet_replica.cc:333] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850: stopping tablet replica
I20260709 13:05:41.739207   431 raft_consensus.cc:2243] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:41.739588   431 raft_consensus.cc:2272] T 28a44d80e37848518b2ea81687fc5320 P aa55094a82634ea3970f61b1897a2850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.766917   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:05:41.782348   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:05:41.804309   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:41.805203   431 tablet_replica.cc:333] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89: stopping tablet replica
I20260709 13:05:41.805979   431 raft_consensus.cc:2243] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:41.806644   431 raft_consensus.cc:2272] T 722311fc8c7841b0ac863ccf37a51cfa P 65655c7d03b04947adcc4cf326691e89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.809432   431 tablet_replica.cc:333] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89: stopping tablet replica
I20260709 13:05:41.810087   431 raft_consensus.cc:2243] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:41.810541   431 raft_consensus.cc:2272] T 8e07cb6d61ca43fab89d6e9d3d47cef9 P 65655c7d03b04947adcc4cf326691e89 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.813099   431 tablet_replica.cc:333] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89: stopping tablet replica
I20260709 13:05:41.813804   431 raft_consensus.cc:2243] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:41.814922   431 raft_consensus.cc:2272] T 28a44d80e37848518b2ea81687fc5320 P 65655c7d03b04947adcc4cf326691e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.817239   431 tablet_replica.cc:333] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89: stopping tablet replica
I20260709 13:05:41.817848   431 raft_consensus.cc:2243] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:41.818279   431 raft_consensus.cc:2272] T 7f990fb0d5e841eeabe2927a4beab3ca P 65655c7d03b04947adcc4cf326691e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.842473   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:05:41.857793   431 master.cc:562] Master@127.0.107.254:34071 shutting down...
I20260709 13:05:41.878093   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:41.878770   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:41.879125   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6131031cfd3a47a6a64754fd57610e37: stopping tablet replica
I20260709 13:05:41.901669   431 master.cc:584] Master@127.0.107.254:34071 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2935 ms)
[----------] 6 tests from LeaderRebalancerTest (416809 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:05:41.933066   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:34951
I20260709 13:05:41.934381   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:41.940042 12940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:41.940382 12941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:41.941849 12943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:41.943132   431 server_base.cc:1061] running on GCE node
I20260709 13:05:41.944145   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:41.944554   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:41.944720   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602341944703 us; error 0 us; skew 500 ppm
I20260709 13:05:41.945336   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:41.947965   431 webserver.cc:533] Webserver started at http://127.0.107.254:34149/ using document root <none> and password file <none>
I20260709 13:05:41.948501   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:41.948731   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:41.949028   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:41.950259   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/master-0-root/instance:
uuid: "4a599bdb73ef4628b5466f030ec99bac"
format_stamp: "Formatted at 2026-07-09 13:05:41 on dist-test-slave-l8g5"
I20260709 13:05:41.954952   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:05:41.958652 12948 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:41.959652   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:05:41.960036   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/master-0-root
uuid: "4a599bdb73ef4628b5466f030ec99bac"
format_stamp: "Formatted at 2026-07-09 13:05:41 on dist-test-slave-l8g5"
I20260709 13:05:41.960367   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:41.977756   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:41.979031   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:42.025839   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:34951
I20260709 13:05:42.025929 12999 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:34951 every 8 connection(s)
I20260709 13:05:42.030938 13000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.044011 13000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac: Bootstrap starting.
I20260709 13:05:42.049898 13000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:42.054893 13000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac: No bootstrap required, opened a new log
I20260709 13:05:42.057349 13000 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a599bdb73ef4628b5466f030ec99bac" member_type: VOTER }
I20260709 13:05:42.057807 13000 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:42.058080 13000 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a599bdb73ef4628b5466f030ec99bac, State: Initialized, Role: FOLLOWER
I20260709 13:05:42.058825 13000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a599bdb73ef4628b5466f030ec99bac" member_type: VOTER }
I20260709 13:05:42.059365 13000 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:42.059590 13000 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:42.060102 13000 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:42.066645 13000 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a599bdb73ef4628b5466f030ec99bac" member_type: VOTER }
I20260709 13:05:42.067348 13000 leader_election.cc:304] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a599bdb73ef4628b5466f030ec99bac; no voters: 
I20260709 13:05:42.068737 13000 leader_election.cc:290] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:42.069177 13003 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:42.071502 13003 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [term 1 LEADER]: Becoming Leader. State: Replica: 4a599bdb73ef4628b5466f030ec99bac, State: Running, Role: LEADER
I20260709 13:05:42.072235 13003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a599bdb73ef4628b5466f030ec99bac" member_type: VOTER }
I20260709 13:05:42.072520 13000 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:42.075460 13004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a599bdb73ef4628b5466f030ec99bac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a599bdb73ef4628b5466f030ec99bac" member_type: VOTER } }
I20260709 13:05:42.075464 13005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a599bdb73ef4628b5466f030ec99bac. Latest consensus state: current_term: 1 leader_uuid: "4a599bdb73ef4628b5466f030ec99bac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a599bdb73ef4628b5466f030ec99bac" member_type: VOTER } }
I20260709 13:05:42.076156 13004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:42.076184 13005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:42.081882 13008 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:42.088176 13008 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:42.092161   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:42.100623 13008 catalog_manager.cc:1383] Generated new cluster ID: edc25015db9f4299855e55a351468f0c
I20260709 13:05:42.100912 13008 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:42.133144 13008 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:42.134964 13008 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:42.159221 13008 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac: Generated new TSK 0
I20260709 13:05:42.159982 13008 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:42.228657   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:42.236583 13021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:42.238201 13022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:42.240034 13024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:42.240809   431 server_base.cc:1061] running on GCE node
I20260709 13:05:42.241914   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:42.242105   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:42.242234   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602342242217 us; error 0 us; skew 500 ppm
I20260709 13:05:42.242767   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:42.245465   431 webserver.cc:533] Webserver started at http://127.0.107.193:37863/ using document root <none> and password file <none>
I20260709 13:05:42.246052   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:42.246239   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:42.246495   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:42.247714   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-0-root/instance:
uuid: "197e5fb7a05e45069d5fe5acb8484e73"
format_stamp: "Formatted at 2026-07-09 13:05:42 on dist-test-slave-l8g5"
I20260709 13:05:42.253219   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:05:42.257858 13029 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:42.258975   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:05:42.259444   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-0-root
uuid: "197e5fb7a05e45069d5fe5acb8484e73"
format_stamp: "Formatted at 2026-07-09 13:05:42 on dist-test-slave-l8g5"
I20260709 13:05:42.259757   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:42.276460   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:42.277921   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:42.279645   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:42.282588   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:42.282840   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:42.283118   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:42.283298   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:42.332127   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:34185
I20260709 13:05:42.332260 13091 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:34185 every 8 connection(s)
I20260709 13:05:42.338589   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:42.346808 13096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:42.348424 13097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:42.350438 13099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:42.351429   431 server_base.cc:1061] running on GCE node
I20260709 13:05:42.352814 13092 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34951
I20260709 13:05:42.353330 13092 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:42.353765   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:42.354061   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:42.354267   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602342354249 us; error 0 us; skew 500 ppm
I20260709 13:05:42.354413 13092 heartbeater.cc:507] Master 127.0.107.254:34951 requested a full tablet report, sending...
I20260709 13:05:42.355093   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:42.357338 12965 ts_manager.cc:194] Registered new tserver with Master: 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185)
I20260709 13:05:42.358438   431 webserver.cc:533] Webserver started at http://127.0.107.194:35125/ using document root <none> and password file <none>
I20260709 13:05:42.359050   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:42.359258   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:42.359660   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:42.359899 12965 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57734
I20260709 13:05:42.361276   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-1-root/instance:
uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
format_stamp: "Formatted at 2026-07-09 13:05:42 on dist-test-slave-l8g5"
I20260709 13:05:42.366907   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:05:42.370981 13104 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:42.372134   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:05:42.372583   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-1-root
uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
format_stamp: "Formatted at 2026-07-09 13:05:42 on dist-test-slave-l8g5"
I20260709 13:05:42.373024   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:42.396713   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:42.398126   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:42.399920   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:42.402951   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:42.403210   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:42.403524   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:42.403733   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:42.450984   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:42761
I20260709 13:05:42.451084 13166 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:42761 every 8 connection(s)
I20260709 13:05:42.457118   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:42.465515 13170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:42.465457 13171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:42.469105 13173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:42.469388 13167 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34951
I20260709 13:05:42.469910 13167 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:42.470223   431 server_base.cc:1061] running on GCE node
I20260709 13:05:42.470935 13167 heartbeater.cc:507] Master 127.0.107.254:34951 requested a full tablet report, sending...
I20260709 13:05:42.471326   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:42.471590   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:42.471784   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602342471738 us; error 0 us; skew 500 ppm
I20260709 13:05:42.472600   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:42.473673 12965 ts_manager.cc:194] Registered new tserver with Master: e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:42.475721   431 webserver.cc:533] Webserver started at http://127.0.107.195:42561/ using document root <none> and password file <none>
I20260709 13:05:42.475840 12965 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57744
I20260709 13:05:42.476492   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:42.476811   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:42.477206   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:42.478564   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-2-root/instance:
uuid: "c7674c4e544d475aaae24558660059ee"
format_stamp: "Formatted at 2026-07-09 13:05:42 on dist-test-slave-l8g5"
I20260709 13:05:42.484453   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:05:42.488474 13178 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:42.489365   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:05:42.489827   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-2-root
uuid: "c7674c4e544d475aaae24558660059ee"
format_stamp: "Formatted at 2026-07-09 13:05:42 on dist-test-slave-l8g5"
I20260709 13:05:42.490185   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601924925497-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:42.520262   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:42.521673   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:42.523430   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:42.528539   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:42.528892   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:42.529232   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:42.529547   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:42.605026   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:34911
I20260709 13:05:42.605244 13240 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:34911 every 8 connection(s)
I20260709 13:05:42.623664 13241 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34951
I20260709 13:05:42.624181 13241 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:42.625206 13241 heartbeater.cc:507] Master 127.0.107.254:34951 requested a full tablet report, sending...
I20260709 13:05:42.627948 12965 ts_manager.cc:194] Registered new tserver with Master: c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:42.628340   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019054884s
I20260709 13:05:42.630084 12965 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57748
I20260709 13:05:42.664026 12965 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57762:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:42.667160 12965 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:42.761384 13206 tablet_service.cc:1511] Processing CreateTablet for tablet d9324f89bc5b4281ac6b495fde2aa95c (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:42.763160 13206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9324f89bc5b4281ac6b495fde2aa95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.762940 13205 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd0304336bd4f0994a5a44bef55efdf (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:42.763572 13057 tablet_service.cc:1511] Processing CreateTablet for tablet d9324f89bc5b4281ac6b495fde2aa95c (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:42.764717 13205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd0304336bd4f0994a5a44bef55efdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.765018 13057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9324f89bc5b4281ac6b495fde2aa95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.766690 13204 tablet_service.cc:1511] Processing CreateTablet for tablet 240ee459545f4baca128c935ccbf9645 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:42.768321 13204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240ee459545f4baca128c935ccbf9645. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.770720 13132 tablet_service.cc:1511] Processing CreateTablet for tablet d9324f89bc5b4281ac6b495fde2aa95c (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:42.772445 13132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9324f89bc5b4281ac6b495fde2aa95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.775177 13131 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd0304336bd4f0994a5a44bef55efdf (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:42.778391 13130 tablet_service.cc:1511] Processing CreateTablet for tablet 240ee459545f4baca128c935ccbf9645 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:42.781524 13129 tablet_service.cc:1511] Processing CreateTablet for tablet 73438a7711574bb9a2ade51d2e041139 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:42.784528 13128 tablet_service.cc:1511] Processing CreateTablet for tablet cb4fd6fa94e8418ab089fec0bc1a633d (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:42.799065 13056 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd0304336bd4f0994a5a44bef55efdf (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:42.800691 13056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd0304336bd4f0994a5a44bef55efdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.802181 13131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd0304336bd4f0994a5a44bef55efdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.803218 13054 tablet_service.cc:1511] Processing CreateTablet for tablet 73438a7711574bb9a2ade51d2e041139 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:42.804702 13054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73438a7711574bb9a2ade51d2e041139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.806491 13055 tablet_service.cc:1511] Processing CreateTablet for tablet 240ee459545f4baca128c935ccbf9645 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:42.808041 13055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240ee459545f4baca128c935ccbf9645. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.807680 13127 tablet_service.cc:1511] Processing CreateTablet for tablet 17700ef9cb6146169ead3706cd5ca107 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:42.808741 13203 tablet_service.cc:1511] Processing CreateTablet for tablet 73438a7711574bb9a2ade51d2e041139 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:42.810326 13203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73438a7711574bb9a2ade51d2e041139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.806510 13129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73438a7711574bb9a2ade51d2e041139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.814162 13127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17700ef9cb6146169ead3706cd5ca107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.816217 13128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb4fd6fa94e8418ab089fec0bc1a633d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.821923 13130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240ee459545f4baca128c935ccbf9645. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.830013 13126 tablet_service.cc:1511] Processing CreateTablet for tablet b6667f62e97949e582d8cc92d1e3c720 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:42.831768 13126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6667f62e97949e582d8cc92d1e3c720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.834635 13202 tablet_service.cc:1511] Processing CreateTablet for tablet cb4fd6fa94e8418ab089fec0bc1a633d (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:42.836179 13202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb4fd6fa94e8418ab089fec0bc1a633d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.838652 13201 tablet_service.cc:1511] Processing CreateTablet for tablet 17700ef9cb6146169ead3706cd5ca107 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:42.840237 13201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17700ef9cb6146169ead3706cd5ca107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.842702 13124 tablet_service.cc:1511] Processing CreateTablet for tablet 95b0957f35a54bc196a443411300ffc4 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:42.844269 13124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b0957f35a54bc196a443411300ffc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.845520 13053 tablet_service.cc:1511] Processing CreateTablet for tablet cb4fd6fa94e8418ab089fec0bc1a633d (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:42.846586 13200 tablet_service.cc:1511] Processing CreateTablet for tablet b6667f62e97949e582d8cc92d1e3c720 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:42.848029 13200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6667f62e97949e582d8cc92d1e3c720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.850528 13051 tablet_service.cc:1511] Processing CreateTablet for tablet b6667f62e97949e582d8cc92d1e3c720 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:42.851570 13199 tablet_service.cc:1511] Processing CreateTablet for tablet cc4c9edf710c40b1a3105b6d94a01ddb (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:42.852483 13050 tablet_service.cc:1511] Processing CreateTablet for tablet cc4c9edf710c40b1a3105b6d94a01ddb (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:42.853083 13199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4c9edf710c40b1a3105b6d94a01ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.854384 13049 tablet_service.cc:1511] Processing CreateTablet for tablet 95b0957f35a54bc196a443411300ffc4 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:42.854975 13125 tablet_service.cc:1511] Processing CreateTablet for tablet cc4c9edf710c40b1a3105b6d94a01ddb (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:42.856314 13125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4c9edf710c40b1a3105b6d94a01ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.850534 13198 tablet_service.cc:1511] Processing CreateTablet for tablet 95b0957f35a54bc196a443411300ffc4 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:42.847653 13052 tablet_service.cc:1511] Processing CreateTablet for tablet 17700ef9cb6146169ead3706cd5ca107 (DEFAULT_TABLE table=test-workload [id=cfedb4ae4c414688aead26b848d2d315]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:42.859872 13051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6667f62e97949e582d8cc92d1e3c720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.860587 13049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b0957f35a54bc196a443411300ffc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.863085 13050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4c9edf710c40b1a3105b6d94a01ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.865936 13053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb4fd6fa94e8418ab089fec0bc1a633d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.870234 13052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17700ef9cb6146169ead3706cd5ca107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.882081 13198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b0957f35a54bc196a443411300ffc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:42.888593 13261 tablet_bootstrap.cc:492] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:42.945204 13261 tablet_bootstrap.cc:654] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.030804 13260 tablet_bootstrap.cc:492] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:43.036271 13259 tablet_bootstrap.cc:492] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:43.038242 13261 tablet_bootstrap.cc:492] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:43.039155 13261 ts_tablet_manager.cc:1403] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.151s	user 0.021s	sys 0.006s
I20260709 13:05:43.047335 13261 raft_consensus.cc:359] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:43.048332 13261 raft_consensus.cc:385] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.048702 13261 raft_consensus.cc:740] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.049829 13261 consensus_queue.cc:260] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:43.058455 13260 tablet_bootstrap.cc:654] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.066085 13259 tablet_bootstrap.cc:654] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.066434 13092 heartbeater.cc:499] Master 127.0.107.254:34951 was elected leader, sending a full tablet report...
I20260709 13:05:43.067440 13260 tablet_bootstrap.cc:492] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:43.067488 13261 ts_tablet_manager.cc:1434] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.028s	user 0.001s	sys 0.021s
I20260709 13:05:43.068106 13260 ts_tablet_manager.cc:1403] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.000s
I20260709 13:05:43.068886 13261 tablet_bootstrap.cc:492] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:43.071861 13260 raft_consensus.cc:359] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.072784 13260 raft_consensus.cc:385] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.073185 13260 raft_consensus.cc:740] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.074358 13260 consensus_queue.cc:260] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.077965 13261 tablet_bootstrap.cc:654] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:43.088735 13013 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:43.105625 13260 ts_tablet_manager.cc:1434] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.037s	user 0.015s	sys 0.021s
I20260709 13:05:43.106504 13261 tablet_bootstrap.cc:492] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:43.106623 13259 tablet_bootstrap.cc:492] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:43.106655 13167 heartbeater.cc:499] Master 127.0.107.254:34951 was elected leader, sending a full tablet report...
I20260709 13:05:43.107225 13261 ts_tablet_manager.cc:1403] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.011s
I20260709 13:05:43.107447 13259 ts_tablet_manager.cc:1403] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.072s	user 0.026s	sys 0.021s
I20260709 13:05:43.110720 13261 raft_consensus.cc:359] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.111024 13259 raft_consensus.cc:359] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.112106 13259 raft_consensus.cc:385] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.112527 13259 raft_consensus.cc:740] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.111672 13261 raft_consensus.cc:385] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.113395 13259 consensus_queue.cc:260] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.114341 13261 raft_consensus.cc:740] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.112106 13260 tablet_bootstrap.cc:492] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:43.115362 13261 consensus_queue.cc:260] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.122572 13261 ts_tablet_manager.cc:1434] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:05:43.124013 13261 tablet_bootstrap.cc:492] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:43.125449 13260 tablet_bootstrap.cc:654] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.127375 13241 heartbeater.cc:499] Master 127.0.107.254:34951 was elected leader, sending a full tablet report...
I20260709 13:05:43.127708 13259 ts_tablet_manager.cc:1434] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.020s	user 0.013s	sys 0.003s
I20260709 13:05:43.129024 13259 tablet_bootstrap.cc:492] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:43.130952 13261 tablet_bootstrap.cc:654] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.135846 13259 tablet_bootstrap.cc:654] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.150095 13259 tablet_bootstrap.cc:492] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:43.151079 13259 ts_tablet_manager.cc:1403] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.003s
I20260709 13:05:43.155488 13259 raft_consensus.cc:359] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:43.156863 13259 raft_consensus.cc:385] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.157341 13259 raft_consensus.cc:740] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.158687 13259 consensus_queue.cc:260] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:43.162139 13259 ts_tablet_manager.cc:1434] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:05:43.163663 13259 tablet_bootstrap.cc:492] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:43.170609 13260 tablet_bootstrap.cc:492] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:43.170627 13261 tablet_bootstrap.cc:492] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:43.171747 13260 ts_tablet_manager.cc:1403] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.060s	user 0.027s	sys 0.020s
I20260709 13:05:43.171818 13259 tablet_bootstrap.cc:654] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.175873 13260 raft_consensus.cc:359] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.176985 13260 raft_consensus.cc:385] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.177433 13260 raft_consensus.cc:740] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.171818 13261 ts_tablet_manager.cc:1403] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.048s	user 0.034s	sys 0.010s
I20260709 13:05:43.179162 13260 consensus_queue.cc:260] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.184376 13259 tablet_bootstrap.cc:492] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:43.182788 13261 raft_consensus.cc:359] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.185101 13261 raft_consensus.cc:385] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.185132 13259 ts_tablet_manager.cc:1403] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.005s
I20260709 13:05:43.185382 13260 ts_tablet_manager.cc:1434] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.004s
I20260709 13:05:43.185674 13261 raft_consensus.cc:740] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.186900 13261 consensus_queue.cc:260] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.186926 13260 tablet_bootstrap.cc:492] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:43.189003 13259 raft_consensus.cc:359] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.189749 13261 ts_tablet_manager.cc:1434] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:05:43.189934 13259 raft_consensus.cc:385] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.190284 13259 raft_consensus.cc:740] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.191062 13259 consensus_queue.cc:260] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.193581 13259 ts_tablet_manager.cc:1434] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:05:43.194766 13259 tablet_bootstrap.cc:492] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:43.196199 13260 tablet_bootstrap.cc:654] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.199680 13261 tablet_bootstrap.cc:492] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:43.201314 13259 tablet_bootstrap.cc:654] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.212556 13260 tablet_bootstrap.cc:492] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:43.213538 13260 ts_tablet_manager.cc:1403] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.009s
I20260709 13:05:43.214483 13261 tablet_bootstrap.cc:654] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.216600 13259 tablet_bootstrap.cc:492] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:43.216702 13260 raft_consensus.cc:359] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:43.217262 13259 ts_tablet_manager.cc:1403] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.007s
I20260709 13:05:43.217850 13260 raft_consensus.cc:385] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.218215 13260 raft_consensus.cc:740] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.219097 13260 consensus_queue.cc:260] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:43.220346 13261 tablet_bootstrap.cc:492] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:43.220973 13261 ts_tablet_manager.cc:1403] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.004s
I20260709 13:05:43.221623 13260 ts_tablet_manager.cc:1434] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:05:43.221009 13259 raft_consensus.cc:359] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.223129 13259 raft_consensus.cc:385] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.223490 13259 raft_consensus.cc:740] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.223793 13260 tablet_bootstrap.cc:492] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:43.224531 13259 consensus_queue.cc:260] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.224673 13261 raft_consensus.cc:359] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.225672 13261 raft_consensus.cc:385] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.226058 13261 raft_consensus.cc:740] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.227070 13261 consensus_queue.cc:260] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.232266 13261 ts_tablet_manager.cc:1434] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:05:43.232431 13260 tablet_bootstrap.cc:654] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.233640 13261 tablet_bootstrap.cc:492] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:43.236253 13259 ts_tablet_manager.cc:1434] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:05:43.242350 13259 tablet_bootstrap.cc:492] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:43.250028 13261 tablet_bootstrap.cc:654] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.251351 13268 raft_consensus.cc:493] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.252259 13268 raft_consensus.cc:515] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.254741 13259 tablet_bootstrap.cc:654] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.254743 13263 raft_consensus.cc:493] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.255990 13260 tablet_bootstrap.cc:492] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:43.255610 13263 raft_consensus.cc:515] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.256834 13261 tablet_bootstrap.cc:492] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:43.257416 13261 ts_tablet_manager.cc:1403] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.000s
I20260709 13:05:43.256606 13260 ts_tablet_manager.cc:1403] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.013s
I20260709 13:05:43.267421 13263 leader_election.cc:290] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.259013 13268 leader_election.cc:290] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:43.268317 13261 raft_consensus.cc:359] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.272692 13261 raft_consensus.cc:385] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.273276 13261 raft_consensus.cc:740] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.274718 13268 raft_consensus.cc:493] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.274763 13261 consensus_queue.cc:260] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.275403 13268 raft_consensus.cc:515] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:43.278488 13260 raft_consensus.cc:359] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.279273 13268 leader_election.cc:290] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:43.279660 13260 raft_consensus.cc:385] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.286295 13260 raft_consensus.cc:740] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.286914 13263 raft_consensus.cc:493] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.287469 13263 raft_consensus.cc:515] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.287380 13260 consensus_queue.cc:260] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.290321 13260 ts_tablet_manager.cc:1434] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.001s
I20260709 13:05:43.290426 13263 leader_election.cc:290] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.291560 13260 tablet_bootstrap.cc:492] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:43.294804 13259 tablet_bootstrap.cc:492] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:43.295581 13259 ts_tablet_manager.cc:1403] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.009s
I20260709 13:05:43.299417 13259 raft_consensus.cc:359] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.300357 13259 raft_consensus.cc:385] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.300738 13259 raft_consensus.cc:740] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.301151 13260 tablet_bootstrap.cc:654] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.301788 13259 consensus_queue.cc:260] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.285670 13261 ts_tablet_manager.cc:1434] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.003s
I20260709 13:05:43.305652 13261 tablet_bootstrap.cc:492] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:43.304481 13259 ts_tablet_manager.cc:1434] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:05:43.308526 13259 tablet_bootstrap.cc:492] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:43.313771 13261 tablet_bootstrap.cc:654] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.314392 13067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:43.315232 13067 raft_consensus.cc:2468] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 0.
I20260709 13:05:43.316926 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9324f89bc5b4281ac6b495fde2aa95c" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:43.317936 13066 raft_consensus.cc:2468] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 0.
I20260709 13:05:43.318553 13179 leader_election.cc:304] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:43.319813 13268 raft_consensus.cc:2804] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.319873 13179 leader_election.cc:304] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:43.320335 13268 raft_consensus.cc:493] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.320776 13268 raft_consensus.cc:3060] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.330224 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73438a7711574bb9a2ade51d2e041139" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:43.332777 13287 raft_consensus.cc:2804] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.334112 13287 raft_consensus.cc:493] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.334575 13287 raft_consensus.cc:3060] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.335510 13259 tablet_bootstrap.cc:654] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.336295 13268 raft_consensus.cc:515] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.343901 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:43.344637 13066 raft_consensus.cc:3060] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.348825 13261 tablet_bootstrap.cc:492] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:43.351394 13260 tablet_bootstrap.cc:492] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:43.352119 13260 ts_tablet_manager.cc:1403] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.061s	user 0.016s	sys 0.015s
I20260709 13:05:43.354631 13066 raft_consensus.cc:2468] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 1.
I20260709 13:05:43.356700 13261 ts_tablet_manager.cc:1403] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.010s
I20260709 13:05:43.358036 13179 leader_election.cc:304] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:43.360288 13266 raft_consensus.cc:493] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.360870 13266 raft_consensus.cc:515] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.332736 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:43.362341 13289 raft_consensus.cc:2804] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.355863 13260 raft_consensus.cc:359] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.356434 13268 leader_election.cc:290] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
W20260709 13:05:43.363749 13033 leader_election.cc:343] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.364482 13266 leader_election.cc:290] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
W20260709 13:05:43.364816 13033 leader_election.cc:343] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.374300 13287 raft_consensus.cc:515] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:43.366384 13259 tablet_bootstrap.cc:492] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:43.366736 13260 raft_consensus.cc:385] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.379827 13260 raft_consensus.cc:740] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.379984 13259 ts_tablet_manager.cc:1403] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.000s
I20260709 13:05:43.360863 13261 raft_consensus.cc:359] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.381522 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9324f89bc5b4281ac6b495fde2aa95c" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:43.381940 13261 raft_consensus.cc:385] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.382189 13066 raft_consensus.cc:3060] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.382294 13261 raft_consensus.cc:740] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.382524 13216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73438a7711574bb9a2ade51d2e041139" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:43.383436 13216 raft_consensus.cc:2468] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:43.383718 13287 leader_election.cc:290] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:43.382746 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:43.384610 13261 consensus_queue.cc:260] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.385825 13031 leader_election.cc:304] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:43.380872 13260 consensus_queue.cc:260] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.387337 13261 ts_tablet_manager.cc:1434] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:05:43.387414 13263 raft_consensus.cc:2804] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.388016 13263 raft_consensus.cc:493] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.388378 13263 raft_consensus.cc:3060] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.388494 13260 ts_tablet_manager.cc:1434] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.002s
I20260709 13:05:43.389698 13260 tablet_bootstrap.cc:492] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:43.365090 13266 raft_consensus.cc:493] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.391880 13266 raft_consensus.cc:515] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:43.397699 13263 raft_consensus.cc:515] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.399531 13266 leader_election.cc:290] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7674c4e544d475aaae24558660059ee (127.0.107.195:34911), 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185)
I20260709 13:05:43.401041 13261 tablet_bootstrap.cc:492] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:43.401117 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73438a7711574bb9a2ade51d2e041139" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:43.401844 13263 leader_election.cc:290] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.405004 13263 raft_consensus.cc:493] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.403599 13141 raft_consensus.cc:3060] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.405929 13216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73438a7711574bb9a2ade51d2e041139" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:43.411093 13216 raft_consensus.cc:3060] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.418136 13263 raft_consensus.cc:515] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.441313 13263 leader_election.cc:290] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.442267 13263 raft_consensus.cc:493] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.442706 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6667f62e97949e582d8cc92d1e3c720" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:43.442828 13263 raft_consensus.cc:515] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.420969 13066 raft_consensus.cc:2468] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 1.
I20260709 13:05:43.418466 13259 raft_consensus.cc:359] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.433502 13067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240ee459545f4baca128c935ccbf9645" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:43.446244 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6667f62e97949e582d8cc92d1e3c720" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:43.446785 13289 raft_consensus.cc:697] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:43.434996 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9324f89bc5b4281ac6b495fde2aa95c" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:43.448256 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4fd6fa94e8418ab089fec0bc1a633d" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:43.447503 13067 raft_consensus.cc:2468] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 0.
I20260709 13:05:43.435508 13216 raft_consensus.cc:2468] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:43.448479 13065 raft_consensus.cc:2393] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e0cf97be34bc41638dc9d31d945b0dd0 in current term 1: Already voted for candidate c7674c4e544d475aaae24558660059ee in this term.
I20260709 13:05:43.449489 13259 raft_consensus.cc:385] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.449920 13259 raft_consensus.cc:740] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.448705 13289 consensus_queue.cc:237] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.450198 13141 raft_consensus.cc:2468] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:43.450891 13259 consensus_queue.cc:260] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.451431 13031 leader_election.cc:304] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:43.452092 13214 raft_consensus.cc:2468] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:43.443284 13260 tablet_bootstrap.cc:654] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.453364 13105 leader_election.cc:304] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
W20260709 13:05:43.447359 13033 leader_election.cc:343] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.448992 13142 raft_consensus.cc:2468] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:43.461021 13031 leader_election.cc:304] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:43.464426 13179 leader_election.cc:304] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:43.466217 13033 leader_election.cc:304] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:43.466572 13287 raft_consensus.cc:2804] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.467121 13287 raft_consensus.cc:697] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:43.467968 13259 ts_tablet_manager.cc:1434] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.087s	user 0.010s	sys 0.000s
I20260709 13:05:43.468014 13287 consensus_queue.cc:237] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:43.469388 13259 tablet_bootstrap.cc:492] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:43.472949 13263 leader_election.cc:290] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.472549 12965 catalog_manager.cc:5697] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 0 to 1, leader changed from <none> to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 1 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:43.472626 13290 raft_consensus.cc:2804] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.476951 13290 raft_consensus.cc:493] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.477980 13290 raft_consensus.cc:3060] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.476945 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:43.481040 13263 raft_consensus.cc:2804] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.490463 13293 raft_consensus.cc:2804] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.482725 13261 tablet_bootstrap.cc:654] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.496970 13263 raft_consensus.cc:493] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.505765 13263 raft_consensus.cc:3060] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.505957 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9324f89bc5b4281ac6b495fde2aa95c" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:43.507071 13142 raft_consensus.cc:2468] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 0.
I20260709 13:05:43.497064 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4fd6fa94e8418ab089fec0bc1a633d" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:43.503080 13293 raft_consensus.cc:697] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:43.512023 13293 consensus_queue.cc:237] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.490598 13031 leader_election.cc:304] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73; no voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:43.517458 13296 raft_consensus.cc:2804] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.518296 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9324f89bc5b4281ac6b495fde2aa95c" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
W20260709 13:05:43.518586 13182 leader_election.cc:343] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.519316 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:43.519886 13141 raft_consensus.cc:3060] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.521927 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240ee459545f4baca128c935ccbf9645" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:43.522614 13215 raft_consensus.cc:2468] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 0.
I20260709 13:05:43.519380 13296 raft_consensus.cc:493] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.523725 13296 raft_consensus.cc:3060] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.518858 13259 tablet_bootstrap.cc:654] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.524564 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9324f89bc5b4281ac6b495fde2aa95c" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:43.526825 13263 raft_consensus.cc:515] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.530606 13263 leader_election.cc:290] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
W20260709 13:05:43.535863 13182 leader_election.cc:343] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.533666 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6667f62e97949e582d8cc92d1e3c720" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
W20260709 13:05:43.538674 13033 leader_election.cc:343] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.540017 13299 raft_consensus.cc:2749] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.549314 13290 raft_consensus.cc:515] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.543457 13141 raft_consensus.cc:2468] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 1.
I20260709 13:05:43.550384 13106 leader_election.cc:304] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0cf97be34bc41638dc9d31d945b0dd0; no voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee
I20260709 13:05:43.547115 13296 raft_consensus.cc:515] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.551380 13266 raft_consensus.cc:2749] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.553076 13290 leader_election.cc:290] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.554394 13296 leader_election.cc:290] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.556974 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4fd6fa94e8418ab089fec0bc1a633d" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:43.558068 13141 raft_consensus.cc:3060] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:43.560205 13031 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34911, user_credentials={real_user=slave}} blocked reactor thread for 69745us
I20260709 13:05:43.561448 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240ee459545f4baca128c935ccbf9645" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:43.562474 13214 raft_consensus.cc:3060] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.563584 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240ee459545f4baca128c935ccbf9645" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:43.564520 13066 raft_consensus.cc:3060] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.567032 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6667f62e97949e582d8cc92d1e3c720" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:43.568037 13215 raft_consensus.cc:3060] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.573019 13216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4fd6fa94e8418ab089fec0bc1a633d" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
W20260709 13:05:43.563529 13031 leader_election.cc:343] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7674c4e544d475aaae24558660059ee (127.0.107.195:34911): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:43.577019 13031 leader_election.cc:343] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7674c4e544d475aaae24558660059ee (127.0.107.195:34911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.578557 13260 tablet_bootstrap.cc:492] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:43.580132 13260 ts_tablet_manager.cc:1403] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.191s	user 0.011s	sys 0.016s
I20260709 13:05:43.580557 12964 catalog_manager.cc:5697] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.584991 13260 raft_consensus.cc:359] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.586208 13260 raft_consensus.cc:385] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.586594 13260 raft_consensus.cc:740] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.590214 13260 consensus_queue.cc:260] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.593710 13214 raft_consensus.cc:2468] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 1.
I20260709 13:05:43.595467 13260 ts_tablet_manager.cc:1434] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.001s
I20260709 13:05:43.596118 13106 leader_election.cc:304] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:43.596763 13260 tablet_bootstrap.cc:492] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:43.598040 13290 raft_consensus.cc:2804] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.598601 13290 raft_consensus.cc:697] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Becoming Leader. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:43.599539 13290 consensus_queue.cc:237] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.605506 13066 raft_consensus.cc:2468] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 1.
I20260709 13:05:43.607898 13215 raft_consensus.cc:2468] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:43.609328 13031 leader_election.cc:304] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:43.610447 13303 raft_consensus.cc:2804] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.611143 13303 raft_consensus.cc:697] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:43.612321 13303 consensus_queue.cc:237] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.623941 13259 tablet_bootstrap.cc:492] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:43.624816 13259 ts_tablet_manager.cc:1403] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.156s	user 0.037s	sys 0.021s
I20260709 13:05:43.614939 13141 raft_consensus.cc:2468] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:43.629802 13033 leader_election.cc:304] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: c7674c4e544d475aaae24558660059ee
I20260709 13:05:43.631202 13296 raft_consensus.cc:2804] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.631749 13296 raft_consensus.cc:697] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:43.631403 13259 raft_consensus.cc:359] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.632359 13259 raft_consensus.cc:385] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.632660 13259 raft_consensus.cc:740] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.632707 13296 consensus_queue.cc:237] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.633745 13259 consensus_queue.cc:260] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.639122 13259 ts_tablet_manager.cc:1434] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:05:43.635337 12965 catalog_manager.cc:5697] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 0 to 1, leader changed from <none> to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 1 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.640455 13259 tablet_bootstrap.cc:492] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:43.641299 13261 tablet_bootstrap.cc:492] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:43.642058 13261 ts_tablet_manager.cc:1403] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.241s	user 0.023s	sys 0.021s
I20260709 13:05:43.647756 13260 tablet_bootstrap.cc:654] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.655253 13259 tablet_bootstrap.cc:654] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.657881 13287 raft_consensus.cc:493] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.658512 13287 raft_consensus.cc:515] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.662824 13287 leader_election.cc:290] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:43.663164 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b0957f35a54bc196a443411300ffc4" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:43.663774 12964 catalog_manager.cc:5697] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:43.666595 13182 leader_election.cc:343] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.664975 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b0957f35a54bc196a443411300ffc4" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
W20260709 13:05:43.669255 13179 leader_election.cc:343] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.669735 13179 leader_election.cc:304] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7674c4e544d475aaae24558660059ee; no voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:43.670187 13266 raft_consensus.cc:493] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.670993 13266 raft_consensus.cc:515] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.671497 13287 raft_consensus.cc:2749] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.673198 13261 raft_consensus.cc:359] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.674068 13261 raft_consensus.cc:385] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.674954 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
W20260709 13:05:43.676352 13106 leader_election.cc:343] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7674c4e544d475aaae24558660059ee (127.0.107.195:34911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.674377 13261 raft_consensus.cc:740] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.677906 13260 tablet_bootstrap.cc:492] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:43.678618 13260 ts_tablet_manager.cc:1403] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.016s
I20260709 13:05:43.678323 13261 consensus_queue.cc:260] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.679250 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:43.680204 12965 catalog_manager.cc:5697] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:43.681316 13105 leader_election.cc:343] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.681929 13261 ts_tablet_manager.cc:1434] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.002s
I20260709 13:05:43.682019 13105 leader_election.cc:304] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0cf97be34bc41638dc9d31d945b0dd0; no voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee
I20260709 13:05:43.682462 13260 raft_consensus.cc:359] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.683257 13290 raft_consensus.cc:2749] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.683413 13261 tablet_bootstrap.cc:492] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:43.683277 13260 raft_consensus.cc:385] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.673645 13266 leader_election.cc:290] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.684600 13260 raft_consensus.cc:740] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.685685 13260 consensus_queue.cc:260] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.686656 13259 tablet_bootstrap.cc:492] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:43.687203 13259 ts_tablet_manager.cc:1403] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.010s
I20260709 13:05:43.688489 13260 ts_tablet_manager.cc:1434] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:05:43.689672 13260 tablet_bootstrap.cc:492] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:43.690568 13259 raft_consensus.cc:359] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.691442 13259 raft_consensus.cc:385] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.691776 13259 raft_consensus.cc:740] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.692780 13259 consensus_queue.cc:260] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.693909 13261 tablet_bootstrap.cc:654] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.693583 12964 catalog_manager.cc:5697] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.698020 13260 tablet_bootstrap.cc:654] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.698175 13259 ts_tablet_manager.cc:1434] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:05:43.699409 13259 tablet_bootstrap.cc:492] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:43.709460 13259 tablet_bootstrap.cc:654] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.710156 13260 tablet_bootstrap.cc:492] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:43.711087 13260 ts_tablet_manager.cc:1403] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.009s
I20260709 13:05:43.718894 13261 tablet_bootstrap.cc:492] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:43.719929 13261 ts_tablet_manager.cc:1403] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.003s
I20260709 13:05:43.722343 13260 raft_consensus.cc:359] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.723498 13266 raft_consensus.cc:493] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.723905 13261 raft_consensus.cc:359] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.724076 13266 raft_consensus.cc:515] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.724723 13261 raft_consensus.cc:385] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.725114 13261 raft_consensus.cc:740] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.727800 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:43.727604 13261 consensus_queue.cc:260] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.728470 13266 leader_election.cc:290] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.728839 13066 raft_consensus.cc:2393] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e0cf97be34bc41638dc9d31d945b0dd0 in current term 1: Already voted for candidate c7674c4e544d475aaae24558660059ee in this term.
I20260709 13:05:43.729009 13260 raft_consensus.cc:385] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.727943 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:43.729411 13260 raft_consensus.cc:740] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.730451 13260 consensus_queue.cc:260] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.731071 13106 leader_election.cc:304] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0cf97be34bc41638dc9d31d945b0dd0; no voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee
I20260709 13:05:43.734157 13266 raft_consensus.cc:3060] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.735407 13261 ts_tablet_manager.cc:1434] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:05:43.737175 13261 tablet_bootstrap.cc:492] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:43.743536 13266 raft_consensus.cc:2749] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:43.744261 13260 ts_tablet_manager.cc:1434] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:05:43.745574 13260 tablet_bootstrap.cc:492] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:43.746316 13261 tablet_bootstrap.cc:654] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.752337 13259 tablet_bootstrap.cc:492] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:43.752945 13259 ts_tablet_manager.cc:1403] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.054s	user 0.021s	sys 0.022s
I20260709 13:05:43.757115 13260 tablet_bootstrap.cc:654] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:43.757256 13259 raft_consensus.cc:359] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.758220 13259 raft_consensus.cc:385] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.758600 13259 raft_consensus.cc:740] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.759581 13259 consensus_queue.cc:260] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.760277 13261 tablet_bootstrap.cc:492] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:43.761018 13261 ts_tablet_manager.cc:1403] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.008s
I20260709 13:05:43.762172 13259 ts_tablet_manager.cc:1434] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:05:43.764671 13261 raft_consensus.cc:359] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.765759 13261 raft_consensus.cc:385] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.766036 13261 raft_consensus.cc:740] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.766486 13296 raft_consensus.cc:493] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.766976 13261 consensus_queue.cc:260] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.767064 13296 raft_consensus.cc:515] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.768049 13260 tablet_bootstrap.cc:492] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:43.768814 13260 ts_tablet_manager.cc:1403] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.004s
I20260709 13:05:43.769955 13261 ts_tablet_manager.cc:1434] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:05:43.771301 13296 leader_election.cc:290] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.771153 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b0957f35a54bc196a443411300ffc4" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:43.772207 13215 raft_consensus.cc:2468] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:43.772943 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b0957f35a54bc196a443411300ffc4" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:43.773633 13031 leader_election.cc:304] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
W20260709 13:05:43.774632 13033 leader_election.cc:343] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:43.774735 13296 raft_consensus.cc:2804] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.775207 13296 raft_consensus.cc:493] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.775683 13296 raft_consensus.cc:3060] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.776546 13260 raft_consensus.cc:359] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.777313 13260 raft_consensus.cc:385] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:43.777613 13260 raft_consensus.cc:740] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.778707 13260 consensus_queue.cc:260] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.781028 13260 ts_tablet_manager.cc:1434] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.001s
I20260709 13:05:43.783718 13296 raft_consensus.cc:515] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.786867 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b0957f35a54bc196a443411300ffc4" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:43.786877 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b0957f35a54bc196a443411300ffc4" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:43.786280 13296 leader_election.cc:290] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.787806 13215 raft_consensus.cc:3060] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.787809 13141 raft_consensus.cc:3060] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.795414 13215 raft_consensus.cc:2468] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:43.795899 13141 raft_consensus.cc:2468] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:43.796651 13031 leader_election.cc:304] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:43.797708 13296 raft_consensus.cc:2804] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.798182 13296 raft_consensus.cc:697] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:43.798892 13296 consensus_queue.cc:237] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.807670 12965 catalog_manager.cc:5697] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.829265 13287 raft_consensus.cc:493] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.830083 13287 raft_consensus.cc:515] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.832828 13287 leader_election.cc:290] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:43.834337 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4c9edf710c40b1a3105b6d94a01ddb" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:43.834617 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4c9edf710c40b1a3105b6d94a01ddb" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:43.835286 13141 raft_consensus.cc:2468] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 0.
I20260709 13:05:43.835286 13066 raft_consensus.cc:2468] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 0.
I20260709 13:05:43.836738 13179 leader_election.cc:304] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:43.837838 13287 raft_consensus.cc:2804] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.838420 13287 raft_consensus.cc:493] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.838812 13287 raft_consensus.cc:3060] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.846268 13287 raft_consensus.cc:515] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.848371 13287 leader_election.cc:290] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:43.849406 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4c9edf710c40b1a3105b6d94a01ddb" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:43.850183 13066 raft_consensus.cc:3060] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.849788 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4c9edf710c40b1a3105b6d94a01ddb" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:43.850704 13141 raft_consensus.cc:3060] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.856660 13066 raft_consensus.cc:2468] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 1.
I20260709 13:05:43.856737 13141 raft_consensus.cc:2468] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 1.
I20260709 13:05:43.858201 13179 leader_election.cc:304] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:43.859164 13287 raft_consensus.cc:2804] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.859647 13287 raft_consensus.cc:697] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:43.860540 13287 consensus_queue.cc:237] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.869807 12965 catalog_manager.cc:5697] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 0 to 1, leader changed from <none> to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 1 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:43.908540 13287 consensus_queue.cc:1048] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:43.924868 13289 consensus_queue.cc:1048] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.926666 13291 consensus_queue.cc:1048] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.942018 13296 raft_consensus.cc:493] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.942634 13296 raft_consensus.cc:515] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.945168 13296 leader_election.cc:290] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.947067 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:43.947249 13289 consensus_queue.cc:1048] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:43.947875 13141 raft_consensus.cc:2468] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:43.946452 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:43.949107 13215 raft_consensus.cc:2468] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:43.949249 13033 leader_election.cc:304] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:43.950223 13296 raft_consensus.cc:2804] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:43.950668 13296 raft_consensus.cc:493] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.951053 13296 raft_consensus.cc:3060] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.954970 13303 consensus_queue.cc:1048] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.960994 13296 raft_consensus.cc:515] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.965025 13296 leader_election.cc:290] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:43.969866 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:43.970688 13215 raft_consensus.cc:3060] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.970748 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:43.971530 13141 raft_consensus.cc:3060] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:43.972800 13299 consensus_queue.cc:1048] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:43.984179 13215 raft_consensus.cc:2468] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:43.986581 13031 leader_election.cc:304] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:43.987169 13141 raft_consensus.cc:2468] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:43.987672 13299 raft_consensus.cc:2804] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:43.988229 13299 raft_consensus.cc:697] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:43.989127 13299 consensus_queue.cc:237] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.991133 13289 raft_consensus.cc:493] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.991794 13289 raft_consensus.cc:515] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:43.994148 13289 leader_election.cc:290] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:43.996639 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4fd6fa94e8418ab089fec0bc1a633d" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:43.997738 13141 raft_consensus.cc:2393] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7674c4e544d475aaae24558660059ee in current term 1: Already voted for candidate 197e5fb7a05e45069d5fe5acb8484e73 in this term.
I20260709 13:05:43.998314 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4fd6fa94e8418ab089fec0bc1a633d" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:44.000260 13179 leader_election.cc:304] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7674c4e544d475aaae24558660059ee; no voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:44.001262 13289 raft_consensus.cc:3060] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.004472 13266 consensus_queue.cc:1048] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:44.014787 13289 raft_consensus.cc:2749] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:44.016043 13266 consensus_queue.cc:1048] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.015938 12965 catalog_manager.cc:5697] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.055992 13141 raft_consensus.cc:3060] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:44.067193 13299 consensus_queue.cc:1048] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.073776 13299 consensus_queue.cc:1048] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.079455 13303 consensus_queue.cc:1048] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.099799 13293 consensus_queue.cc:1048] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.221004 13293 consensus_queue.cc:1048] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:44.233976 13293 consensus_queue.cc:1048] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.255659 13289 consensus_queue.cc:1048] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.269754 13291 consensus_queue.cc:1048] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:44.322364 13214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9324f89bc5b4281ac6b495fde2aa95c"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
mode: GRACEFUL
new_leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:48984
I20260709 13:05:44.323007 13214 raft_consensus.cc:606] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Received request to transfer leadership to TS 197e5fb7a05e45069d5fe5acb8484e73
I20260709 13:05:44.329632 13214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
mode: GRACEFUL
new_leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:48984
I20260709 13:05:44.330250 13214 raft_consensus.cc:606] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Received request to transfer leadership to TS 197e5fb7a05e45069d5fe5acb8484e73
I20260709 13:05:44.351039 13142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "240ee459545f4baca128c935ccbf9645"
dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
mode: GRACEFUL
new_leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:46042
I20260709 13:05:44.351714 13142 raft_consensus.cc:606] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Received request to transfer leadership to TS 197e5fb7a05e45069d5fe5acb8484e73
I20260709 13:05:44.375974 13066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb4fd6fa94e8418ab089fec0bc1a633d"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
mode: GRACEFUL
new_leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:35272
I20260709 13:05:44.376690 13066 raft_consensus.cc:606] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Received request to transfer leadership to TS e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:44.383275 13066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
mode: GRACEFUL
new_leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:35272
I20260709 13:05:44.384025 13066 raft_consensus.cc:606] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Received request to transfer leadership to TS e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:44.390686 13066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6667f62e97949e582d8cc92d1e3c720"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
mode: GRACEFUL
new_leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:35272
I20260709 13:05:44.391346 13066 raft_consensus.cc:606] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Received request to transfer leadership to TS e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:44.397548 13214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc4c9edf710c40b1a3105b6d94a01ddb"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
mode: GRACEFUL
new_leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:48984
I20260709 13:05:44.398291 13214 raft_consensus.cc:606] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Received request to transfer leadership to TS e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:44.404547 13066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95b0957f35a54bc196a443411300ffc4"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
mode: GRACEFUL
new_leader_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:35272
I20260709 13:05:44.405174 13066 raft_consensus.cc:606] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Received request to transfer leadership to TS c7674c4e544d475aaae24558660059ee
I20260709 13:05:44.439704 13289 raft_consensus.cc:993] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee: : Instructing follower 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:44.440255 13287 raft_consensus.cc:1081] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Signalling peer 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:44.442219 13066 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:35250
I20260709 13:05:44.442867 13066 raft_consensus.cc:493] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:44.443252 13066 raft_consensus.cc:3060] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.450313 13066 raft_consensus.cc:515] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.452399 13066 leader_election.cc:290] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:44.453408 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:44.453683 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd0304336bd4f0994a5a44bef55efdf" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:44.454238 13142 raft_consensus.cc:3060] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.454298 13214 raft_consensus.cc:3055] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:44.454782 13214 raft_consensus.cc:740] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:44.455765 13214 consensus_queue.cc:260] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.457213 13214 raft_consensus.cc:3060] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.463546 13142 raft_consensus.cc:2468] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:44.465888 13033 leader_election.cc:304] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:44.466018 13214 raft_consensus.cc:2468] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:44.467260 13293 raft_consensus.cc:2804] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:44.467831 13293 raft_consensus.cc:697] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:44.468482 13299 consensus_queue.cc:1048] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:44.468739 13293 consensus_queue.cc:237] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.478401 13303 raft_consensus.cc:993] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73: : Instructing follower e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:44.478977 13303 raft_consensus.cc:1081] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Signalling peer e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:44.478771 12964 catalog_manager.cc:5697] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 1 to 2, leader changed from c7674c4e544d475aaae24558660059ee (127.0.107.195) to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.484170 13141 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107"
dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:46040
I20260709 13:05:44.485711 13141 raft_consensus.cc:493] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:44.486195 13141 raft_consensus.cc:3060] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.488278 13303 consensus_queue.cc:1048] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:44.497318 13287 raft_consensus.cc:993] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee: : Instructing follower 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:44.497926 13287 raft_consensus.cc:1081] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Signalling peer 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:44.499693 13141 raft_consensus.cc:515] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.500072 13066 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d9324f89bc5b4281ac6b495fde2aa95c"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:35250
I20260709 13:05:44.500981 13066 raft_consensus.cc:493] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:44.501513 13066 raft_consensus.cc:3060] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.507851 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:44.508862 13215 raft_consensus.cc:3060] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.510592 13066 raft_consensus.cc:515] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:44.513263 13066 leader_election.cc:290] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Requested vote from peers c7674c4e544d475aaae24558660059ee (127.0.107.195:34911), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:44.514747 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9324f89bc5b4281ac6b495fde2aa95c" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:44.515554 13142 raft_consensus.cc:3060] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.515911 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:44.516628 13065 raft_consensus.cc:3055] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:44.516983 13065 raft_consensus.cc:740] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:44.518319 13215 raft_consensus.cc:2468] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:44.518409 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9324f89bc5b4281ac6b495fde2aa95c" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:44.519284 13214 raft_consensus.cc:3055] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:44.519657 13214 raft_consensus.cc:740] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:44.519887 13330 raft_consensus.cc:993] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0: : Instructing follower 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:44.520408 13330 raft_consensus.cc:1081] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Signalling peer 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:44.518178 13065 consensus_queue.cc:260] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.522298 13141 leader_election.cc:290] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:44.522763 13065 raft_consensus.cc:3060] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.524463 13106 leader_election.cc:304] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:44.524896 13142 raft_consensus.cc:2468] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:44.525553 13266 raft_consensus.cc:2804] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:44.526157 13266 raft_consensus.cc:697] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Becoming Leader. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:44.527014 13266 consensus_queue.cc:237] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.529065 13066 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "240ee459545f4baca128c935ccbf9645"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:35256
I20260709 13:05:44.521087 13214 consensus_queue.cc:260] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:44.529714 13066 raft_consensus.cc:493] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:44.530213 13066 raft_consensus.cc:3060] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.530573 13214 raft_consensus.cc:3060] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.530905 13033 leader_election.cc:304] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:44.531875 13303 raft_consensus.cc:2804] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:44.532430 13303 raft_consensus.cc:697] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:44.533372 13303 consensus_queue.cc:237] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } }
I20260709 13:05:44.538587 12964 catalog_manager.cc:5697] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 reported cstate change: term changed from 1 to 2, leader changed from 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193) to e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.540103 13065 raft_consensus.cc:2468] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:44.543587 13214 raft_consensus.cc:2468] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:44.542843 12962 catalog_manager.cc:5697] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 1 to 2, leader changed from c7674c4e544d475aaae24558660059ee (127.0.107.195) to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.551322 13066 raft_consensus.cc:515] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.553756 13066 leader_election.cc:290] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:44.554634 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240ee459545f4baca128c935ccbf9645" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:44.554838 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240ee459545f4baca128c935ccbf9645" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:44.555492 13142 raft_consensus.cc:3055] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:44.555495 13215 raft_consensus.cc:3060] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.556061 13142 raft_consensus.cc:740] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:44.556907 13142 consensus_queue.cc:260] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.558238 13142 raft_consensus.cc:3060] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.562193 13293 raft_consensus.cc:993] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73: : Instructing follower e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:44.562896 13303 raft_consensus.cc:1081] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Signalling peer e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:44.563395 13215 raft_consensus.cc:2468] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:44.565004 13141 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb4fd6fa94e8418ab089fec0bc1a633d"
dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:46040
I20260709 13:05:44.565007 13031 leader_election.cc:304] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:44.566026 13141 raft_consensus.cc:493] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:44.566356 13303 raft_consensus.cc:2804] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:44.566496 13141 raft_consensus.cc:3060] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.566952 13303 raft_consensus.cc:697] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:44.567309 13142 raft_consensus.cc:2468] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:44.567711 13303 consensus_queue.cc:237] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.576068 13141 raft_consensus.cc:515] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.578816 13141 leader_election.cc:290] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:44.579294 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4fd6fa94e8418ab089fec0bc1a633d" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:44.579726 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4fd6fa94e8418ab089fec0bc1a633d" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:44.580415 13215 raft_consensus.cc:3060] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.580268 13066 raft_consensus.cc:3055] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:44.580140 12962 catalog_manager.cc:5697] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 1 to 2, leader changed from e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194) to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.581399 13066 raft_consensus.cc:740] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:44.582477 13066 consensus_queue.cc:260] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.584017 13066 raft_consensus.cc:3060] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.588128 13215 raft_consensus.cc:2468] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:44.589592 13106 leader_election.cc:304] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:44.590502 13266 raft_consensus.cc:2804] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:44.591039 13266 raft_consensus.cc:697] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Becoming Leader. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:44.591802 13266 consensus_queue.cc:237] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.592567 13066 raft_consensus.cc:2468] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:44.600720 12964 catalog_manager.cc:5697] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 reported cstate change: term changed from 1 to 2, leader changed from 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193) to e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.612684 13293 raft_consensus.cc:993] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73: : Instructing follower e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:44.613268 13293 raft_consensus.cc:1081] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Signalling peer e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:44.615258 13141 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6667f62e97949e582d8cc92d1e3c720"
dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:46040
I20260709 13:05:44.615978 13141 raft_consensus.cc:493] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:44.616364 13141 raft_consensus.cc:3060] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.623662 13141 raft_consensus.cc:515] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.625653 13141 leader_election.cc:290] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:44.626662 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6667f62e97949e582d8cc92d1e3c720" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:44.627168 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6667f62e97949e582d8cc92d1e3c720" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:44.627444 13066 raft_consensus.cc:3055] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:44.627884 13066 raft_consensus.cc:740] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:44.627928 13215 raft_consensus.cc:3060] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.628692 13066 consensus_queue.cc:260] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.629894 13066 raft_consensus.cc:3060] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.635141 13215 raft_consensus.cc:2468] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:44.636335 13066 raft_consensus.cc:2468] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:44.636854 13106 leader_election.cc:304] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:44.638072 13266 raft_consensus.cc:2804] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:44.638608 13266 raft_consensus.cc:697] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Becoming Leader. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:44.639479 13266 consensus_queue.cc:237] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.646749 12964 catalog_manager.cc:5697] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 reported cstate change: term changed from 1 to 2, leader changed from 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193) to e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.823045 13303 raft_consensus.cc:993] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73: : Instructing follower c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:44.823611 13293 raft_consensus.cc:1081] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Signalling peer c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:44.825579 13215 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95b0957f35a54bc196a443411300ffc4"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:05:44.826282 13215 raft_consensus.cc:493] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:44.826598 13215 raft_consensus.cc:3060] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.833938 13215 raft_consensus.cc:515] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.836431 13215 leader_election.cc:290] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 2 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:44.837318 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b0957f35a54bc196a443411300ffc4" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:44.837838 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b0957f35a54bc196a443411300ffc4" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:44.838201 13066 raft_consensus.cc:3055] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:44.838594 13141 raft_consensus.cc:3060] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.838579 13066 raft_consensus.cc:740] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:44.840437 13066 consensus_queue.cc:260] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.841883 13066 raft_consensus.cc:3060] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.842028 13287 raft_consensus.cc:993] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee: : Instructing follower e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:44.842721 13291 raft_consensus.cc:1081] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Signalling peer e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:44.844550 13142 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc4c9edf710c40b1a3105b6d94a01ddb"
dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:46034
I20260709 13:05:44.845149 13142 raft_consensus.cc:493] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:44.845585 13142 raft_consensus.cc:3060] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.847709 13141 raft_consensus.cc:2468] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 2.
I20260709 13:05:44.849258 13182 leader_election.cc:304] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:44.850539 13291 raft_consensus.cc:2804] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:44.851119 13291 raft_consensus.cc:697] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 2 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:44.851527 13066 raft_consensus.cc:2468] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 2.
I20260709 13:05:44.852160 13291 consensus_queue.cc:237] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.854492 13142 raft_consensus.cc:515] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.856820 13142 leader_election.cc:290] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:44.858013 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4c9edf710c40b1a3105b6d94a01ddb" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:44.858390 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4c9edf710c40b1a3105b6d94a01ddb" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:44.858795 13066 raft_consensus.cc:3060] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.859117 13215 raft_consensus.cc:3055] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:44.859520 13215 raft_consensus.cc:740] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:44.860426 13215 consensus_queue.cc:260] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.861959 13215 raft_consensus.cc:3060] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.861306 12964 catalog_manager.cc:5697] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 1 to 2, leader changed from 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193) to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 2 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:44.866890 13066 raft_consensus.cc:2468] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:44.868506 13105 leader_election.cc:304] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:44.869552 13266 raft_consensus.cc:2804] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:44.871111 13266 raft_consensus.cc:697] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Becoming Leader. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:44.871618 13215 raft_consensus.cc:2468] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:44.872174 13266 consensus_queue.cc:237] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:44.881877 12964 catalog_manager.cc:5697] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 reported cstate change: term changed from 1 to 2, leader changed from c7674c4e544d475aaae24558660059ee (127.0.107.195) to e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.956081 13215 raft_consensus.cc:1275] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:44.957935 13266 consensus_queue.cc:1048] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:44.966784 13066 raft_consensus.cc:1275] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:44.970923 13330 consensus_queue.cc:1048] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:44.971760 13142 raft_consensus.cc:1275] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:44.974154 13293 consensus_queue.cc:1048] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:44.988210 13215 raft_consensus.cc:1275] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:44.990065 13296 consensus_queue.cc:1048] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.002089 13214 raft_consensus.cc:1275] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.003805 13290 consensus_queue.cc:1048] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.012384 13066 raft_consensus.cc:1275] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.014094 13330 consensus_queue.cc:1048] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.039884 13142 raft_consensus.cc:1275] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.041864 13296 consensus_queue.cc:1048] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:45.053540 13214 raft_consensus.cc:1275] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.055389 13299 consensus_queue.cc:1048] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.084420 13066 raft_consensus.cc:1275] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.086258 13330 consensus_queue.cc:1048] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.097213 13214 raft_consensus.cc:1275] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.099205 13330 consensus_queue.cc:1048] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.108603 13215 raft_consensus.cc:1275] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.111275 13299 consensus_queue.cc:1048] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.123943 13142 raft_consensus.cc:1275] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.125955 13293 consensus_queue.cc:1048] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.366118 13066 raft_consensus.cc:1275] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.367851 13330 consensus_queue.cc:1048] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.370361 13066 raft_consensus.cc:1275] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.372125 13291 consensus_queue.cc:1048] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.380687 13215 raft_consensus.cc:1275] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.382323 13290 consensus_queue.cc:1048] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.405709 13142 raft_consensus.cc:1275] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.410714 13291 consensus_queue.cc:1048] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:47.435875 12965 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57770:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:47.438640 12965 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:47.497723 13052 tablet_service.cc:1511] Processing CreateTablet for tablet ae440740b8324a3190399bba1d3b1c79 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:47.499320 13052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae440740b8324a3190399bba1d3b1c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.499624 13124 tablet_service.cc:1511] Processing CreateTablet for tablet ae440740b8324a3190399bba1d3b1c79 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:47.501171 13124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae440740b8324a3190399bba1d3b1c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.501389 13201 tablet_service.cc:1511] Processing CreateTablet for tablet ae440740b8324a3190399bba1d3b1c79 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:47.502874 13201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae440740b8324a3190399bba1d3b1c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.503055 13049 tablet_service.cc:1511] Processing CreateTablet for tablet 74e4a050557d4bb781e7c518b48dda1c (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:47.504473 13049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e4a050557d4bb781e7c518b48dda1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.507879 13126 tablet_service.cc:1511] Processing CreateTablet for tablet 74e4a050557d4bb781e7c518b48dda1c (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:47.509996 13125 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5ee8c1d8d64a4597c891bcaace5176 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:47.513857 13126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e4a050557d4bb781e7c518b48dda1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.513896 13050 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5ee8c1d8d64a4597c891bcaace5176 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:47.516629 13050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5ee8c1d8d64a4597c891bcaace5176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.518494 13125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5ee8c1d8d64a4597c891bcaace5176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.520498 13130 tablet_service.cc:1511] Processing CreateTablet for tablet 76658aa264e54c3bad96d43577ca3db0 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:47.521557 13051 tablet_service.cc:1511] Processing CreateTablet for tablet 76658aa264e54c3bad96d43577ca3db0 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:47.522913 13051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76658aa264e54c3bad96d43577ca3db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.526479 13204 tablet_service.cc:1511] Processing CreateTablet for tablet 74e4a050557d4bb781e7c518b48dda1c (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:47.528031 13204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e4a050557d4bb781e7c518b48dda1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.528816 13375 tablet_bootstrap.cc:492] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:47.529232 13131 tablet_service.cc:1511] Processing CreateTablet for tablet 450e4cfaaaf5453a8d06427a21d8abef (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:47.532729 13200 tablet_service.cc:1511] Processing CreateTablet for tablet 76658aa264e54c3bad96d43577ca3db0 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:47.534611 13054 tablet_service.cc:1511] Processing CreateTablet for tablet 450e4cfaaaf5453a8d06427a21d8abef (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:47.536043 13054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 450e4cfaaaf5453a8d06427a21d8abef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.541724 13202 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5ee8c1d8d64a4597c891bcaace5176 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:47.520498 13053 tablet_service.cc:1511] Processing CreateTablet for tablet 012804e1fc9244889c16e6d4606f47ce (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:47.544580 13053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012804e1fc9244889c16e6d4606f47ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.546551 13056 tablet_service.cc:1511] Processing CreateTablet for tablet 85b2db696e40443fb9e93c2e64498685 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:47.548009 13056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b2db696e40443fb9e93c2e64498685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.548688 13124 tablet_service.cc:1511] Processing CreateTablet for tablet 85b2db696e40443fb9e93c2e64498685 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:47.518575 13128 tablet_service.cc:1511] Processing CreateTablet for tablet 012804e1fc9244889c16e6d4606f47ce (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:47.541201 13198 tablet_service.cc:1511] Processing CreateTablet for tablet 450e4cfaaaf5453a8d06427a21d8abef (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:47.541201 13199 tablet_service.cc:1511] Processing CreateTablet for tablet 012804e1fc9244889c16e6d4606f47ce (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:47.522007 13130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76658aa264e54c3bad96d43577ca3db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.553699 13203 tablet_service.cc:1511] Processing CreateTablet for tablet 85b2db696e40443fb9e93c2e64498685 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:47.554280 13128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012804e1fc9244889c16e6d4606f47ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.555274 13131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 450e4cfaaaf5453a8d06427a21d8abef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.556315 13055 tablet_service.cc:1511] Processing CreateTablet for tablet 80f9bf328c0b4484b1bcb051cbddaca3 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:47.557802 13055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80f9bf328c0b4484b1bcb051cbddaca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.559773 13124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b2db696e40443fb9e93c2e64498685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.558584 13126 tablet_service.cc:1511] Processing CreateTablet for tablet 80f9bf328c0b4484b1bcb051cbddaca3 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:47.560850 13126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80f9bf328c0b4484b1bcb051cbddaca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.563392 13202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5ee8c1d8d64a4597c891bcaace5176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.565414 13200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76658aa264e54c3bad96d43577ca3db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.567647 13198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 450e4cfaaaf5453a8d06427a21d8abef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.567298 13057 tablet_service.cc:1511] Processing CreateTablet for tablet 29f6e683b23f4646830ba9885c9765c4 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:47.568821 13057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29f6e683b23f4646830ba9885c9765c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.570705 13203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b2db696e40443fb9e93c2e64498685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.575083 13205 tablet_service.cc:1511] Processing CreateTablet for tablet 80f9bf328c0b4484b1bcb051cbddaca3 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:47.577353 13205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80f9bf328c0b4484b1bcb051cbddaca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.580252 13199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012804e1fc9244889c16e6d4606f47ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.598023 13129 tablet_service.cc:1511] Processing CreateTablet for tablet 29f6e683b23f4646830ba9885c9765c4 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:47.601431 13206 tablet_service.cc:1511] Processing CreateTablet for tablet 29f6e683b23f4646830ba9885c9765c4 (DEFAULT_TABLE table=soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:47.603152 13206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29f6e683b23f4646830ba9885c9765c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.599738 13129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29f6e683b23f4646830ba9885c9765c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.616074 13376 tablet_bootstrap.cc:492] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:47.624361 13376 tablet_bootstrap.cc:654] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.695802 13375 tablet_bootstrap.cc:654] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.717931 13379 tablet_bootstrap.cc:492] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:47.731724 13379 tablet_bootstrap.cc:654] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.746790 13379 tablet_bootstrap.cc:492] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:47.747725 13379 ts_tablet_manager.cc:1403] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.010s
I20260709 13:05:47.753870 13379 raft_consensus.cc:359] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.755170 13379 raft_consensus.cc:385] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.755678 13379 raft_consensus.cc:740] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.756907 13379 consensus_queue.cc:260] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.760679 13379 ts_tablet_manager.cc:1434] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:05:47.810334 13379 tablet_bootstrap.cc:492] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:47.820798 13376 tablet_bootstrap.cc:492] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:47.821620 13376 ts_tablet_manager.cc:1403] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.206s	user 0.065s	sys 0.126s
I20260709 13:05:47.821975 13375 tablet_bootstrap.cc:492] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:47.822681 13375 ts_tablet_manager.cc:1403] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.294s	user 0.011s	sys 0.010s
I20260709 13:05:47.825320 13376 raft_consensus.cc:359] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.826296 13376 raft_consensus.cc:385] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.826642 13376 raft_consensus.cc:740] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.827505 13376 consensus_queue.cc:260] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
W20260709 13:05:47.817140 13106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34911, user_credentials={real_user=slave}} blocked reactor thread for 96280.2us
I20260709 13:05:47.833781 13376 ts_tablet_manager.cc:1434] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:05:47.846930 13375 raft_consensus.cc:359] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.849429 13375 raft_consensus.cc:385] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.849898 13375 raft_consensus.cc:740] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.850729 13375 consensus_queue.cc:260] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.847783 13376 tablet_bootstrap.cc:492] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:47.848989 13379 tablet_bootstrap.cc:654] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.857031 13375 ts_tablet_manager.cc:1434] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:05:47.858426 13375 tablet_bootstrap.cc:492] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:47.860762 13376 tablet_bootstrap.cc:654] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.863786 13379 tablet_bootstrap.cc:492] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:47.864569 13379 ts_tablet_manager.cc:1403] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.001s
I20260709 13:05:47.869392 13379 raft_consensus.cc:359] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.870199 13379 raft_consensus.cc:385] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.870523 13379 raft_consensus.cc:740] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.871313 13379 consensus_queue.cc:260] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.872090 13376 tablet_bootstrap.cc:492] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:47.872817 13376 ts_tablet_manager.cc:1403] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.036s	user 0.006s	sys 0.013s
I20260709 13:05:47.876722 13376 raft_consensus.cc:359] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.877579 13376 raft_consensus.cc:385] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.877981 13376 raft_consensus.cc:740] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.879312 13375 tablet_bootstrap.cc:654] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.878844 13376 consensus_queue.cc:260] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.881850 13376 ts_tablet_manager.cc:1434] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:05:47.883101 13376 tablet_bootstrap.cc:492] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:47.883111 13379 ts_tablet_manager.cc:1434] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.003s
I20260709 13:05:47.884443 13379 tablet_bootstrap.cc:492] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:47.891773 13376 tablet_bootstrap.cc:654] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.891885 13379 tablet_bootstrap.cc:654] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.894666 13375 tablet_bootstrap.cc:492] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:47.895442 13375 ts_tablet_manager.cc:1403] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.000s
I20260709 13:05:47.900314 13375 raft_consensus.cc:359] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.901408 13375 raft_consensus.cc:385] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.901927 13375 raft_consensus.cc:740] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.903080 13375 consensus_queue.cc:260] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.904708 13379 tablet_bootstrap.cc:492] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:47.904910 13376 tablet_bootstrap.cc:492] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:47.905391 13379 ts_tablet_manager.cc:1403] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 13:05:47.905591 13376 ts_tablet_manager.cc:1403] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.010s
I20260709 13:05:47.905643 13375 ts_tablet_manager.cc:1434] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:05:47.907337 13375 tablet_bootstrap.cc:492] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:47.908968 13379 raft_consensus.cc:359] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.910015 13379 raft_consensus.cc:385] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.909796 13376 raft_consensus.cc:359] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.910418 13379 raft_consensus.cc:740] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.910895 13376 raft_consensus.cc:385] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.911345 13376 raft_consensus.cc:740] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.911729 13379 consensus_queue.cc:260] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.912381 13376 consensus_queue.cc:260] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.915228 13376 ts_tablet_manager.cc:1434] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:05:47.916412 13376 tablet_bootstrap.cc:492] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:47.918637 13375 tablet_bootstrap.cc:654] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.922431 13379 ts_tablet_manager.cc:1434] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:05:47.923713 13379 tablet_bootstrap.cc:492] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:47.924484 13376 tablet_bootstrap.cc:654] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.939028 13375 tablet_bootstrap.cc:492] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:47.939618 13375 ts_tablet_manager.cc:1403] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.033s	user 0.019s	sys 0.000s
I20260709 13:05:47.939851 13376 tablet_bootstrap.cc:492] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:47.940506 13376 ts_tablet_manager.cc:1403] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.004s
I20260709 13:05:47.942107 13379 tablet_bootstrap.cc:654] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.942711 13375 raft_consensus.cc:359] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.943573 13375 raft_consensus.cc:385] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.944080 13375 raft_consensus.cc:740] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.944099 13376 raft_consensus.cc:359] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.945183 13376 raft_consensus.cc:385] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.945102 13375 consensus_queue.cc:260] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.945701 13376 raft_consensus.cc:740] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.946727 13376 consensus_queue.cc:260] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.948402 13375 ts_tablet_manager.cc:1434] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:05:47.949719 13375 tablet_bootstrap.cc:492] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:47.950994 13379 tablet_bootstrap.cc:492] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:47.951602 13379 ts_tablet_manager.cc:1403] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.000s
I20260709 13:05:47.955338 13379 raft_consensus.cc:359] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.956166 13379 raft_consensus.cc:385] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.956514 13379 raft_consensus.cc:740] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.957332 13375 tablet_bootstrap.cc:654] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.957410 13379 consensus_queue.cc:260] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.958144 13376 ts_tablet_manager.cc:1434] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.001s
I20260709 13:05:47.959507 13376 tablet_bootstrap.cc:492] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:47.960645 13379 ts_tablet_manager.cc:1434] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:05:47.962778 13379 tablet_bootstrap.cc:492] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:47.967942 13376 tablet_bootstrap.cc:654] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.973187 13379 tablet_bootstrap.cc:654] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.977126 13375 tablet_bootstrap.cc:492] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:47.978259 13375 ts_tablet_manager.cc:1403] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.015s
I20260709 13:05:47.982888 13375 raft_consensus.cc:359] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.984884 13375 raft_consensus.cc:385] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.985266 13375 raft_consensus.cc:740] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.986227 13375 consensus_queue.cc:260] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.986729 13376 tablet_bootstrap.cc:492] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:47.987206 13379 tablet_bootstrap.cc:492] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:47.987418 13376 ts_tablet_manager.cc:1403] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.003s
I20260709 13:05:47.987813 13379 ts_tablet_manager.cc:1403] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.004s
I20260709 13:05:47.988937 13375 ts_tablet_manager.cc:1434] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:47.990162 13375 tablet_bootstrap.cc:492] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:47.991360 13376 raft_consensus.cc:359] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.991842 13379 raft_consensus.cc:359] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.992326 13376 raft_consensus.cc:385] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.992780 13379 raft_consensus.cc:385] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.992885 13376 raft_consensus.cc:740] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.993252 13379 raft_consensus.cc:740] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.994019 13376 consensus_queue.cc:260] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.994361 13379 consensus_queue.cc:260] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.996968 13376 ts_tablet_manager.cc:1434] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:05:47.997210 13363 raft_consensus.cc:493] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.997789 13363 raft_consensus.cc:515] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:47.998211 13376 tablet_bootstrap.cc:492] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:48.002681 13375 tablet_bootstrap.cc:654] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.007891 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae440740b8324a3190399bba1d3b1c79" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:48.009883 13142 raft_consensus.cc:2468] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 0.
I20260709 13:05:48.010010 13376 tablet_bootstrap.cc:654] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.008420 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae440740b8324a3190399bba1d3b1c79" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:48.011437 13182 leader_election.cc:304] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:48.011652 13065 raft_consensus.cc:2468] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 0.
I20260709 13:05:48.009039 13363 leader_election.cc:290] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:48.014020 13363 raft_consensus.cc:2804] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.014652 13363 raft_consensus.cc:493] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.015045 13363 raft_consensus.cc:3060] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.018807 13375 tablet_bootstrap.cc:492] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:48.019665 13375 ts_tablet_manager.cc:1403] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.008s
I20260709 13:05:48.020196 13376 tablet_bootstrap.cc:492] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:48.020918 13376 ts_tablet_manager.cc:1403] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.005s
I20260709 13:05:48.022362 13379 ts_tablet_manager.cc:1434] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:05:48.023941 13375 raft_consensus.cc:359] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.024621 13376 raft_consensus.cc:359] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.025849 13376 raft_consensus.cc:385] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.026266 13376 raft_consensus.cc:740] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.027251 13376 consensus_queue.cc:260] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.028512 13379 tablet_bootstrap.cc:492] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:48.030149 13376 ts_tablet_manager.cc:1434] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.007s
I20260709 13:05:48.031493 13376 tablet_bootstrap.cc:492] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:48.025110 13375 raft_consensus.cc:385] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.033613 13299 raft_consensus.cc:493] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.033767 13375 raft_consensus.cc:740] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.034475 13299 raft_consensus.cc:515] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.034952 13375 consensus_queue.cc:260] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.037945 13375 ts_tablet_manager.cc:1434] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:05:48.038374 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450e4cfaaaf5453a8d06427a21d8abef" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:48.039140 13142 raft_consensus.cc:2468] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:48.039211 13299 leader_election.cc:290] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.041323 13033 leader_election.cc:304] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:48.043069 13375 tablet_bootstrap.cc:492] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:48.046025 13299 raft_consensus.cc:2804] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.046837 13299 raft_consensus.cc:493] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.047483 13299 raft_consensus.cc:3060] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.056155 13363 raft_consensus.cc:515] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.063942 13396 raft_consensus.cc:493] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.067616 13396 raft_consensus.cc:515] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.070725 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae440740b8324a3190399bba1d3b1c79" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:48.070960 13396 leader_election.cc:290] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:48.071609 13065 raft_consensus.cc:3060] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.072041 13375 tablet_bootstrap.cc:654] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:48.072999 13182 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:42761, user_credentials={real_user=slave}} blocked reactor thread for 61660.4us
I20260709 13:05:48.064742 13376 tablet_bootstrap.cc:654] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.075681 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f6e683b23f4646830ba9885c9765c4" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
W20260709 13:05:48.078307 13182 leader_election.cc:343] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.065688 13394 raft_consensus.cc:493] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.078341 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450e4cfaaaf5453a8d06427a21d8abef" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
W20260709 13:05:48.080025 13031 leader_election.cc:343] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7674c4e544d475aaae24558660059ee (127.0.107.195:34911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.079909 13394 raft_consensus.cc:515] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.082839 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f6e683b23f4646830ba9885c9765c4" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:48.083272 13067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2db696e40443fb9e93c2e64498685" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:48.084043 13067 raft_consensus.cc:2468] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 0.
I20260709 13:05:48.085353 13375 tablet_bootstrap.cc:492] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
W20260709 13:05:48.085186 13179 leader_election.cc:343] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.085644 13394 leader_election.cc:290] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:48.086086 13375 ts_tablet_manager.cc:1403] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.011s
I20260709 13:05:48.086282 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2db696e40443fb9e93c2e64498685" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:48.086072 13179 leader_election.cc:304] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7674c4e544d475aaae24558660059ee; no voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:48.087417 13142 raft_consensus.cc:2468] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 0.
I20260709 13:05:48.087610 13394 raft_consensus.cc:2749] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:48.088663 13179 leader_election.cc:304] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:48.089700 13394 raft_consensus.cc:2804] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.089885 13065 raft_consensus.cc:2468] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 1.
I20260709 13:05:48.090325 13394 raft_consensus.cc:493] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.089746 13375 raft_consensus.cc:359] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.090797 13394 raft_consensus.cc:3060] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.090951 13375 raft_consensus.cc:385] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.091346 13375 raft_consensus.cc:740] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.092243 13375 consensus_queue.cc:260] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.093686 13363 leader_election.cc:290] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:48.093573 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae440740b8324a3190399bba1d3b1c79" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:48.093519 13179 leader_election.cc:304] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:48.095474 13375 ts_tablet_manager.cc:1434] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:05:48.095535 13142 raft_consensus.cc:3060] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.095984 13376 tablet_bootstrap.cc:492] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:48.096577 13376 ts_tablet_manager.cc:1403] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.065s	user 0.011s	sys 0.006s
I20260709 13:05:48.096853 13375 tablet_bootstrap.cc:492] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:48.100412 13376 raft_consensus.cc:359] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.101430 13376 raft_consensus.cc:385] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.101854 13376 raft_consensus.cc:740] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.096719 13396 raft_consensus.cc:2804] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.102751 13376 consensus_queue.cc:260] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.106194 13375 tablet_bootstrap.cc:654] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.094919 13379 tablet_bootstrap.cc:654] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.109964 13376 ts_tablet_manager.cc:1434] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:05:48.111545 13376 tablet_bootstrap.cc:492] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:48.112737 13142 raft_consensus.cc:2468] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 1.
I20260709 13:05:48.116998 13396 raft_consensus.cc:697] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:48.118554 13299 raft_consensus.cc:515] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.118520 13396 consensus_queue.cc:237] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.122696 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450e4cfaaaf5453a8d06427a21d8abef" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:48.123672 13142 raft_consensus.cc:3060] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.125455 13376 tablet_bootstrap.cc:654] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.126885 13375 tablet_bootstrap.cc:492] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:48.127606 13375 ts_tablet_manager.cc:1403] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.012s
I20260709 13:05:48.131897 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450e4cfaaaf5453a8d06427a21d8abef" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:48.132077 13375 raft_consensus.cc:359] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.133170 13375 raft_consensus.cc:385] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:48.134824 13031 leader_election.cc:343] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7674c4e544d475aaae24558660059ee (127.0.107.195:34911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.136729 13375 raft_consensus.cc:740] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.137531 13299 leader_election.cc:290] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.137907 13375 consensus_queue.cc:260] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.141295 13379 tablet_bootstrap.cc:492] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:48.143328 13379 ts_tablet_manager.cc:1403] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.115s	user 0.014s	sys 0.004s
I20260709 13:05:48.144409 13375 ts_tablet_manager.cc:1434] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:05:48.146124 13375 tablet_bootstrap.cc:492] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:48.150846 13142 raft_consensus.cc:2468] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:48.141721 13394 raft_consensus.cc:515] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.153086 13033 leader_election.cc:304] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: c7674c4e544d475aaae24558660059ee
I20260709 13:05:48.154884 13299 raft_consensus.cc:2804] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.155558 13299 raft_consensus.cc:697] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:48.156282 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2db696e40443fb9e93c2e64498685" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:48.156455 13299 consensus_queue.cc:237] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.157050 13065 raft_consensus.cc:3060] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.153359 13375 tablet_bootstrap.cc:654] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.155146 13394 leader_election.cc:290] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:48.160041 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2db696e40443fb9e93c2e64498685" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:48.160728 13141 raft_consensus.cc:3060] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.161111 13379 raft_consensus.cc:359] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.161991 13379 raft_consensus.cc:385] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.162292 13379 raft_consensus.cc:740] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.163113 13379 consensus_queue.cc:260] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.158217 12965 catalog_manager.cc:5697] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 0 to 1, leader changed from <none> to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 1 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:48.166605 13065 raft_consensus.cc:2468] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 1.
I20260709 13:05:48.170406 13179 leader_election.cc:304] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:48.168195 13379 ts_tablet_manager.cc:1434] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:05:48.171494 13394 raft_consensus.cc:2804] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.172431 13394 raft_consensus.cc:697] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:48.172643 13379 tablet_bootstrap.cc:492] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:48.173378 13394 consensus_queue.cc:237] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.168780 13376 tablet_bootstrap.cc:492] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:48.174961 13376 ts_tablet_manager.cc:1403] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.064s	user 0.021s	sys 0.016s
I20260709 13:05:48.179263 13141 raft_consensus.cc:2468] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 1.
W20260709 13:05:48.184805 13013 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:48.178797 13376 raft_consensus.cc:359] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.189291 13376 raft_consensus.cc:385] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.189682 13376 raft_consensus.cc:740] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.190451 13376 consensus_queue.cc:260] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.192715 13376 ts_tablet_manager.cc:1434] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:05:48.193857 13376 tablet_bootstrap.cc:492] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73: Bootstrap starting.
I20260709 13:05:48.200179 13394 raft_consensus.cc:493] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.201061 13394 raft_consensus.cc:515] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.204291 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:48.205607 13394 leader_election.cc:290] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:48.206310 13140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:48.207160 13140 raft_consensus.cc:2468] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 0.
W20260709 13:05:48.208606 13179 leader_election.cc:343] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.209168 13379 tablet_bootstrap.cc:654] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.208069 12962 catalog_manager.cc:5697] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:48.209249 13179 leader_election.cc:304] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 197e5fb7a05e45069d5fe5acb8484e73
I20260709 13:05:48.211151 13394 raft_consensus.cc:2804] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.211629 13394 raft_consensus.cc:493] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.212028 13394 raft_consensus.cc:3060] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.214426 13299 raft_consensus.cc:493] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.215013 13299 raft_consensus.cc:515] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.219558 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e4a050557d4bb781e7c518b48dda1c" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:48.220057 13376 tablet_bootstrap.cc:654] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.220422 13141 raft_consensus.cc:2468] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:48.222170 13299 leader_election.cc:290] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.223299 13394 raft_consensus.cc:515] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.224944 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e4a050557d4bb781e7c518b48dda1c" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:48.226531 13215 raft_consensus.cc:2468] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:48.221980 13033 leader_election.cc:304] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:48.228508 13299 raft_consensus.cc:2804] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.228858 13299 raft_consensus.cc:493] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.229223 13299 raft_consensus.cc:3060] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.230768 13375 tablet_bootstrap.cc:492] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:48.231487 13375 ts_tablet_manager.cc:1403] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.086s	user 0.024s	sys 0.010s
I20260709 13:05:48.235190 13375 raft_consensus.cc:359] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.238796 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:48.241827 13375 raft_consensus.cc:385] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.241340 13299 raft_consensus.cc:515] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.242215 13375 raft_consensus.cc:740] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.240206 13394 leader_election.cc:290] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:48.245880 13299 leader_election.cc:290] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.245927 13375 consensus_queue.cc:260] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.247032 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e4a050557d4bb781e7c518b48dda1c" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:48.247761 13142 raft_consensus.cc:3060] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.247210 12965 catalog_manager.cc:5697] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 0 to 1, leader changed from <none> to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 1 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:48.249059 13375 ts_tablet_manager.cc:1434] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:05:48.242008 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
W20260709 13:05:48.243001 13179 leader_election.cc:343] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.245122 13376 tablet_bootstrap.cc:492] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73: No bootstrap required, opened a new log
I20260709 13:05:48.247650 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e4a050557d4bb781e7c518b48dda1c" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:48.250733 13376 ts_tablet_manager.cc:1403] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73: Time spent bootstrapping tablet: real 0.057s	user 0.010s	sys 0.007s
I20260709 13:05:48.250808 13215 raft_consensus.cc:3060] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.250159 13141 raft_consensus.cc:3060] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.255288 13376 raft_consensus.cc:359] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.256024 13376 raft_consensus.cc:385] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.256296 13376 raft_consensus.cc:740] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.257038 13376 consensus_queue.cc:260] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.259321 13376 ts_tablet_manager.cc:1434] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:48.250121 13375 tablet_bootstrap.cc:492] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0: Bootstrap starting.
I20260709 13:05:48.262595 13142 raft_consensus.cc:2468] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:48.264062 13033 leader_election.cc:304] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:48.265616 13398 raft_consensus.cc:2804] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.266170 13398 raft_consensus.cc:697] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:48.266901 13398 consensus_queue.cc:237] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.267382 13375 tablet_bootstrap.cc:654] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.272285 13266 raft_consensus.cc:493] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.272907 13266 raft_consensus.cc:515] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.275470 13266 leader_election.cc:290] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.277437 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5ee8c1d8d64a4597c891bcaace5176" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:48.278290 13065 raft_consensus.cc:2468] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 0.
I20260709 13:05:48.279348 13215 raft_consensus.cc:2468] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:48.279850 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5ee8c1d8d64a4597c891bcaace5176" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:48.282675 13105 leader_election.cc:304] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:48.283397 13141 raft_consensus.cc:2468] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 1.
W20260709 13:05:48.283891 13106 leader_election.cc:343] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7674c4e544d475aaae24558660059ee (127.0.107.195:34911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.239279 13379 tablet_bootstrap.cc:492] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:48.284559 13266 raft_consensus.cc:2804] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.285050 13266 raft_consensus.cc:493] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.285624 13266 raft_consensus.cc:3060] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.285182 13379 ts_tablet_manager.cc:1403] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.113s	user 0.017s	sys 0.004s
I20260709 13:05:48.285488 13182 leader_election.cc:304] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 197e5fb7a05e45069d5fe5acb8484e73
I20260709 13:05:48.292212 13394 raft_consensus.cc:2804] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.292773 13394 raft_consensus.cc:697] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:48.293761 13394 consensus_queue.cc:237] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.299448 13299 raft_consensus.cc:493] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.300174 13299 raft_consensus.cc:515] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.300660 13266 raft_consensus.cc:515] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.302994 13299 leader_election.cc:290] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.305840 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5ee8c1d8d64a4597c891bcaace5176" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:48.306030 13379 raft_consensus.cc:359] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.304171 13375 tablet_bootstrap.cc:492] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0: No bootstrap required, opened a new log
I20260709 13:05:48.304262 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f6e683b23f4646830ba9885c9765c4" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:48.306566 13065 raft_consensus.cc:3060] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.308773 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5ee8c1d8d64a4597c891bcaace5176" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
W20260709 13:05:48.310494 13106 leader_election.cc:343] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7674c4e544d475aaae24558660059ee (127.0.107.195:34911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.303476 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f6e683b23f4646830ba9885c9765c4" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:48.307359 13375 ts_tablet_manager.cc:1403] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent bootstrapping tablet: real 0.057s	user 0.023s	sys 0.010s
I20260709 13:05:48.311738 13141 raft_consensus.cc:2468] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:48.310950 12962 catalog_manager.cc:5697] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:48.313145 13033 leader_election.cc:304] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:48.314781 13375 raft_consensus.cc:359] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.315496 13375 raft_consensus.cc:385] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.315789 13299 raft_consensus.cc:2804] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.307351 13214 raft_consensus.cc:2468] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:48.316625 13299 raft_consensus.cc:493] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.317135 13299 raft_consensus.cc:3060] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.318500 13398 raft_consensus.cc:493] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.319087 13398 raft_consensus.cc:515] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.314285 13379 raft_consensus.cc:385] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.323634 13398 leader_election.cc:290] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.323776 13379 raft_consensus.cc:740] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.323071 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76658aa264e54c3bad96d43577ca3db0" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:48.323076 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76658aa264e54c3bad96d43577ca3db0" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:48.313861 13266 leader_election.cc:290] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.325573 13379 consensus_queue.cc:260] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.326674 13214 raft_consensus.cc:2468] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:48.329867 13031 leader_election.cc:304] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:48.315838 13375 raft_consensus.cc:740] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.332455 13398 raft_consensus.cc:2804] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.333261 13398 raft_consensus.cc:493] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.334043 13379 ts_tablet_manager.cc:1434] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 13:05:48.334003 13398 raft_consensus.cc:3060] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.332741 13375 consensus_queue.cc:260] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.335680 13141 raft_consensus.cc:2468] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 0.
I20260709 13:05:48.339890 13375 ts_tablet_manager.cc:1434] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 13:05:48.333108 13065 raft_consensus.cc:2468] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 1.
I20260709 13:05:48.341109 13379 tablet_bootstrap.cc:492] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:48.342021 13105 leader_election.cc:304] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: c7674c4e544d475aaae24558660059ee
I20260709 13:05:48.344017 13380 raft_consensus.cc:2804] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.345963 13380 raft_consensus.cc:697] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Becoming Leader. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:48.347980 13380 consensus_queue.cc:237] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.353698 13398 raft_consensus.cc:515] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.354444 13299 raft_consensus.cc:515] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.359402 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76658aa264e54c3bad96d43577ca3db0" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:48.359899 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f6e683b23f4646830ba9885c9765c4" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:48.360406 13141 raft_consensus.cc:3060] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.361011 13142 raft_consensus.cc:3060] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.359454 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f6e683b23f4646830ba9885c9765c4" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:48.367583 13214 raft_consensus.cc:3060] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.370438 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76658aa264e54c3bad96d43577ca3db0" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:48.371371 13215 raft_consensus.cc:3060] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.371771 13142 raft_consensus.cc:2468] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:48.373288 13033 leader_election.cc:304] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:48.374287 13397 raft_consensus.cc:2804] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.363154 13299 leader_election.cc:290] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.376214 13398 leader_election.cc:290] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.381091 13215 raft_consensus.cc:2468] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:48.384292 13141 raft_consensus.cc:2468] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:48.395021 13214 raft_consensus.cc:2468] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 1.
I20260709 13:05:48.385613 13397 raft_consensus.cc:697] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:48.389513 12965 catalog_manager.cc:5697] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 0 to 1, leader changed from <none> to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 1 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:48.392928 12962 catalog_manager.cc:5697] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:48.408166 13397 consensus_queue.cc:237] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.390520 13379 tablet_bootstrap.cc:654] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.386348 13031 leader_election.cc:304] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:48.411531 13420 raft_consensus.cc:2804] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.412137 13420 raft_consensus.cc:697] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:48.413272 13420 consensus_queue.cc:237] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.431200 12965 catalog_manager.cc:5697] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:48.434482 13379 tablet_bootstrap.cc:492] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:48.435385 13380 raft_consensus.cc:493] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.435963 13380 raft_consensus.cc:515] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.436429 13379 ts_tablet_manager.cc:1403] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.096s	user 0.018s	sys 0.018s
I20260709 13:05:48.438496 13380 leader_election.cc:290] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.438498 13420 raft_consensus.cc:493] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.439538 13420 raft_consensus.cc:515] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.440241 13379 raft_consensus.cc:359] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.441169 13379 raft_consensus.cc:385] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.441874 13379 raft_consensus.cc:740] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.442130 13420 leader_election.cc:290] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.441969 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f9bf328c0b4484b1bcb051cbddaca3" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73" is_pre_election: true
I20260709 13:05:48.442821 13379 consensus_queue.cc:260] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.443434 13065 raft_consensus.cc:2468] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 0.
I20260709 13:05:48.443930 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f9bf328c0b4484b1bcb051cbddaca3" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:48.444257 13216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee" is_pre_election: true
I20260709 13:05:48.444581 13215 raft_consensus.cc:2468] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 0.
I20260709 13:05:48.445533 13379 ts_tablet_manager.cc:1434] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:05:48.445820 13106 leader_election.cc:304] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:48.446682 13379 tablet_bootstrap.cc:492] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee: Bootstrap starting.
I20260709 13:05:48.447296 13380 raft_consensus.cc:2804] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.447712 13380 raft_consensus.cc:493] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.448058 13380 raft_consensus.cc:3060] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.443339 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" is_pre_election: true
I20260709 13:05:48.450675 13141 raft_consensus.cc:2393] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 197e5fb7a05e45069d5fe5acb8484e73 in current term 1: Already voted for candidate c7674c4e544d475aaae24558660059ee in this term.
I20260709 13:05:48.452633 13031 leader_election.cc:304] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73; no voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:48.453848 13420 raft_consensus.cc:3060] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.463033 12962 catalog_manager.cc:5697] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:48.470124 13380 raft_consensus.cc:515] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.471515 13420 raft_consensus.cc:2749] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:48.472553 13379 tablet_bootstrap.cc:654] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.474036 13065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f9bf328c0b4484b1bcb051cbddaca3" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:48.474882 13065 raft_consensus.cc:3060] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.474962 13380 leader_election.cc:290] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.479305 13379 tablet_bootstrap.cc:492] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee: No bootstrap required, opened a new log
I20260709 13:05:48.479771 13379 ts_tablet_manager.cc:1403] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.004s
I20260709 13:05:48.482524 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f9bf328c0b4484b1bcb051cbddaca3" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:48.482913 13065 raft_consensus.cc:2468] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 1.
I20260709 13:05:48.482594 13379 raft_consensus.cc:359] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.483354 13214 raft_consensus.cc:3060] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.483595 13379 raft_consensus.cc:385] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.483992 13379 raft_consensus.cc:740] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.484257 13105 leader_election.cc:304] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:48.485414 13380 raft_consensus.cc:2804] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.484920 13379 consensus_queue.cc:260] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.486127 13380 raft_consensus.cc:697] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Becoming Leader. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:48.487136 13380 consensus_queue.cc:237] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.488065 13379 ts_tablet_manager.cc:1434] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:48.495342 13214 raft_consensus.cc:2468] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 1.
I20260709 13:05:48.497987 12962 catalog_manager.cc:5697] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:48.560837 13214 raft_consensus.cc:3060] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.561707 13420 consensus_queue.cc:1048] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.570425 13420 consensus_queue.cc:1048] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.640664 13394 consensus_queue.cc:1048] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:48.654050 13432 consensus_queue.cc:1048] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.668802 13397 consensus_queue.cc:1048] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.680708 13432 consensus_queue.cc:1048] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.681672 13214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae440740b8324a3190399bba1d3b1c79"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
mode: GRACEFUL
new_leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:48984
I20260709 13:05:48.682497 13214 raft_consensus.cc:606] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Received request to transfer leadership to TS 197e5fb7a05e45069d5fe5acb8484e73
I20260709 13:05:48.687686 13299 consensus_queue.cc:1048] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.694751 13141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a5ee8c1d8d64a4597c891bcaace5176"
dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
mode: GRACEFUL
new_leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:46042
I20260709 13:05:48.695652 13141 raft_consensus.cc:606] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Received request to transfer leadership to TS 197e5fb7a05e45069d5fe5acb8484e73
I20260709 13:05:48.701925 13432 raft_consensus.cc:993] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee: : Instructing follower 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:48.702611 13432 raft_consensus.cc:1081] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Signalling peer 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:48.709264 13064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ae440740b8324a3190399bba1d3b1c79"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:35250
I20260709 13:05:48.710175 13064 raft_consensus.cc:493] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:48.710632 13064 raft_consensus.cc:3060] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:48.715528 13394 consensus_queue.cc:1048] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.720691 13394 consensus_queue.cc:1048] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:48.706959 13214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
mode: GRACEFUL
new_leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:48984
I20260709 13:05:48.737349 13214 raft_consensus.cc:606] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Received request to transfer leadership to TS 197e5fb7a05e45069d5fe5acb8484e73
I20260709 13:05:48.746569 13067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76658aa264e54c3bad96d43577ca3db0"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
mode: GRACEFUL
new_leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:35272
I20260709 13:05:48.747388 13067 raft_consensus.cc:606] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Received request to transfer leadership to TS e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:48.746363 13064 raft_consensus.cc:515] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.754809 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae440740b8324a3190399bba1d3b1c79" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:48.755318 13064 leader_election.cc:290] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.755718 13142 raft_consensus.cc:3060] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:48.756762 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae440740b8324a3190399bba1d3b1c79" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:48.758409 13394 consensus_queue.cc:1048] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.759209 13214 raft_consensus.cc:3055] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:48.759721 13214 raft_consensus.cc:740] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:48.761345 13214 consensus_queue.cc:260] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.763008 13214 raft_consensus.cc:3060] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:48.767984 13142 raft_consensus.cc:2468] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:48.769583 13033 leader_election.cc:304] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:48.770759 13397 raft_consensus.cc:2804] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:48.771450 13397 raft_consensus.cc:697] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:48.772284 13214 raft_consensus.cc:2468] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:48.773136 13397 consensus_queue.cc:237] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.845638 13215 raft_consensus.cc:3060] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.850215 13394 raft_consensus.cc:993] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee: : Instructing follower 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:48.851253 13394 raft_consensus.cc:1081] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Signalling peer 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:48.859325 13064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "450e4cfaaaf5453a8d06427a21d8abef"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
mode: GRACEFUL
new_leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:35272
I20260709 13:05:48.860683 13064 raft_consensus.cc:606] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Received request to transfer leadership to TS e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:48.860659 13397 consensus_queue.cc:1048] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:48.878002 13216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85b2db696e40443fb9e93c2e64498685"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
mode: GRACEFUL
new_leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:48984
I20260709 13:05:48.879396 13216 raft_consensus.cc:606] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Received request to transfer leadership to TS e0cf97be34bc41638dc9d31d945b0dd0
I20260709 13:05:48.883798 13418 consensus_queue.cc:1048] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.862586 13067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:35250
I20260709 13:05:48.896813 13418 consensus_queue.cc:1048] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.900787 13420 raft_consensus.cc:993] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73: : Instructing follower e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:48.902007 13299 raft_consensus.cc:1081] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Signalling peer e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:48.901831 13067 raft_consensus.cc:493] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:48.903055 13067 raft_consensus.cc:3060] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:48.912743 13141 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76658aa264e54c3bad96d43577ca3db0"
dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:46040
I20260709 13:05:48.913892 13141 raft_consensus.cc:493] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:48.914335 13141 raft_consensus.cc:3060] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:48.923537 13398 consensus_queue.cc:1048] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.926775 13064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29f6e683b23f4646830ba9885c9765c4"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
mode: GRACEFUL
new_leader_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:35272
I20260709 13:05:48.928213 13064 raft_consensus.cc:606] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Received request to transfer leadership to TS c7674c4e544d475aaae24558660059ee
I20260709 13:05:48.930696 12962 catalog_manager.cc:5697] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 1 to 2, leader changed from c7674c4e544d475aaae24558660059ee (127.0.107.195) to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:48.948448 13380 consensus_queue.cc:1048] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.957563 13141 raft_consensus.cc:515] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.959800 13142 raft_consensus.cc:1240] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Rejecting Update request from peer 197e5fb7a05e45069d5fe5acb8484e73 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:48.973131 13141 leader_election.cc:290] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:48.974771 13064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76658aa264e54c3bad96d43577ca3db0" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:48.976052 13064 raft_consensus.cc:3055] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:48.976573 13215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76658aa264e54c3bad96d43577ca3db0" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:48.976619 13064 raft_consensus.cc:740] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:48.985669 13064 consensus_queue.cc:260] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:48.987834 13064 raft_consensus.cc:3060] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:49.017546 13419 log.cc:927] Time spent T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0: Append to log took a long time: real 0.051s	user 0.009s	sys 0.000s
I20260709 13:05:49.028177 13397 raft_consensus.cc:993] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73: : Instructing follower e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:49.029160 13397 raft_consensus.cc:1081] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Signalling peer e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:49.030643 13064 raft_consensus.cc:2468] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
W20260709 13:05:49.031380 13372 log.cc:927] Time spent T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac: Append to log took a long time: real 0.057s	user 0.006s	sys 0.018s
I20260709 13:05:49.032407 13105 leader_election.cc:304] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: c7674c4e544d475aaae24558660059ee
I20260709 13:05:49.033730 13420 consensus_queue.cc:1048] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:49.039939 13442 log.cc:927] Time spent T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73: Append to log took a long time: real 0.063s	user 0.000s	sys 0.003s
I20260709 13:05:49.043176 13380 raft_consensus.cc:2804] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:49.044215 13380 raft_consensus.cc:697] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Becoming Leader. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:49.039951 13067 raft_consensus.cc:515] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.047880 13380 consensus_queue.cc:237] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.048643 13067 leader_election.cc:290] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:49.055713 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:49.056252 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:49.056641 13214 raft_consensus.cc:3055] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:49.057029 13141 raft_consensus.cc:3060] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.059257 13398 consensus_queue.cc:1048] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:49.057036 13214 raft_consensus.cc:740] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:49.062198 13299 raft_consensus.cc:993] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73: : Instructing follower c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:49.062901 13299 raft_consensus.cc:1081] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Signalling peer c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:49.064446 13214 consensus_queue.cc:260] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.067639 13213 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29f6e683b23f4646830ba9885c9765c4"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:05:49.068614 13213 raft_consensus.cc:493] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:49.069240 13213 raft_consensus.cc:3060] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.070703 13214 raft_consensus.cc:3060] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.080231 13142 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "450e4cfaaaf5453a8d06427a21d8abef"
dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:46040
I20260709 13:05:49.083904 13418 raft_consensus.cc:993] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0: : Instructing follower 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:49.089905 13142 raft_consensus.cc:493] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:49.090085 13418 raft_consensus.cc:1081] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Signalling peer 197e5fb7a05e45069d5fe5acb8484e73 to start an election
I20260709 13:05:49.090528 13142 raft_consensus.cc:3060] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.093334 13213 raft_consensus.cc:515] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.096640 13213 leader_election.cc:290] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 2 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:49.099323 13141 raft_consensus.cc:2468] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:49.108596 13141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f6e683b23f4646830ba9885c9765c4" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:49.108374 13067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29f6e683b23f4646830ba9885c9765c4" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:49.109957 13141 raft_consensus.cc:3060] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.110028 13067 raft_consensus.cc:3055] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:49.110791 13067 raft_consensus.cc:740] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:49.112054 13067 consensus_queue.cc:260] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.113606 13067 raft_consensus.cc:3060] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.123361 13065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a5ee8c1d8d64a4597c891bcaace5176"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
 from {username='slave'} at 127.0.0.1:35256
I20260709 13:05:49.124359 13065 raft_consensus.cc:493] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:49.124871 13065 raft_consensus.cc:3060] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.125106 13141 raft_consensus.cc:2468] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 2.
I20260709 13:05:49.127699 13182 leader_election.cc:304] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:49.128948 13401 raft_consensus.cc:2804] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:49.129588 13401 raft_consensus.cc:697] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 2 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:49.129923 13141 raft_consensus.cc:1240] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Rejecting Update request from peer 197e5fb7a05e45069d5fe5acb8484e73 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:05:49.130159 13033 leader_election.cc:304] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:49.131217 13398 raft_consensus.cc:2804] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:49.131021 13401 consensus_queue.cc:237] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.131398 13067 raft_consensus.cc:2468] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 2.
I20260709 13:05:49.132332 13398 raft_consensus.cc:697] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:49.134092 13398 consensus_queue.cc:237] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.141052 13449 consensus_queue.cc:1048] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:49.141673 13214 raft_consensus.cc:2468] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:49.149818 13142 raft_consensus.cc:515] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.140940 12963 catalog_manager.cc:5697] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 reported cstate change: term changed from 1 to 2, leader changed from 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193) to e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.153800 13142 leader_election.cc:290] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:49.153949 13064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450e4cfaaaf5453a8d06427a21d8abef" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:49.154841 13064 raft_consensus.cc:3055] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:49.155251 13064 raft_consensus.cc:740] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:49.155504 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450e4cfaaaf5453a8d06427a21d8abef" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:49.156356 13214 raft_consensus.cc:3060] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.156335 12965 catalog_manager.cc:5697] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 1 to 2, leader changed from 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193) to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 2 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.167888 13214 raft_consensus.cc:2468] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:49.156440 13064 consensus_queue.cc:260] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.175201 13106 leader_election.cc:304] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:49.177019 13449 raft_consensus.cc:2804] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:49.177757 13449 raft_consensus.cc:697] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Becoming Leader. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:49.179636 13449 consensus_queue.cc:237] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.188159 13064 raft_consensus.cc:3060] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.196534 13065 raft_consensus.cc:515] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.200909 12963 catalog_manager.cc:5697] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 1 to 2, leader changed from c7674c4e544d475aaae24558660059ee (127.0.107.195) to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.212260 13142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5ee8c1d8d64a4597c891bcaace5176" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:49.213037 13142 raft_consensus.cc:3055] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:49.213366 13142 raft_consensus.cc:740] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:49.214478 13142 consensus_queue.cc:260] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.215718 13142 raft_consensus.cc:3060] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.216447 12962 catalog_manager.cc:5697] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 reported cstate change: term changed from 1 to 2, leader changed from 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193) to e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.226913 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5ee8c1d8d64a4597c891bcaace5176" candidate_uuid: "197e5fb7a05e45069d5fe5acb8484e73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:49.227782 13214 raft_consensus.cc:3060] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.229928 13142 raft_consensus.cc:2468] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:49.235126 13065 leader_election.cc:290] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Requested vote from peers e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:49.224524 13064 raft_consensus.cc:2468] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:49.240476 13033 leader_election.cc:304] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:49.241457 13420 raft_consensus.cc:2804] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:49.242137 13420 raft_consensus.cc:697] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Becoming Leader. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:49.243618 13420 consensus_queue.cc:237] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.247756 13214 raft_consensus.cc:2468] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197e5fb7a05e45069d5fe5acb8484e73 in term 2.
I20260709 13:05:49.251497 13394 raft_consensus.cc:993] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee: : Instructing follower e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:49.252103 13394 raft_consensus.cc:1081] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Signalling peer e0cf97be34bc41638dc9d31d945b0dd0 to start an election
I20260709 13:05:49.256947 13142 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85b2db696e40443fb9e93c2e64498685"
dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
 from {username='slave'} at 127.0.0.1:46034
I20260709 13:05:49.257678 13142 raft_consensus.cc:493] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:49.258065 13142 raft_consensus.cc:3060] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.261518 12962 catalog_manager.cc:5697] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 reported cstate change: term changed from 1 to 2, leader changed from e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194) to 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "197e5fb7a05e45069d5fe5acb8484e73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.266360 13142 raft_consensus.cc:515] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.268476 13142 leader_election.cc:290] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), c7674c4e544d475aaae24558660059ee (127.0.107.195:34911)
I20260709 13:05:49.269716 13214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2db696e40443fb9e93c2e64498685" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7674c4e544d475aaae24558660059ee"
I20260709 13:05:49.270545 13214 raft_consensus.cc:3055] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:49.270910 13214 raft_consensus.cc:740] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:49.269774 13064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2db696e40443fb9e93c2e64498685" candidate_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:49.271970 13214 consensus_queue.cc:260] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.272764 13064 raft_consensus.cc:3060] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.273607 13214 raft_consensus.cc:3060] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.282442 13064 raft_consensus.cc:2468] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:49.282824 13214 raft_consensus.cc:2468] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0cf97be34bc41638dc9d31d945b0dd0 in term 2.
I20260709 13:05:49.285038 13106 leader_election.cc:304] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:49.286444 13449 raft_consensus.cc:2804] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:49.286998 13449 raft_consensus.cc:697] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Becoming Leader. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:49.288172 13449 consensus_queue.cc:237] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:49.296799 12962 catalog_manager.cc:5697] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 reported cstate change: term changed from 1 to 2, leader changed from c7674c4e544d475aaae24558660059ee (127.0.107.195) to e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.375871 13216 raft_consensus.cc:1275] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.378521 13397 consensus_queue.cc:1048] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:49.384505 13142 raft_consensus.cc:1275] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.386411 13397 consensus_queue.cc:1048] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:49.516532 13216 raft_consensus.cc:1275] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.518059 13397 consensus_queue.cc:1048] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:49.527865 13142 raft_consensus.cc:1275] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.529412 13398 consensus_queue.cc:1048] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:49.555831 13064 raft_consensus.cc:1275] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.557318 13401 consensus_queue.cc:1048] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:49.567431 13142 raft_consensus.cc:1275] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.570308 13432 consensus_queue.cc:1048] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:49.596060 13064 raft_consensus.cc:1275] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.597663 13380 consensus_queue.cc:1048] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:49.605423 13065 raft_consensus.cc:1275] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.608963 13380 consensus_queue.cc:1048] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:49.609982 13216 raft_consensus.cc:1275] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.612182 13380 consensus_queue.cc:1048] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:49.633212 13214 raft_consensus.cc:3060] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.643800 13214 raft_consensus.cc:1275] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.645673 13456 consensus_queue.cc:1048] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:49.784256 13214 raft_consensus.cc:1275] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.786000 13065 raft_consensus.cc:1275] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.786173 13420 consensus_queue.cc:1048] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:49.787997 13456 consensus_queue.cc:1048] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:49.797277 13142 raft_consensus.cc:1275] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Refusing update from remote peer 197e5fb7a05e45069d5fe5acb8484e73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.799486 13420 consensus_queue.cc:1048] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:49.816947 13214 raft_consensus.cc:1275] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Refusing update from remote peer e0cf97be34bc41638dc9d31d945b0dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:49.819444 13457 consensus_queue.cc:1048] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:51.942036 12965 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57770:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:05:51.975805 12965 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57770:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:05:52.019311   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:05:52.454555 13505 log.cc:927] Time spent T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
W20260709 13:05:52.451185 13105 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:34185, user_credentials={real_user=slave}} blocked reactor thread for 80241.1us
W20260709 13:05:52.462086 13497 log.cc:927] Time spent T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee: Append to log took a long time: real 0.137s	user 0.002s	sys 0.001s
I20260709 13:05:52.458143 13511 tablet.cc:1724] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:05:52.464084 13504 log.cc:927] Time spent T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:05:52.471850 13512 tablet.cc:1724] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:05:52.494882 13506 log.cc:927] Time spent T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73: Append to log took a long time: real 0.092s	user 0.000s	sys 0.002s
W20260709 13:05:52.516619 13510 log.cc:927] Time spent T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 13:05:52.516836 13494 log.cc:927] Time spent T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73: Append to log took a long time: real 0.197s	user 0.004s	sys 0.000s
W20260709 13:05:52.525303 13515 log.cc:927] Time spent T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0: Append to log took a long time: real 0.069s	user 0.000s	sys 0.003s
I20260709 13:05:52.525327 13062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
mode: GRACEFUL
new_leader_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:48036
I20260709 13:05:52.562558 13517 tablet.cc:1724] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.582314 13524 tablet.cc:1724] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.566864 13062 raft_consensus.cc:606] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Received request to transfer leadership to TS c7674c4e544d475aaae24558660059ee
I20260709 13:05:52.595794 13527 tablet.cc:1724] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.598279 13525 tablet.cc:1724] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:05:52.611389 13105 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:34185, user_credentials={real_user=slave}} blocked reactor thread for 54374.8us
I20260709 13:05:52.614542 13523 tablet.cc:1724] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.614831 13516 tablet.cc:1724] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:05:52.626816 13513 log.cc:927] Time spent T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:05:52.623446 13182 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:42761, user_credentials={real_user=slave}} blocked reactor thread for 43751.3us
I20260709 13:05:52.614655 13518 tablet.cc:1724] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:05:52.654139 13168 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:7a5ee8c1d8d64a4597c891bcaace5176)
I20260709 13:05:52.648197 13526 tablet.cc:1724] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:05:52.655404 13168 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:450e4cfaaaf5453a8d06427a21d8abef)
I20260709 13:05:52.652094 13528 tablet.cc:1724] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.661366 13529 tablet.cc:1724] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:05:52.662904 13520 log.cc:927] Time spent T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
W20260709 13:05:52.662914 13519 log.cc:927] Time spent T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0: Append to log took a long time: real 0.119s	user 0.002s	sys 0.000s
I20260709 13:05:52.675338 13511 tablet.cc:1724] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.686667 13498 raft_consensus.cc:993] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73: : Instructing follower c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:52.687232 13498 raft_consensus.cc:1081] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Signalling peer c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:52.692538 13210 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:05:52.693439 13210 raft_consensus.cc:493] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:52.693915 13210 raft_consensus.cc:3060] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:52.695858 13242 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:29f6e683b23f4646830ba9885c9765c4)
I20260709 13:05:52.709966 12965 catalog_manager.cc:5789] TS 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185) does not have the latest schema for tablet 012804e1fc9244889c16e6d4606f47ce (table soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]). Expected version 1 got 0
I20260709 13:05:52.724337 13523 tablet.cc:1724] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.733793 13517 tablet.cc:1724] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.746225 13525 tablet.cc:1724] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.750085 13528 tablet.cc:1724] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.758934 13518 tablet.cc:1724] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.780133 13523 tablet.cc:1724] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.769770 13524 tablet.cc:1724] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.753907 13210 raft_consensus.cc:515] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:52.795601 13216 raft_consensus.cc:1240] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 3 FOLLOWER]: Rejecting Update request from peer 197e5fb7a05e45069d5fe5acb8484e73 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:52.800367 13493 consensus_queue.cc:1059] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:52.801466 13486 raft_consensus.cc:3055] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:52.801874 13486 raft_consensus.cc:740] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:52.803182 13138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80f9bf328c0b4484b1bcb051cbddaca3"
dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
mode: GRACEFUL
new_leader_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:53168
I20260709 13:05:52.802878 13486 consensus_queue.cc:260] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:52.803855 13138 raft_consensus.cc:606] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Received request to transfer leadership to TS c7674c4e544d475aaae24558660059ee
I20260709 13:05:52.804323 13486 raft_consensus.cc:3060] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:52.805604   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:05:52.806421   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:52.769898 13512 tablet.cc:1724] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.787288 13529 tablet.cc:1724] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.782104 13516 tablet.cc:1724] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.772898 13527 tablet.cc:1724] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.821867 13138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107"
dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
mode: GRACEFUL
new_leader_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:53168
I20260709 13:05:52.823825 13138 raft_consensus.cc:606] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Received request to transfer leadership to TS c7674c4e544d475aaae24558660059ee
I20260709 13:05:52.822809 13511 tablet.cc:1724] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.831620 13210 leader_election.cc:290] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 3 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:52.835886 13066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:52.788501 13526 tablet.cc:1724] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:52.838886 13138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012804e1fc9244889c16e6d4606f47ce" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:52.839743 13138 raft_consensus.cc:3060] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:52.868126 13066 raft_consensus.cc:2468] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 3.
I20260709 13:05:52.874281 13062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "240ee459545f4baca128c935ccbf9645"
dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
mode: GRACEFUL
new_leader_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:48036
I20260709 13:05:52.875017 13062 raft_consensus.cc:606] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Received request to transfer leadership to TS c7674c4e544d475aaae24558660059ee
I20260709 13:05:52.877146 13179 leader_election.cc:304] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:52.878219 13401 raft_consensus.cc:2804] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:52.878700 13401 raft_consensus.cc:697] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 3 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:52.879487 13401 consensus_queue.cc:237] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:52.881798   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:52.884488   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:52.887641 13138 raft_consensus.cc:2468] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 3.
I20260709 13:05:52.901456 12963 catalog_manager.cc:5697] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 2 to 3, leader changed from 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193) to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 3 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:52.906526 12963 catalog_manager.cc:5789] TS c7674c4e544d475aaae24558660059ee (127.0.107.195:34911) does not have the latest schema for tablet 012804e1fc9244889c16e6d4606f47ce (table soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba]). Expected version 1 got 0
I20260709 13:05:52.929113 12962 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac: soft_deleted_table [id=fae1e1fd29a9403c9e7491b63ddca4ba] alter complete (version 1)
I20260709 13:05:53.030609 13521 raft_consensus.cc:993] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0: : Instructing follower c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:53.031348 13508 raft_consensus.cc:1081] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Signalling peer c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:53.033602 13210 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:48968
I20260709 13:05:53.034271 13210 raft_consensus.cc:493] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:53.034749 13210 raft_consensus.cc:3060] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:53.044857 13210 raft_consensus.cc:515] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:53.047933 13210 leader_election.cc:290] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 3 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:53.048967 13062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:53.049825 13062 raft_consensus.cc:3060] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:53.050388 13138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17700ef9cb6146169ead3706cd5ca107" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:53.051100 13138 raft_consensus.cc:3055] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:53.051429 13138 raft_consensus.cc:740] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:53.052387 13138 consensus_queue.cc:260] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:53.053957 13138 raft_consensus.cc:3060] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:53.059749 13493 raft_consensus.cc:993] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73: : Instructing follower c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:53.060482 13493 raft_consensus.cc:1081] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Signalling peer c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:53.062335 13062 raft_consensus.cc:2468] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 3.
I20260709 13:05:53.063812 13179 leader_election.cc:304] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:53.064236 13138 raft_consensus.cc:2468] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 3.
I20260709 13:05:53.064510 13216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "240ee459545f4baca128c935ccbf9645"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:05:53.064793 13401 raft_consensus.cc:2804] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:53.065351 13216 raft_consensus.cc:493] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:53.065460 13401 raft_consensus.cc:697] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 3 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:53.065874 13216 raft_consensus.cc:3060] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:53.066690 13401 consensus_queue.cc:237] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:53.075934 13216 raft_consensus.cc:515] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:53.078888 13216 leader_election.cc:290] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 3 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:53.079521 13062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240ee459545f4baca128c935ccbf9645" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:53.078915 12965 catalog_manager.cc:5697] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 2 to 3, leader changed from e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194) to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 3 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:53.080585 13062 raft_consensus.cc:3055] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:53.080618 13138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240ee459545f4baca128c935ccbf9645" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:53.080999 13062 raft_consensus.cc:740] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 197e5fb7a05e45069d5fe5acb8484e73, State: Running, Role: LEADER
I20260709 13:05:53.081336 13138 raft_consensus.cc:3060] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:53.081929 13062 consensus_queue.cc:260] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:53.083698 13062 raft_consensus.cc:3060] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:53.091266 13138 raft_consensus.cc:2468] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 3.
I20260709 13:05:53.093106 13182 leader_election.cc:304] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7674c4e544d475aaae24558660059ee, e0cf97be34bc41638dc9d31d945b0dd0; no voters: 
I20260709 13:05:53.094655 13401 raft_consensus.cc:2804] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:53.095206 13401 raft_consensus.cc:697] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 3 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:53.096194 13401 consensus_queue.cc:237] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:53.097201 13062 raft_consensus.cc:2468] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 3.
I20260709 13:05:53.108428 12965 catalog_manager.cc:5697] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 2 to 3, leader changed from 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193) to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 3 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:53.225121 13521 raft_consensus.cc:993] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0: : Instructing follower c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:53.226078 13521 raft_consensus.cc:1081] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Signalling peer c7674c4e544d475aaae24558660059ee to start an election
I20260709 13:05:53.228314 13216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "80f9bf328c0b4484b1bcb051cbddaca3"
dest_uuid: "c7674c4e544d475aaae24558660059ee"
 from {username='slave'} at 127.0.0.1:48968
I20260709 13:05:53.229009 13216 raft_consensus.cc:493] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:53.229446 13216 raft_consensus.cc:3060] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.237424 13216 raft_consensus.cc:515] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:53.240623 13216 leader_election.cc:290] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 2 election: Requested vote from peers 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185), e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761)
I20260709 13:05:53.243049 13138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f9bf328c0b4484b1bcb051cbddaca3" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "e0cf97be34bc41638dc9d31d945b0dd0"
I20260709 13:05:53.244287 13138 raft_consensus.cc:3055] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:53.244819 13138 raft_consensus.cc:740] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e0cf97be34bc41638dc9d31d945b0dd0, State: Running, Role: LEADER
I20260709 13:05:53.246423 13062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f9bf328c0b4484b1bcb051cbddaca3" candidate_uuid: "c7674c4e544d475aaae24558660059ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "197e5fb7a05e45069d5fe5acb8484e73"
I20260709 13:05:53.245993 13138 consensus_queue.cc:260] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:53.247139 13062 raft_consensus.cc:3060] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.247890 13138 raft_consensus.cc:3060] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:53.257292 13138 raft_consensus.cc:2468] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 2.
I20260709 13:05:53.257611 13062 raft_consensus.cc:2468] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7674c4e544d475aaae24558660059ee in term 2.
I20260709 13:05:53.259193 13179 leader_election.cc:304] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197e5fb7a05e45069d5fe5acb8484e73, c7674c4e544d475aaae24558660059ee; no voters: 
I20260709 13:05:53.260432 13401 raft_consensus.cc:2804] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:53.261006 13401 raft_consensus.cc:697] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 2 LEADER]: Becoming Leader. State: Replica: c7674c4e544d475aaae24558660059ee, State: Running, Role: LEADER
I20260709 13:05:53.262176 13401 consensus_queue.cc:237] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } }
I20260709 13:05:53.265653 13138 raft_consensus.cc:1275] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:53.267424 13401 consensus_queue.cc:1048] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:53.274987 13529 tablet.cc:1724] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:53.273188 12965 catalog_manager.cc:5697] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee reported cstate change: term changed from 1 to 2, leader changed from e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194) to c7674c4e544d475aaae24558660059ee (127.0.107.195). New cstate: current_term: 2 leader_uuid: "c7674c4e544d475aaae24558660059ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7674c4e544d475aaae24558660059ee" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:53.284207 13062 raft_consensus.cc:1275] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:53.289680 13401 consensus_queue.cc:1048] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:53.310391 13528 tablet.cc:1719] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0: Skipping requested alter to schema version 1, tablet already version 1
I20260709 13:05:53.313616 13538 mvcc.cc:204] Tried to move back new op lower bound from 7305635239090630656 to 7305635237409234944. Current Snapshot: MvccSnapshot[applied={T|T < 7305635239090630656}]
I20260709 13:05:53.315466 13529 tablet.cc:1719] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee: Skipping requested alter to schema version 1, tablet already version 1
I20260709 13:05:53.328137 13516 tablet.cc:1719] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73: Skipping requested alter to schema version 1, tablet already version 1
I20260709 13:05:53.453464 13062 raft_consensus.cc:1275] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:53.455621 13401 consensus_queue.cc:1048] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:53.474093 13139 raft_consensus.cc:1275] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:53.475871 13541 consensus_queue.cc:1048] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:53.488909 13066 raft_consensus.cc:1275] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:53.491639 13507 consensus_queue.cc:1048] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:53.505466 13139 raft_consensus.cc:1275] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:53.507886 13541 consensus_queue.cc:1048] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:53.748569 13062 raft_consensus.cc:1275] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:53.750491 13541 consensus_queue.cc:1048] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "197e5fb7a05e45069d5fe5acb8484e73" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:53.760846 13139 raft_consensus.cc:1275] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Refusing update from remote peer c7674c4e544d475aaae24558660059ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:53.765728 13507 consensus_queue.cc:1048] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cf97be34bc41638dc9d31d945b0dd0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:53.960711   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:05:53.966913   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:05:53.967473   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:53.972807   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:05:53.973336   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:55.010624   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:05:55.016144 13105 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 10 similar messages]
W20260709 13:05:55.023047 13105 consensus_peers.cc:597] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 -> Peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Couldn't send request to peer 197e5fb7a05e45069d5fe5acb8484e73. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.062680   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:55.064098   431 tablet_replica.cc:333] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.065853   431 raft_consensus.cc:2243] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.066699   431 raft_consensus.cc:2272] T 012804e1fc9244889c16e6d4606f47ce P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.069447   431 tablet_replica.cc:333] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.070402   431 raft_consensus.cc:2243] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.070870   431 raft_consensus.cc:2272] T 85b2db696e40443fb9e93c2e64498685 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.074293   431 tablet_replica.cc:333] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.075711   431 raft_consensus.cc:2243] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.076610   431 raft_consensus.cc:2272] T 450e4cfaaaf5453a8d06427a21d8abef P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.079320   431 tablet_replica.cc:333] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.080104   431 raft_consensus.cc:2243] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.080672   431 raft_consensus.cc:2272] T 76658aa264e54c3bad96d43577ca3db0 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.083375   431 tablet_replica.cc:333] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.084339   431 raft_consensus.cc:2243] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.085964   431 raft_consensus.cc:2272] T 74e4a050557d4bb781e7c518b48dda1c P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.090341   431 tablet_replica.cc:333] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.091804   431 raft_consensus.cc:2243] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.093099   431 raft_consensus.cc:2272] T d9324f89bc5b4281ac6b495fde2aa95c P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:55.094264 13179 consensus_peers.cc:597] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee -> Peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Couldn't send request to peer 197e5fb7a05e45069d5fe5acb8484e73. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.096529   431 tablet_replica.cc:333] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.097407   431 raft_consensus.cc:2243] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.098491   431 raft_consensus.cc:2272] T cc4c9edf710c40b1a3105b6d94a01ddb P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.101454   431 tablet_replica.cc:333] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.102303   431 raft_consensus.cc:2243] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.102924   431 raft_consensus.cc:2272] T cb4fd6fa94e8418ab089fec0bc1a633d P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.108038   431 tablet_replica.cc:333] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.109596   431 raft_consensus.cc:2243] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.111749   431 raft_consensus.cc:2272] T 7a5ee8c1d8d64a4597c891bcaace5176 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:55.111866 13179 consensus_peers.cc:597] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee -> Peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Couldn't send request to peer 197e5fb7a05e45069d5fe5acb8484e73. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.119081   431 tablet_replica.cc:333] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.119844   431 raft_consensus.cc:2243] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.120532   431 raft_consensus.cc:2272] T 80f9bf328c0b4484b1bcb051cbddaca3 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:55.121312 13179 consensus_peers.cc:597] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee -> Peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Couldn't send request to peer 197e5fb7a05e45069d5fe5acb8484e73. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.123986   431 tablet_replica.cc:333] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.125111   431 raft_consensus.cc:2243] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.126194   431 raft_consensus.cc:2272] T 73438a7711574bb9a2ade51d2e041139 P 197e5fb7a05e45069d5fe5acb8484e73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.128899   431 tablet_replica.cc:333] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.129711   431 raft_consensus.cc:2243] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.130224   431 raft_consensus.cc:2272] T b6667f62e97949e582d8cc92d1e3c720 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.132999   431 tablet_replica.cc:333] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.133719   431 raft_consensus.cc:2243] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.134243   431 raft_consensus.cc:2272] T 29f6e683b23f4646830ba9885c9765c4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.136734   431 tablet_replica.cc:333] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.137377   431 raft_consensus.cc:2243] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.138700   431 raft_consensus.cc:2272] T 5bd0304336bd4f0994a5a44bef55efdf P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.141546   431 tablet_replica.cc:333] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.142422   431 raft_consensus.cc:2243] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.142987   431 raft_consensus.cc:2272] T 95b0957f35a54bc196a443411300ffc4 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.145870   431 tablet_replica.cc:333] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.146850   431 raft_consensus.cc:2243] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.147894   431 raft_consensus.cc:2272] T ae440740b8324a3190399bba1d3b1c79 P 197e5fb7a05e45069d5fe5acb8484e73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.150794   431 tablet_replica.cc:333] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.151373   431 raft_consensus.cc:2243] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.151927   431 raft_consensus.cc:2272] T 240ee459545f4baca128c935ccbf9645 P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.154991   431 tablet_replica.cc:333] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73: stopping tablet replica
I20260709 13:05:55.156081   431 raft_consensus.cc:2243] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.156661   431 raft_consensus.cc:2272] T 17700ef9cb6146169ead3706cd5ca107 P 197e5fb7a05e45069d5fe5acb8484e73 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:55.170517 13179 consensus_peers.cc:597] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee -> Peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Couldn't send request to peer 197e5fb7a05e45069d5fe5acb8484e73. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.212235   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:05:55.261276   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:05:55.313431   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:55.314457   431 tablet_replica.cc:333] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.315335   431 raft_consensus.cc:2243] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.316516   431 raft_consensus.cc:2272] T 76658aa264e54c3bad96d43577ca3db0 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.319203   431 tablet_replica.cc:333] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.319905   431 raft_consensus.cc:2243] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.320448   431 raft_consensus.cc:2272] T 7a5ee8c1d8d64a4597c891bcaace5176 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.322854   431 tablet_replica.cc:333] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.323529   431 raft_consensus.cc:2243] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.324100   431 raft_consensus.cc:2272] T 012804e1fc9244889c16e6d4606f47ce P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.326918   431 tablet_replica.cc:333] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.327674   431 raft_consensus.cc:2243] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.328653   431 raft_consensus.cc:2272] T 85b2db696e40443fb9e93c2e64498685 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.331022   431 tablet_replica.cc:333] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.331699   431 raft_consensus.cc:2243] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.332671   431 raft_consensus.cc:2272] T 450e4cfaaaf5453a8d06427a21d8abef P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.335937   431 tablet_replica.cc:333] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.336728   431 raft_consensus.cc:2243] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.337543   431 raft_consensus.cc:2272] T 74e4a050557d4bb781e7c518b48dda1c P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.341150   431 tablet_replica.cc:333] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.342263   431 raft_consensus.cc:2243] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.342978   431 raft_consensus.cc:2272] T 240ee459545f4baca128c935ccbf9645 P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.347011   431 tablet_replica.cc:333] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.347855   431 raft_consensus.cc:2243] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.348412   431 raft_consensus.cc:2272] T ae440740b8324a3190399bba1d3b1c79 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.351143   431 tablet_replica.cc:333] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.351843   431 raft_consensus.cc:2243] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.352361   431 raft_consensus.cc:2272] T 80f9bf328c0b4484b1bcb051cbddaca3 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.354952   431 tablet_replica.cc:333] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.356129   431 raft_consensus.cc:2243] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.357008   431 raft_consensus.cc:2272] T 73438a7711574bb9a2ade51d2e041139 P e0cf97be34bc41638dc9d31d945b0dd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.359791   431 tablet_replica.cc:333] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.360438   431 raft_consensus.cc:2243] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.361407   431 raft_consensus.cc:2272] T b6667f62e97949e582d8cc92d1e3c720 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.364006   431 tablet_replica.cc:333] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.364637   431 raft_consensus.cc:2243] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.365195   431 raft_consensus.cc:2272] T 29f6e683b23f4646830ba9885c9765c4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.367569   431 tablet_replica.cc:333] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.368330   431 raft_consensus.cc:2243] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.368875   431 raft_consensus.cc:2272] T 5bd0304336bd4f0994a5a44bef55efdf P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.373412   431 tablet_replica.cc:333] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.374471   431 raft_consensus.cc:2243] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.375068   431 raft_consensus.cc:2272] T 95b0957f35a54bc196a443411300ffc4 P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:55.375429 13105 consensus_peers.cc:597] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 -> Peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Couldn't send request to peer 197e5fb7a05e45069d5fe5acb8484e73. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:34185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.378001   431 tablet_replica.cc:333] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.378741   431 raft_consensus.cc:2243] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.379369   431 raft_consensus.cc:2272] T 17700ef9cb6146169ead3706cd5ca107 P e0cf97be34bc41638dc9d31d945b0dd0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.382066   431 tablet_replica.cc:333] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.382699   431 raft_consensus.cc:2243] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.383244   431 raft_consensus.cc:2272] T d9324f89bc5b4281ac6b495fde2aa95c P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.385885   431 tablet_replica.cc:333] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.386492   431 raft_consensus.cc:2243] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.387437   431 raft_consensus.cc:2272] T cc4c9edf710c40b1a3105b6d94a01ddb P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.390067   431 tablet_replica.cc:333] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0: stopping tablet replica
I20260709 13:05:55.390699   431 raft_consensus.cc:2243] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.391772   431 raft_consensus.cc:2272] T cb4fd6fa94e8418ab089fec0bc1a633d P e0cf97be34bc41638dc9d31d945b0dd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.442155   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:05:55.444377 13179 consensus_peers.cc:597] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee -> Peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Couldn't send request to peer 197e5fb7a05e45069d5fe5acb8484e73. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:34185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:55.455619 13182 consensus_peers.cc:597] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee -> Peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Couldn't send request to peer e0cf97be34bc41638dc9d31d945b0dd0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:42761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.497083   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:05:55.537070 13179 consensus_peers.cc:597] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee -> Peer 197e5fb7a05e45069d5fe5acb8484e73 (127.0.107.193:34185): Couldn't send request to peer 197e5fb7a05e45069d5fe5acb8484e73. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:34185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:55.553690 13182 consensus_peers.cc:597] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee -> Peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Couldn't send request to peer e0cf97be34bc41638dc9d31d945b0dd0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:42761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.562871   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:55.564877   431 tablet_replica.cc:333] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.566218   431 raft_consensus.cc:2243] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.566996   431 raft_consensus.cc:2272] T 7a5ee8c1d8d64a4597c891bcaace5176 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.569777   431 tablet_replica.cc:333] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.570492   431 raft_consensus.cc:2243] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.571108   431 raft_consensus.cc:2272] T 450e4cfaaaf5453a8d06427a21d8abef P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.574291   431 tablet_replica.cc:333] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.575071   431 raft_consensus.cc:2243] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.575855   431 raft_consensus.cc:2272] T 76658aa264e54c3bad96d43577ca3db0 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.579248   431 tablet_replica.cc:333] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.580525   431 raft_consensus.cc:2243] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.581341   431 raft_consensus.cc:2272] T 74e4a050557d4bb781e7c518b48dda1c P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:55.584024 13182 consensus_peers.cc:597] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee -> Peer e0cf97be34bc41638dc9d31d945b0dd0 (127.0.107.194:42761): Couldn't send request to peer e0cf97be34bc41638dc9d31d945b0dd0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:42761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:55.584573   431 tablet_replica.cc:333] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.585543   431 raft_consensus.cc:2243] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.586316   431 raft_consensus.cc:2272] T 85b2db696e40443fb9e93c2e64498685 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.589378   431 tablet_replica.cc:333] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.591199   431 raft_consensus.cc:2243] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.592434   431 raft_consensus.cc:2272] T 012804e1fc9244889c16e6d4606f47ce P c7674c4e544d475aaae24558660059ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.595651   431 tablet_replica.cc:333] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.596706   431 raft_consensus.cc:2243] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.597783   431 raft_consensus.cc:2272] T d9324f89bc5b4281ac6b495fde2aa95c P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.602187   431 tablet_replica.cc:333] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.603434   431 raft_consensus.cc:2243] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.604264   431 raft_consensus.cc:2272] T cc4c9edf710c40b1a3105b6d94a01ddb P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.606957   431 tablet_replica.cc:333] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.607637   431 raft_consensus.cc:2243] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.608439   431 raft_consensus.cc:2272] T cb4fd6fa94e8418ab089fec0bc1a633d P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.612775   431 tablet_replica.cc:333] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.613772   431 raft_consensus.cc:2243] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.615633   431 raft_consensus.cc:2272] T 80f9bf328c0b4484b1bcb051cbddaca3 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.618849   431 tablet_replica.cc:333] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.619678   431 raft_consensus.cc:2243] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.620396   431 raft_consensus.cc:2272] T b6667f62e97949e582d8cc92d1e3c720 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.623055   431 tablet_replica.cc:333] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.623718   431 raft_consensus.cc:2243] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.624265   431 raft_consensus.cc:2272] T 73438a7711574bb9a2ade51d2e041139 P c7674c4e544d475aaae24558660059ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.626817   431 tablet_replica.cc:333] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.627465   431 raft_consensus.cc:2243] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.628504   431 raft_consensus.cc:2272] T 95b0957f35a54bc196a443411300ffc4 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.631035   431 tablet_replica.cc:333] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.631642   431 raft_consensus.cc:2243] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.632577   431 raft_consensus.cc:2272] T 29f6e683b23f4646830ba9885c9765c4 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.634974   431 tablet_replica.cc:333] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.635560   431 raft_consensus.cc:2243] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.636089   431 raft_consensus.cc:2272] T 5bd0304336bd4f0994a5a44bef55efdf P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.638585   431 tablet_replica.cc:333] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.639202   431 raft_consensus.cc:2243] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.640139   431 raft_consensus.cc:2272] T 17700ef9cb6146169ead3706cd5ca107 P c7674c4e544d475aaae24558660059ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.642786   431 tablet_replica.cc:333] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.643661   431 raft_consensus.cc:2243] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:55.644258   431 raft_consensus.cc:2272] T ae440740b8324a3190399bba1d3b1c79 P c7674c4e544d475aaae24558660059ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.647732   431 tablet_replica.cc:333] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee: stopping tablet replica
I20260709 13:05:55.648365   431 raft_consensus.cc:2243] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.649458   431 raft_consensus.cc:2272] T 240ee459545f4baca128c935ccbf9645 P c7674c4e544d475aaae24558660059ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.689209   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:05:55.734239   431 master.cc:562] Master@127.0.107.254:34951 shutting down...
I20260709 13:05:55.774322   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:55.775101   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:55.775480   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4a599bdb73ef4628b5466f030ec99bac: stopping tablet replica
I20260709 13:05:55.801043   431 master.cc:584] Master@127.0.107.254:34951 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13935 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:05:55.869033   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:45415
I20260709 13:05:55.870546   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:55.878588 13559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:55.878474 13557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:55.878432 13556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:55.879606   431 server_base.cc:1061] running on GCE node
I20260709 13:05:55.880766   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:55.880963   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:55.881170   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602355881149 us; error 0 us; skew 500 ppm
I20260709 13:05:55.882329   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:55.886142   431 webserver.cc:533] Webserver started at http://127.0.107.254:40173/ using document root <none> and password file <none>
I20260709 13:05:55.887008   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:55.887179   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:55.887537   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:55.888849   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/master-0-root/instance:
uuid: "2e2333acdb674bf5acd7328fef059d81"
format_stamp: "Formatted at 2026-07-09 13:05:55 on dist-test-slave-l8g5"
I20260709 13:05:55.894239   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:05:55.900417 13564 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:55.901938   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:05:55.902335   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/master-0-root
uuid: "2e2333acdb674bf5acd7328fef059d81"
format_stamp: "Formatted at 2026-07-09 13:05:55 on dist-test-slave-l8g5"
I20260709 13:05:55.902659   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:55.928596   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:55.930569   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:55.986518   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:45415
I20260709 13:05:55.986660 13615 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:45415 every 8 connection(s)
I20260709 13:05:55.991125 13616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.005949 13616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81: Bootstrap starting.
I20260709 13:05:56.012147 13616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:56.018347 13616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81: No bootstrap required, opened a new log
I20260709 13:05:56.021363 13616 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2333acdb674bf5acd7328fef059d81" member_type: VOTER }
I20260709 13:05:56.022003 13616 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:56.022215 13616 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e2333acdb674bf5acd7328fef059d81, State: Initialized, Role: FOLLOWER
I20260709 13:05:56.022949 13616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2333acdb674bf5acd7328fef059d81" member_type: VOTER }
I20260709 13:05:56.023449 13616 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:56.023707 13616 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:56.024093 13616 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:56.032246 13616 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2333acdb674bf5acd7328fef059d81" member_type: VOTER }
I20260709 13:05:56.032972 13616 leader_election.cc:304] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e2333acdb674bf5acd7328fef059d81; no voters: 
I20260709 13:05:56.035025 13616 leader_election.cc:290] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:56.035337 13619 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:56.037205 13619 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [term 1 LEADER]: Becoming Leader. State: Replica: 2e2333acdb674bf5acd7328fef059d81, State: Running, Role: LEADER
I20260709 13:05:56.038662 13619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2333acdb674bf5acd7328fef059d81" member_type: VOTER }
I20260709 13:05:56.039330 13616 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:56.045424 13621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e2333acdb674bf5acd7328fef059d81. Latest consensus state: current_term: 1 leader_uuid: "2e2333acdb674bf5acd7328fef059d81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2333acdb674bf5acd7328fef059d81" member_type: VOTER } }
I20260709 13:05:56.046312 13621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:56.048085 13620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e2333acdb674bf5acd7328fef059d81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2333acdb674bf5acd7328fef059d81" member_type: VOTER } }
I20260709 13:05:56.048774 13628 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:56.048902 13620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:56.057760 13628 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:56.059319   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:56.072813 13628 catalog_manager.cc:1383] Generated new cluster ID: de170fc020b442a2a382d92ed8132b5b
I20260709 13:05:56.073275 13628 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:56.096071 13628 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:56.099679 13628 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:56.125155 13628 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81: Generated new TSK 0
I20260709 13:05:56.126283 13628 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:56.132737   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:56.141753 13637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:56.143046 13638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:56.145045 13640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:56.145915   431 server_base.cc:1061] running on GCE node
I20260709 13:05:56.147018   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:56.147501   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:56.147742   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602356147719 us; error 0 us; skew 500 ppm
I20260709 13:05:56.148471   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:56.153415   431 webserver.cc:533] Webserver started at http://127.0.107.193:42829/ using document root <none> and password file <none>
I20260709 13:05:56.154340   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:56.154573   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:56.154873   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:56.156188   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-0-root/instance:
uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
format_stamp: "Formatted at 2026-07-09 13:05:56 on dist-test-slave-l8g5"
I20260709 13:05:56.162220   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:05:56.167248 13645 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:56.168495   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:05:56.168913   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-0-root
uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
format_stamp: "Formatted at 2026-07-09 13:05:56 on dist-test-slave-l8g5"
I20260709 13:05:56.169229   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:56.206580   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:56.208983   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:56.211402   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:56.215029   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:56.215308   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:56.215600   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:56.215947   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:56.293622   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:42701
I20260709 13:05:56.294034 13707 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:42701 every 8 connection(s)
I20260709 13:05:56.299645   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:56.312865 13712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:56.313426 13713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:56.321807   431 server_base.cc:1061] running on GCE node
W20260709 13:05:56.322570 13715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:56.325261   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:56.325748   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:56.326004   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602356325979 us; error 0 us; skew 500 ppm
I20260709 13:05:56.326615 13708 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45415
I20260709 13:05:56.327662 13708 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:56.326886   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:56.328764 13708 heartbeater.cc:507] Master 127.0.107.254:45415 requested a full tablet report, sending...
I20260709 13:05:56.334117   431 webserver.cc:533] Webserver started at http://127.0.107.194:40665/ using document root <none> and password file <none>
I20260709 13:05:56.334926   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:56.335173   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:56.335479   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:56.336874   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-1-root/instance:
uuid: "d8474050515746e49b6550d9f1cacf9a"
format_stamp: "Formatted at 2026-07-09 13:05:56 on dist-test-slave-l8g5"
I20260709 13:05:56.337334 13581 ts_manager.cc:194] Registered new tserver with Master: 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701)
I20260709 13:05:56.340507 13581 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46024
I20260709 13:05:56.344594   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.004s
I20260709 13:05:56.351753 13720 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:56.353827   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 13:05:56.354228   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-1-root
uuid: "d8474050515746e49b6550d9f1cacf9a"
format_stamp: "Formatted at 2026-07-09 13:05:56 on dist-test-slave-l8g5"
I20260709 13:05:56.354542   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:56.380923   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:56.383113   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:56.387235   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:56.392407   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:56.392795   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:56.393100   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:56.393308   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:56.473140   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:39199
I20260709 13:05:56.473307 13782 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:39199 every 8 connection(s)
I20260709 13:05:56.479625   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:56.491664 13786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:56.494769 13787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:56.500898 13783 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45415
I20260709 13:05:56.501652 13783 heartbeater.cc:461] Registering TS with master...
W20260709 13:05:56.502640 13789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:56.502750 13783 heartbeater.cc:507] Master 127.0.107.254:45415 requested a full tablet report, sending...
I20260709 13:05:56.503186   431 server_base.cc:1061] running on GCE node
I20260709 13:05:56.507474   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:05:56.507565 13581 ts_manager.cc:194] Registered new tserver with Master: d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
W20260709 13:05:56.507737   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:56.508584   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602356508555 us; error 0 us; skew 500 ppm
I20260709 13:05:56.510013   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:56.510845 13581 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46038
I20260709 13:05:56.513888   431 webserver.cc:533] Webserver started at http://127.0.107.195:38701/ using document root <none> and password file <none>
I20260709 13:05:56.514725   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:56.514997   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:56.515321   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:56.517004   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-2-root/instance:
uuid: "fef52998d77a4a089f489e74975261b4"
format_stamp: "Formatted at 2026-07-09 13:05:56 on dist-test-slave-l8g5"
I20260709 13:05:56.523741   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:05:56.528635 13794 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:56.530427   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:05:56.530951   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-2-root
uuid: "fef52998d77a4a089f489e74975261b4"
format_stamp: "Formatted at 2026-07-09 13:05:56 on dist-test-slave-l8g5"
I20260709 13:05:56.531313   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNXjA8S/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601924925497-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:56.546926   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:56.549101   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:56.550843   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:56.553222   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:56.553481   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:56.553799   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:56.554009   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:56.622921   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:37789
I20260709 13:05:56.622996 13856 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:37789 every 8 connection(s)
I20260709 13:05:56.646281 13857 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45415
I20260709 13:05:56.646963 13857 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:56.647964 13857 heartbeater.cc:507] Master 127.0.107.254:45415 requested a full tablet report, sending...
I20260709 13:05:56.650933 13581 ts_manager.cc:194] Registered new tserver with Master: fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:56.651892   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.023768295s
I20260709 13:05:56.653224 13581 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46042
I20260709 13:05:56.689432 13581 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46048:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:56.692575 13581 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:56.798177 13747 tablet_service.cc:1511] Processing CreateTablet for tablet 3b06e5df1f9d475e95bf3e17ff21a125 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:56.798576 13748 tablet_service.cc:1511] Processing CreateTablet for tablet d12988af49734f7abf650836e11ddcac (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:56.800606 13747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b06e5df1f9d475e95bf3e17ff21a125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.801316 13748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d12988af49734f7abf650836e11ddcac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.811558 13746 tablet_service.cc:1511] Processing CreateTablet for tablet 8f97f3e20011461d8a3c4301a7e4a39c (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:56.813879 13746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f97f3e20011461d8a3c4301a7e4a39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.823557 13673 tablet_service.cc:1511] Processing CreateTablet for tablet d12988af49734f7abf650836e11ddcac (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:56.825801 13673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d12988af49734f7abf650836e11ddcac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.826107 13670 tablet_service.cc:1511] Processing CreateTablet for tablet 469ecfaf70c64d4492c9ad1d9ecf1960 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:56.828898 13670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 469ecfaf70c64d4492c9ad1d9ecf1960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.830873 13671 tablet_service.cc:1511] Processing CreateTablet for tablet 8f97f3e20011461d8a3c4301a7e4a39c (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:56.832841 13671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f97f3e20011461d8a3c4301a7e4a39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.834759 13672 tablet_service.cc:1511] Processing CreateTablet for tablet 3b06e5df1f9d475e95bf3e17ff21a125 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:56.837916 13672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b06e5df1f9d475e95bf3e17ff21a125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.846863 13875 tablet_bootstrap.cc:492] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:05:56.856382 13875 tablet_bootstrap.cc:654] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:56.864387 13822 tablet_service.cc:1511] Processing CreateTablet for tablet d12988af49734f7abf650836e11ddcac (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:56.866286 13822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d12988af49734f7abf650836e11ddcac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.872308 13821 tablet_service.cc:1511] Processing CreateTablet for tablet 3b06e5df1f9d475e95bf3e17ff21a125 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:56.875016 13820 tablet_service.cc:1511] Processing CreateTablet for tablet 8f97f3e20011461d8a3c4301a7e4a39c (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:56.875684 13821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b06e5df1f9d475e95bf3e17ff21a125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.877733 13820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f97f3e20011461d8a3c4301a7e4a39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.884419 13876 tablet_bootstrap.cc:492] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:05:56.889638 13746 tablet_service.cc:1511] Processing CreateTablet for tablet 469ecfaf70c64d4492c9ad1d9ecf1960 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:56.891669 13746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 469ecfaf70c64d4492c9ad1d9ecf1960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.894536 13672 tablet_service.cc:1511] Processing CreateTablet for tablet 612d2b206b1e4f3f94e3d8443198708a (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:56.899472 13819 tablet_service.cc:1511] Processing CreateTablet for tablet 469ecfaf70c64d4492c9ad1d9ecf1960 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:56.901446 13819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 469ecfaf70c64d4492c9ad1d9ecf1960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.904107 13818 tablet_service.cc:1511] Processing CreateTablet for tablet 612d2b206b1e4f3f94e3d8443198708a (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:56.905876 13818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612d2b206b1e4f3f94e3d8443198708a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.905808 13671 tablet_service.cc:1511] Processing CreateTablet for tablet 6fc0f16f48904e228480fedb878f5d81 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:56.908006 13672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612d2b206b1e4f3f94e3d8443198708a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.910614 13671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fc0f16f48904e228480fedb878f5d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.917896 13748 tablet_service.cc:1511] Processing CreateTablet for tablet 6fc0f16f48904e228480fedb878f5d81 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:56.920027 13748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fc0f16f48904e228480fedb878f5d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.922570 13747 tablet_service.cc:1511] Processing CreateTablet for tablet 612d2b206b1e4f3f94e3d8443198708a (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:56.924978 13747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612d2b206b1e4f3f94e3d8443198708a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.930581 13817 tablet_service.cc:1511] Processing CreateTablet for tablet 6fc0f16f48904e228480fedb878f5d81 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:56.932296 13817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fc0f16f48904e228480fedb878f5d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.938169 13673 tablet_service.cc:1511] Processing CreateTablet for tablet 8f62acdae3954f39998b708ae870b104 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:56.943838 13673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f62acdae3954f39998b708ae870b104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.950335 13876 tablet_bootstrap.cc:654] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:56.955024 13745 tablet_service.cc:1511] Processing CreateTablet for tablet 8f62acdae3954f39998b708ae870b104 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:56.957860 13745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f62acdae3954f39998b708ae870b104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:56.984184 13879 tablet_bootstrap.cc:492] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:05:57.017110 13672 tablet_service.cc:1511] Processing CreateTablet for tablet 27618f7c04aa4fe38d0dba2ad47936ae (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:57.018821 13672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27618f7c04aa4fe38d0dba2ad47936ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:57.019824 13746 tablet_service.cc:1511] Processing CreateTablet for tablet 27618f7c04aa4fe38d0dba2ad47936ae (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:57.021374 13746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27618f7c04aa4fe38d0dba2ad47936ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:57.034560 13818 tablet_service.cc:1511] Processing CreateTablet for tablet 8f62acdae3954f39998b708ae870b104 (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:57.036217 13818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f62acdae3954f39998b708ae870b104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:57.038519 13819 tablet_service.cc:1511] Processing CreateTablet for tablet 27618f7c04aa4fe38d0dba2ad47936ae (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:57.043892 13876 tablet_bootstrap.cc:492] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:05:57.043526 13671 tablet_service.cc:1511] Processing CreateTablet for tablet e1552d38be8d443e913dff6849747abb (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:57.044564 13876 ts_tablet_manager.cc:1403] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.161s	user 0.020s	sys 0.025s
I20260709 13:05:57.045567 13671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1552d38be8d443e913dff6849747abb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:57.048557 13819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27618f7c04aa4fe38d0dba2ad47936ae. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:57.051887 13629 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:57.066815 13875 tablet_bootstrap.cc:492] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:05:57.067624 13875 ts_tablet_manager.cc:1403] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.221s	user 0.027s	sys 0.020s
I20260709 13:05:57.073844 13876 raft_consensus.cc:359] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.074923 13876 raft_consensus.cc:385] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.075315 13747 tablet_service.cc:1511] Processing CreateTablet for tablet e1552d38be8d443e913dff6849747abb (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:57.076821 13747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1552d38be8d443e913dff6849747abb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:57.075265 13876 raft_consensus.cc:740] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.076886 13875 raft_consensus.cc:359] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:57.078576 13875 raft_consensus.cc:385] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.078922 13875 raft_consensus.cc:740] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.080368 13875 consensus_queue.cc:260] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:57.082868 13876 consensus_queue.cc:260] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.087980 13879 tablet_bootstrap.cc:654] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.094914 13876 ts_tablet_manager.cc:1434] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.005s
I20260709 13:05:57.096441 13708 heartbeater.cc:499] Master 127.0.107.254:45415 was elected leader, sending a full tablet report...
I20260709 13:05:57.098788 13818 tablet_service.cc:1511] Processing CreateTablet for tablet e1552d38be8d443e913dff6849747abb (DEFAULT_TABLE table=test-workload [id=6c4f087479894923ab32b06c8ea87cfb]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:57.100953 13818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1552d38be8d443e913dff6849747abb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:57.114301 13876 tablet_bootstrap.cc:492] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:05:57.114850 13783 heartbeater.cc:499] Master 127.0.107.254:45415 was elected leader, sending a full tablet report...
I20260709 13:05:57.123299 13875 ts_tablet_manager.cc:1434] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.055s	user 0.012s	sys 0.004s
I20260709 13:05:57.124778 13875 tablet_bootstrap.cc:492] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:05:57.133659 13879 tablet_bootstrap.cc:492] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:05:57.134814 13879 ts_tablet_manager.cc:1403] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.151s	user 0.025s	sys 0.012s
I20260709 13:05:57.136945 13876 tablet_bootstrap.cc:654] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.138803 13879 raft_consensus.cc:359] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.139986 13879 raft_consensus.cc:385] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.140457 13879 raft_consensus.cc:740] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.141990 13879 consensus_queue.cc:260] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.146360 13875 tablet_bootstrap.cc:654] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.147811 13876 tablet_bootstrap.cc:492] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:05:57.148633 13876 ts_tablet_manager.cc:1403] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.004s
I20260709 13:05:57.153252 13876 raft_consensus.cc:359] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.154665 13876 raft_consensus.cc:385] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.155448 13876 raft_consensus.cc:740] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.157670 13876 consensus_queue.cc:260] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.160063 13876 ts_tablet_manager.cc:1434] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 13:05:57.161226 13876 tablet_bootstrap.cc:492] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:05:57.163347 13879 ts_tablet_manager.cc:1434] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.028s	user 0.011s	sys 0.011s
I20260709 13:05:57.164609 13857 heartbeater.cc:499] Master 127.0.107.254:45415 was elected leader, sending a full tablet report...
I20260709 13:05:57.165194 13879 tablet_bootstrap.cc:492] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:05:57.175354 13879 tablet_bootstrap.cc:654] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.178846 13876 tablet_bootstrap.cc:654] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.181173 13875 tablet_bootstrap.cc:492] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:05:57.182317 13875 ts_tablet_manager.cc:1403] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.058s	user 0.007s	sys 0.017s
I20260709 13:05:57.188671 13875 raft_consensus.cc:359] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.189662 13875 raft_consensus.cc:385] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.190037 13875 raft_consensus.cc:740] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.191015 13879 tablet_bootstrap.cc:492] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:05:57.192109 13879 ts_tablet_manager.cc:1403] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.004s
I20260709 13:05:57.191738 13875 consensus_queue.cc:260] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.197255 13879 raft_consensus.cc:359] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:57.199785 13879 raft_consensus.cc:385] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.199882 13875 ts_tablet_manager.cc:1434] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.005s
I20260709 13:05:57.200326 13879 raft_consensus.cc:740] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.201417 13875 tablet_bootstrap.cc:492] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:05:57.201537 13876 tablet_bootstrap.cc:492] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:05:57.202986 13876 ts_tablet_manager.cc:1403] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.015s
I20260709 13:05:57.201431 13879 consensus_queue.cc:260] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:57.205343 13879 ts_tablet_manager.cc:1434] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:05:57.206983 13879 tablet_bootstrap.cc:492] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:05:57.208045 13876 raft_consensus.cc:359] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:57.209383 13876 raft_consensus.cc:385] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.209808 13876 raft_consensus.cc:740] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.210978 13876 consensus_queue.cc:260] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:57.216892 13876 ts_tablet_manager.cc:1434] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:05:57.218029 13879 tablet_bootstrap.cc:654] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.218026 13875 tablet_bootstrap.cc:654] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.218524 13876 tablet_bootstrap.cc:492] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:05:57.228672 13876 tablet_bootstrap.cc:654] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:57.228715 13784 tablet.cc:2406] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:05:57.234138 13879 tablet_bootstrap.cc:492] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:05:57.234937 13879 ts_tablet_manager.cc:1403] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.003s
I20260709 13:05:57.236661 13875 tablet_bootstrap.cc:492] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:05:57.237334 13875 ts_tablet_manager.cc:1403] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.001s
I20260709 13:05:57.239298 13876 tablet_bootstrap.cc:492] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:05:57.239243 13879 raft_consensus.cc:359] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.240059 13876 ts_tablet_manager.cc:1403] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.008s
I20260709 13:05:57.240203 13879 raft_consensus.cc:385] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.240633 13879 raft_consensus.cc:740] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.241196 13875 raft_consensus.cc:359] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.242174 13875 raft_consensus.cc:385] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.242590 13875 raft_consensus.cc:740] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.241873 13879 consensus_queue.cc:260] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.243577 13875 consensus_queue.cc:260] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.244006 13876 raft_consensus.cc:359] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.244783 13876 raft_consensus.cc:385] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.245080 13876 raft_consensus.cc:740] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.245553 13879 ts_tablet_manager.cc:1434] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:05:57.246086 13876 consensus_queue.cc:260] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.246964 13879 tablet_bootstrap.cc:492] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:05:57.251628 13876 ts_tablet_manager.cc:1434] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:05:57.252915 13876 tablet_bootstrap.cc:492] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:05:57.257359 13879 tablet_bootstrap.cc:654] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.264562 13876 tablet_bootstrap.cc:654] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.259588 13875 ts_tablet_manager.cc:1434] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:05:57.269268 13881 raft_consensus.cc:493] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.270133 13881 raft_consensus.cc:515] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:57.273684 13879 tablet_bootstrap.cc:492] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:05:57.274327 13881 leader_election.cc:290] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.274675 13879 ts_tablet_manager.cc:1403] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.011s
I20260709 13:05:57.268589 13875 tablet_bootstrap.cc:492] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:05:57.275234 13880 raft_consensus.cc:493] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.275817 13880 raft_consensus.cc:515] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.279965 13880 leader_election.cc:290] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.278473 13879 raft_consensus.cc:359] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.290472 13879 raft_consensus.cc:385] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.294155 13879 raft_consensus.cc:740] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.297164 13879 consensus_queue.cc:260] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.298975 13875 tablet_bootstrap.cc:654] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.304886 13876 tablet_bootstrap.cc:492] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:05:57.306375 13879 ts_tablet_manager.cc:1434] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 13:05:57.307817 13879 tablet_bootstrap.cc:492] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:05:57.310812 13880 raft_consensus.cc:493] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.311866 13880 raft_consensus.cc:515] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:57.315203 13880 leader_election.cc:290] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789), 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701)
I20260709 13:05:57.316241 13879 tablet_bootstrap.cc:654] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.316260 13876 ts_tablet_manager.cc:1403] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.064s	user 0.010s	sys 0.011s
I20260709 13:05:57.323752 13876 raft_consensus.cc:359] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.324728 13876 raft_consensus.cc:385] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.325184 13876 raft_consensus.cc:740] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.326134 13876 consensus_queue.cc:260] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.328698 13876 ts_tablet_manager.cc:1434] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:05:57.337069 13876 tablet_bootstrap.cc:492] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:05:57.338280 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97f3e20011461d8a3c4301a7e4a39c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:05:57.338634 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:05:57.339331 13683 raft_consensus.cc:2468] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:05:57.339334 13682 raft_consensus.cc:2468] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:05:57.340155 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:05:57.341459 13758 raft_consensus.cc:2468] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 0.
I20260709 13:05:57.344044 13722 leader_election.cc:304] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:05:57.345425 13880 raft_consensus.cc:2804] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.346302 13880 raft_consensus.cc:493] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.345309 13876 tablet_bootstrap.cc:654] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.346725 13880 raft_consensus.cc:3060] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.350169 13879 tablet_bootstrap.cc:492] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:05:57.350921 13879 ts_tablet_manager.cc:1403] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.043s	user 0.022s	sys 0.008s
I20260709 13:05:57.354216 13722 leader_election.cc:304] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:05:57.355261 13879 raft_consensus.cc:359] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.356480 13648 leader_election.cc:304] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:05:57.356549 13879 raft_consensus.cc:385] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.357451 13879 raft_consensus.cc:740] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.357717 13881 raft_consensus.cc:2804] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.358150 13881 raft_consensus.cc:493] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.358510 13881 raft_consensus.cc:3060] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.358629 13879 consensus_queue.cc:260] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.360378 13901 raft_consensus.cc:2804] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.360872 13901 raft_consensus.cc:493] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.361223 13901 raft_consensus.cc:3060] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.361745 13876 tablet_bootstrap.cc:492] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:05:57.362417 13876 ts_tablet_manager.cc:1403] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.006s
I20260709 13:05:57.362632 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:05:57.363489 13832 raft_consensus.cc:2468] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 0.
I20260709 13:05:57.366360 13876 raft_consensus.cc:359] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.367403 13876 raft_consensus.cc:385] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.367779 13876 raft_consensus.cc:740] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.368743 13876 consensus_queue.cc:260] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.369262 13881 raft_consensus.cc:515] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:57.374469 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:05:57.376322 13876 ts_tablet_manager.cc:1434] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 13:05:57.378545 13876 tablet_bootstrap.cc:492] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:05:57.378830 13881 leader_election.cc:290] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.380141 13881 raft_consensus.cc:493] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.384943 13884 raft_consensus.cc:493] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.387171 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97f3e20011461d8a3c4301a7e4a39c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:05:57.390038 13832 raft_consensus.cc:2468] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:05:57.388228 13831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:05:57.391603 13831 raft_consensus.cc:2468] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:05:57.380697 13881 raft_consensus.cc:515] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.394090 13881 leader_election.cc:290] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.389020 13884 raft_consensus.cc:515] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.403929 13901 raft_consensus.cc:515] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:57.409998 13758 raft_consensus.cc:2393] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 66a96c01be0e4bc2b60dc695623d1dd7 in current term 1: Already voted for candidate d8474050515746e49b6550d9f1cacf9a in this term.
I20260709 13:05:57.410954 13901 leader_election.cc:290] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Requested vote from peers fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789), 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701)
I20260709 13:05:57.410801 13831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:57.411887 13831 raft_consensus.cc:3060] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.413612 13879 ts_tablet_manager.cc:1434] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.004s
I20260709 13:05:57.415714 13879 tablet_bootstrap.cc:492] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:05:57.418109 13884 leader_election.cc:290] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:05:57.420864 13880 raft_consensus.cc:515] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.433631 13875 tablet_bootstrap.cc:492] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:05:57.435314 13875 ts_tablet_manager.cc:1403] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.167s	user 0.027s	sys 0.034s
I20260709 13:05:57.437836 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97f3e20011461d8a3c4301a7e4a39c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:57.438647 13832 raft_consensus.cc:3060] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.439607 13876 tablet_bootstrap.cc:654] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.439708 13831 raft_consensus.cc:2468] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:05:57.441239 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:57.442282 13830 raft_consensus.cc:2393] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 66a96c01be0e4bc2b60dc695623d1dd7 in current term 1: Already voted for candidate d8474050515746e49b6550d9f1cacf9a in this term.
I20260709 13:05:57.443858 13723 leader_election.cc:304] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:57.444273 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:05:57.445278 13682 raft_consensus.cc:2393] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d8474050515746e49b6550d9f1cacf9a in current term 1: Already voted for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in this term.
I20260709 13:05:57.444465 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97f3e20011461d8a3c4301a7e4a39c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:05:57.445827 13901 raft_consensus.cc:2804] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:57.445727 13875 raft_consensus.cc:359] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.446344 13683 raft_consensus.cc:3060] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.446610 13875 raft_consensus.cc:385] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.447041 13875 raft_consensus.cc:740] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.447922 13875 consensus_queue.cc:260] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.448243 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b06e5df1f9d475e95bf3e17ff21a125" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:05:57.453871 13758 raft_consensus.cc:2468] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 0.
I20260709 13:05:57.454317 13879 tablet_bootstrap.cc:654] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.449049 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b06e5df1f9d475e95bf3e17ff21a125" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:05:57.455709 13830 raft_consensus.cc:2468] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 0.
I20260709 13:05:57.461571 13648 leader_election.cc:304] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:05:57.469666 13901 raft_consensus.cc:697] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:05:57.474587 13901 consensus_queue.cc:237] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:57.470474 13880 leader_election.cc:290] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.481467 13903 raft_consensus.cc:2804] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.482306 13903 raft_consensus.cc:493] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.482739 13903 raft_consensus.cc:3060] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.486022 13832 raft_consensus.cc:2468] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:05:57.488204 13723 leader_election.cc:304] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:57.489006 13902 raft_consensus.cc:493] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.489746 13902 raft_consensus.cc:515] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.492643 13902 leader_election.cc:290] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.494590 13908 raft_consensus.cc:493] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.494212 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:05:57.495187 13908 raft_consensus.cc:515] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.495477 13682 raft_consensus.cc:2468] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 0.
I20260709 13:05:57.496014 13875 ts_tablet_manager.cc:1434] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.059s	user 0.015s	sys 0.007s
I20260709 13:05:57.472344 13683 raft_consensus.cc:2468] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:05:57.498368 13901 raft_consensus.cc:2804] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:57.497979 13796 leader_election.cc:304] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:57.503724 13901 raft_consensus.cc:697] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:05:57.504343 13884 raft_consensus.cc:2804] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.502199 13908 leader_election.cc:290] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.504997 13884 raft_consensus.cc:493] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.504607 13879 tablet_bootstrap.cc:492] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:05:57.504767 13901 consensus_queue.cc:237] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.505731 13884 raft_consensus.cc:3060] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.506074 13879 ts_tablet_manager.cc:1403] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.091s	user 0.017s	sys 0.014s
I20260709 13:05:57.510651 13879 raft_consensus.cc:359] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.511709 13879 raft_consensus.cc:385] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.512302 13879 raft_consensus.cc:740] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.514163 13879 consensus_queue.cc:260] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.518075 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:05:57.521011 13879 ts_tablet_manager.cc:1434] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.002s
I20260709 13:05:57.522295 13879 tablet_bootstrap.cc:492] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:05:57.526755 13875 tablet_bootstrap.cc:492] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
W20260709 13:05:57.521566 13648 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:39199, user_credentials={real_user=slave}} blocked reactor thread for 60194us
I20260709 13:05:57.533979 13648 leader_election.cc:304] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7; no voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4
I20260709 13:05:57.535488 13875 tablet_bootstrap.cc:654] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.542213 13879 tablet_bootstrap.cc:654] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.536767 13580 catalog_manager.cc:5697] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.543421 13902 raft_consensus.cc:2749] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:57.544385 13580 catalog_manager.cc:5697] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.549751 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:05:57.550876 13758 raft_consensus.cc:2468] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 0.
I20260709 13:05:57.552446 13875 tablet_bootstrap.cc:492] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:05:57.555404 13884 raft_consensus.cc:515] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.553158 13875 ts_tablet_manager.cc:1403] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.027s	user 0.006s	sys 0.009s
I20260709 13:05:57.563726 13832 raft_consensus.cc:2393] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8474050515746e49b6550d9f1cacf9a in current term 1: Already voted for candidate fef52998d77a4a089f489e74975261b4 in this term.
I20260709 13:05:57.562737 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:05:57.565724 13876 tablet_bootstrap.cc:492] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:05:57.566211 13683 raft_consensus.cc:2468] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:05:57.566892 13876 ts_tablet_manager.cc:1403] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.188s	user 0.053s	sys 0.071s
I20260709 13:05:57.567373 13875 raft_consensus.cc:359] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.568212 13875 raft_consensus.cc:385] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.568545 13875 raft_consensus.cc:740] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.556365 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:05:57.562644 13648 leader_election.cc:304] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:05:57.570218 13758 raft_consensus.cc:2468] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 0.
I20260709 13:05:57.558255 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:05:57.571265 13902 raft_consensus.cc:2804] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.571092 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:05:57.571780 13902 raft_consensus.cc:493] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.571960 13830 raft_consensus.cc:2393] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 66a96c01be0e4bc2b60dc695623d1dd7 in current term 1: Already voted for candidate fef52998d77a4a089f489e74975261b4 in this term.
I20260709 13:05:57.559863 13903 raft_consensus.cc:515] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.572209 13902 raft_consensus.cc:3060] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.572064 13876 raft_consensus.cc:359] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.573207 13876 raft_consensus.cc:385] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.573993 13876 raft_consensus.cc:740] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.573225 13884 leader_election.cc:290] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:05:57.575063 13903 leader_election.cc:290] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.575448 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:05:57.574826 13876 consensus_queue.cc:260] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.576169 13758 raft_consensus.cc:3060] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.575795 13875 consensus_queue.cc:260] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.578094 13876 ts_tablet_manager.cc:1434] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:05:57.579119 13875 ts_tablet_manager.cc:1434] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:05:57.579548 13876 tablet_bootstrap.cc:492] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:05:57.580439 13875 tablet_bootstrap.cc:492] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:05:57.586383 13722 leader_election.cc:304] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: fef52998d77a4a089f489e74975261b4
I20260709 13:05:57.586851 13879 tablet_bootstrap.cc:492] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:05:57.587857 13879 ts_tablet_manager.cc:1403] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.066s	user 0.024s	sys 0.000s
I20260709 13:05:57.589346 13876 tablet_bootstrap.cc:654] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.589566 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b06e5df1f9d475e95bf3e17ff21a125" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:05:57.590545 13757 raft_consensus.cc:3060] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.592167 13879 raft_consensus.cc:359] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.593053 13879 raft_consensus.cc:385] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.593452 13879 raft_consensus.cc:740] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.594735 13879 consensus_queue.cc:260] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.598259 13879 ts_tablet_manager.cc:1434] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.006s
I20260709 13:05:57.599707 13879 tablet_bootstrap.cc:492] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:05:57.600914 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b06e5df1f9d475e95bf3e17ff21a125" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:57.602190 13830 raft_consensus.cc:3060] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.608421 13875 tablet_bootstrap.cc:654] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.616125 13757 raft_consensus.cc:2468] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 1.
I20260709 13:05:57.618363 13902 raft_consensus.cc:515] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.623401 13876 tablet_bootstrap.cc:492] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:05:57.623574 13902 leader_election.cc:290] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.624231 13876 ts_tablet_manager.cc:1403] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.045s	user 0.028s	sys 0.012s
I20260709 13:05:57.618417 13830 raft_consensus.cc:2468] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 1.
I20260709 13:05:57.629056 13875 tablet_bootstrap.cc:492] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:05:57.629328 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:57.629861 13875 ts_tablet_manager.cc:1403] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.050s	user 0.009s	sys 0.013s
I20260709 13:05:57.630213 13830 raft_consensus.cc:2393] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 66a96c01be0e4bc2b60dc695623d1dd7 in current term 1: Already voted for candidate fef52998d77a4a089f489e74975261b4 in this term.
I20260709 13:05:57.631515 13648 leader_election.cc:304] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:05:57.620911 13758 raft_consensus.cc:2468] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 1.
I20260709 13:05:57.624704 13683 raft_consensus.cc:2393] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fef52998d77a4a089f489e74975261b4 in current term 1: Already voted for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in this term.
I20260709 13:05:57.632754 13902 raft_consensus.cc:2804] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:57.633651 13902 raft_consensus.cc:697] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 LEADER]: Becoming Leader. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Running, Role: LEADER
I20260709 13:05:57.634662 13902 consensus_queue.cc:237] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.636976 13910 raft_consensus.cc:2764] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:05:57.637996 13797 leader_election.cc:304] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:57.628154 13876 raft_consensus.cc:359] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.620644 13879 tablet_bootstrap.cc:654] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.653043 13876 raft_consensus.cc:385] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.653503 13876 raft_consensus.cc:740] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.654515 13876 consensus_queue.cc:260] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.638492 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:05:57.656350 13757 raft_consensus.cc:2393] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 66a96c01be0e4bc2b60dc695623d1dd7 in current term 1: Already voted for candidate fef52998d77a4a089f489e74975261b4 in this term.
I20260709 13:05:57.652627 13875 raft_consensus.cc:359] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.657908 13884 raft_consensus.cc:2804] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:57.658066 13875 raft_consensus.cc:385] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.658339 13648 leader_election.cc:304] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7; no voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4
I20260709 13:05:57.658648 13875 raft_consensus.cc:740] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.658794 13884 raft_consensus.cc:697] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:05:57.659313 13903 raft_consensus.cc:2749] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:57.659823 13875 consensus_queue.cc:260] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.661842 13876 ts_tablet_manager.cc:1434] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.007s
I20260709 13:05:57.659914 13884 consensus_queue.cc:237] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.668213 13876 tablet_bootstrap.cc:492] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:05:57.676290 13875 ts_tablet_manager.cc:1434] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.046s	user 0.010s	sys 0.000s
I20260709 13:05:57.677894 13875 tablet_bootstrap.cc:492] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:05:57.679965 13876 tablet_bootstrap.cc:654] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.679458 13581 catalog_manager.cc:5697] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.686842 13875 tablet_bootstrap.cc:654] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.692448 13884 raft_consensus.cc:493] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.695413 13884 raft_consensus.cc:515] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.703449 13879 tablet_bootstrap.cc:492] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:05:57.704466 13879 ts_tablet_manager.cc:1403] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.105s	user 0.010s	sys 0.010s
I20260709 13:05:57.704772 13876 tablet_bootstrap.cc:492] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:05:57.706226 13876 ts_tablet_manager.cc:1403] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.003s
I20260709 13:05:57.707343 13879 raft_consensus.cc:359] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.708029 13879 raft_consensus.cc:385] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.708256 13879 raft_consensus.cc:740] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.709744 13902 raft_consensus.cc:493] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.709069 13879 consensus_queue.cc:260] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.711292 13902 raft_consensus.cc:515] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.713003 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612d2b206b1e4f3f94e3d8443198708a" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:05:57.713291 13884 leader_election.cc:290] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:05:57.713842 13683 raft_consensus.cc:2468] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 0.
I20260709 13:05:57.714324 13876 raft_consensus.cc:359] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.715215 13876 raft_consensus.cc:385] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.715610 13876 raft_consensus.cc:740] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.715679 13879 ts_tablet_manager.cc:1434] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:05:57.716590 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612d2b206b1e4f3f94e3d8443198708a" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:05:57.716452 13876 consensus_queue.cc:260] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.714336 13902 leader_election.cc:290] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.716912 13879 tablet_bootstrap.cc:492] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
W20260709 13:05:57.718541 13648 leader_election.cc:343] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:57.718044 13796 leader_election.cc:304] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:57.717588 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612d2b206b1e4f3f94e3d8443198708a" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:05:57.730796 13876 ts_tablet_manager.cc:1434] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.002s
I20260709 13:05:57.716902 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612d2b206b1e4f3f94e3d8443198708a" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:05:57.733842 13913 raft_consensus.cc:2804] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.734376 13913 raft_consensus.cc:493] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.734745 13913 raft_consensus.cc:3060] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:57.732792 13797 leader_election.cc:343] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:57.738624 13910 raft_consensus.cc:493] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.741402 13879 tablet_bootstrap.cc:654] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.739240 13910 raft_consensus.cc:515] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.746701 13910 leader_election.cc:290] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.747686 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc0f16f48904e228480fedb878f5d81" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:05:57.748772 13832 raft_consensus.cc:2468] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:05:57.752295 13723 leader_election.cc:304] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:57.758535 13579 catalog_manager.cc:5697] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.761958 13913 raft_consensus.cc:515] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.763892 13830 raft_consensus.cc:2393] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 66a96c01be0e4bc2b60dc695623d1dd7 in current term 1: Already voted for candidate fef52998d77a4a089f489e74975261b4 in this term.
I20260709 13:05:57.766407 13913 leader_election.cc:290] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:05:57.770282 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612d2b206b1e4f3f94e3d8443198708a" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:05:57.771255 13683 raft_consensus.cc:3060] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.773983 13903 raft_consensus.cc:493] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.774778 13648 leader_election.cc:304] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7; no voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4
I20260709 13:05:57.774865 13903 raft_consensus.cc:515] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.776683 13910 raft_consensus.cc:2804] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.777177 13910 raft_consensus.cc:493] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.777545 13910 raft_consensus.cc:3060] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.780236 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc0f16f48904e228480fedb878f5d81" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:05:57.784214 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc0f16f48904e228480fedb878f5d81" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:05:57.785907 13830 raft_consensus.cc:2468] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 0.
I20260709 13:05:57.787866 13648 leader_election.cc:304] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:57.788668 13910 raft_consensus.cc:515] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.789186 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc0f16f48904e228480fedb878f5d81" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:05:57.789310 13881 raft_consensus.cc:2804] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.790198 13881 raft_consensus.cc:493] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.791703 13881 raft_consensus.cc:3060] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.791812 13758 raft_consensus.cc:2393] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 66a96c01be0e4bc2b60dc695623d1dd7 in current term 1: Already voted for candidate d8474050515746e49b6550d9f1cacf9a in this term.
I20260709 13:05:57.794212 13875 tablet_bootstrap.cc:492] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:05:57.795179 13875 ts_tablet_manager.cc:1403] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.118s	user 0.034s	sys 0.017s
I20260709 13:05:57.784801 13903 leader_election.cc:290] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.799269 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612d2b206b1e4f3f94e3d8443198708a" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:05:57.799168 13875 raft_consensus.cc:359] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.804150 13875 raft_consensus.cc:385] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.804808 13875 raft_consensus.cc:740] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.806998 13681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc0f16f48904e228480fedb878f5d81" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:05:57.807444 13910 leader_election.cc:290] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.806281 13875 consensus_queue.cc:260] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.809845 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc0f16f48904e228480fedb878f5d81" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:57.810618 13830 raft_consensus.cc:3060] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.812258 13903 raft_consensus.cc:493] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.812592 13875 ts_tablet_manager.cc:1434] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:05:57.813167 13903 raft_consensus.cc:515] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.814071 13875 tablet_bootstrap.cc:492] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:05:57.816185 13903 leader_election.cc:290] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.817165 13757 raft_consensus.cc:3060] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.818130 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1552d38be8d443e913dff6849747abb" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:05:57.818156 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1552d38be8d443e913dff6849747abb" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
W20260709 13:05:57.820214 13648 leader_election.cc:343] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:57.821110 13648 leader_election.cc:343] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:57.821609 13648 leader_election.cc:304] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7; no voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4
I20260709 13:05:57.822980 13875 tablet_bootstrap.cc:654] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.830288 13903 raft_consensus.cc:2749] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:57.832995 13903 raft_consensus.cc:493] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.834815 13903 raft_consensus.cc:515] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.838223 13879 tablet_bootstrap.cc:492] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:05:57.839025 13879 ts_tablet_manager.cc:1403] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.122s	user 0.049s	sys 0.056s
I20260709 13:05:57.844878 13903 leader_election.cc:290] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.849293 13830 raft_consensus.cc:2468] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:05:57.849013 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:05:57.851018 13832 raft_consensus.cc:2468] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 0.
I20260709 13:05:57.848969 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:05:57.854015 13723 leader_election.cc:304] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:05:57.854665 13881 raft_consensus.cc:515] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.855281 13910 raft_consensus.cc:2804] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:57.855916 13910 raft_consensus.cc:697] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:05:57.856232 13682 raft_consensus.cc:2393] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8474050515746e49b6550d9f1cacf9a in current term 1: Already voted for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in this term.
I20260709 13:05:57.856369 13648 leader_election.cc:304] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:57.856925 13910 consensus_queue.cc:237] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.857702 13881 leader_election.cc:290] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
W20260709 13:05:57.858141 13648 leader_election.cc:343] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:57.858412 13881 raft_consensus.cc:2804] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.858870 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc0f16f48904e228480fedb878f5d81" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:05:57.858850 13881 raft_consensus.cc:493] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.864792 13881 raft_consensus.cc:3060] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.865818 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc0f16f48904e228480fedb878f5d81" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:57.867043 13832 raft_consensus.cc:2393] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 66a96c01be0e4bc2b60dc695623d1dd7 in current term 1: Already voted for candidate d8474050515746e49b6550d9f1cacf9a in this term.
I20260709 13:05:57.867033 13757 raft_consensus.cc:2468] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 1.
I20260709 13:05:57.869328 13875 tablet_bootstrap.cc:492] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:05:57.870730 13875 ts_tablet_manager.cc:1403] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.057s	user 0.016s	sys 0.009s
I20260709 13:05:57.872455 13879 raft_consensus.cc:359] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.873392 13879 raft_consensus.cc:385] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.873713 13879 raft_consensus.cc:740] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.875943 13879 consensus_queue.cc:260] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.882249 13648 leader_election.cc:304] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7; no voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4
I20260709 13:05:57.883323 13881 raft_consensus.cc:515] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.885608 13879 ts_tablet_manager.cc:1434] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.046s	user 0.013s	sys 0.000s
I20260709 13:05:57.888495 13913 raft_consensus.cc:493] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.889236 13903 raft_consensus.cc:2749] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:57.889254 13913 raft_consensus.cc:515] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.883814 13683 raft_consensus.cc:2468] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 1.
I20260709 13:05:57.891106 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:05:57.893106 13797 leader_election.cc:304] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:57.893425 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1552d38be8d443e913dff6849747abb" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:05:57.890867 13875 raft_consensus.cc:359] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.889240 13881 leader_election.cc:290] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.892056 13913 leader_election.cc:290] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:05:57.894129 13902 raft_consensus.cc:2749] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:57.899672 13875 raft_consensus.cc:385] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:57.894243 13682 raft_consensus.cc:2468] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 0.
I20260709 13:05:57.902217 13875 raft_consensus.cc:740] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:05:57.894129 13884 raft_consensus.cc:2804] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:57.904491 13875 consensus_queue.cc:260] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.904547 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:57.906796 13832 raft_consensus.cc:3060] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.904510 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1552d38be8d443e913dff6849747abb" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:05:57.911069 13758 raft_consensus.cc:3060] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.911720 13875 ts_tablet_manager.cc:1434] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.004s
W20260709 13:05:57.911702 13797 leader_election.cc:343] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:57.911358 13579 catalog_manager.cc:5697] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.912921 13884 raft_consensus.cc:697] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:05:57.913341 13875 tablet_bootstrap.cc:492] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:05:57.914973 13884 consensus_queue.cc:237] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.922853 13758 raft_consensus.cc:2468] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 1.
I20260709 13:05:57.925702 13648 leader_election.cc:304] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:05:57.927951 13832 raft_consensus.cc:2468] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 1.
I20260709 13:05:57.927317 13796 leader_election.cc:304] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: d8474050515746e49b6550d9f1cacf9a
I20260709 13:05:57.935015 13902 raft_consensus.cc:2804] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:57.935612 13902 raft_consensus.cc:697] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 LEADER]: Becoming Leader. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Running, Role: LEADER
I20260709 13:05:57.932708 13875 tablet_bootstrap.cc:654] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:57.933547 13910 raft_consensus.cc:493] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.936870 13902 consensus_queue.cc:237] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.937170 13910 raft_consensus.cc:515] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.941565 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f62acdae3954f39998b708ae870b104" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:05:57.942143 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f62acdae3954f39998b708ae870b104" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:05:57.942387 13832 raft_consensus.cc:2468] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:05:57.942902 13682 raft_consensus.cc:2468] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:05:57.941082 13578 catalog_manager.cc:5697] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.943679 13723 leader_election.cc:304] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:57.944680 13908 raft_consensus.cc:2804] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.945010 13908 raft_consensus.cc:493] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.945538 13908 raft_consensus.cc:3060] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.946113 13910 leader_election.cc:290] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:57.947803 13884 raft_consensus.cc:2804] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:57.956718 13910 consensus_queue.cc:1048] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:57.951373 13884 raft_consensus.cc:493] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.961222 13884 raft_consensus.cc:3060] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:57.977124 13581 catalog_manager.cc:5697] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.981294 13910 consensus_queue.cc:1048] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:57.964660 13908 raft_consensus.cc:515] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:57.994854 13884 raft_consensus.cc:515] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.005723 13875 tablet_bootstrap.cc:492] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:05:58.006413 13875 ts_tablet_manager.cc:1403] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.093s	user 0.023s	sys 0.018s
I20260709 13:05:58.010542 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1552d38be8d443e913dff6849747abb" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:05:58.010304 13875 raft_consensus.cc:359] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.011355 13683 raft_consensus.cc:3060] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:58.011679 13884 leader_election.cc:290] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:05:58.015434 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1552d38be8d443e913dff6849747abb" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:05:58.018229 13910 consensus_queue.cc:1048] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:58.019784 13908 leader_election.cc:290] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:58.020212 13681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f62acdae3954f39998b708ae870b104" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:05:58.021039 13681 raft_consensus.cc:3060] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:58.021528 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f62acdae3954f39998b708ae870b104" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:58.022614 13830 raft_consensus.cc:3060] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:58.027765 13683 raft_consensus.cc:2468] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 1.
I20260709 13:05:58.029565 13796 leader_election.cc:304] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:58.031123 13884 raft_consensus.cc:2804] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:58.031682 13884 raft_consensus.cc:697] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:05:58.032630 13884 consensus_queue.cc:237] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.011345 13875 raft_consensus.cc:385] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:58.036082 13875 raft_consensus.cc:740] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:05:58.037416 13875 consensus_queue.cc:260] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.038430 13758 raft_consensus.cc:3060] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:58.039954 13681 raft_consensus.cc:2468] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:05:58.042711 13722 leader_election.cc:304] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:05:58.044282 13908 raft_consensus.cc:2804] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:58.045202 13908 raft_consensus.cc:697] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:05:58.046270 13908 consensus_queue.cc:237] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.063664 13758 raft_consensus.cc:2468] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 1.
I20260709 13:05:58.070565 13875 ts_tablet_manager.cc:1434] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.064s	user 0.012s	sys 0.000s
I20260709 13:05:58.056396 13830 raft_consensus.cc:2468] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:05:58.102702 13884 consensus_queue.cc:1048] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:58.101838 13581 catalog_manager.cc:5697] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:58.109146 13901 consensus_queue.cc:1048] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:58.131788 13579 catalog_manager.cc:5697] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:58.155160 13884 consensus_queue.cc:1048] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:58.137393 13629 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:58.227231 13902 consensus_queue.cc:1048] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:58.239459 13902 consensus_queue.cc:1048] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:58.279147 13758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d12988af49734f7abf650836e11ddcac"
dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
mode: GRACEFUL
new_leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:55618
I20260709 13:05:58.279804 13758 raft_consensus.cc:606] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Received request to transfer leadership to TS 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:05:58.290104 13758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f97f3e20011461d8a3c4301a7e4a39c"
dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
mode: GRACEFUL
new_leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:55618
I20260709 13:05:58.290891 13758 raft_consensus.cc:606] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Received request to transfer leadership to TS 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:05:58.302647 13948 consensus_queue.cc:1048] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:58.303776 13910 consensus_queue.cc:1048] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:58.318534 13830 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960"
dest_uuid: "fef52998d77a4a089f489e74975261b4"
mode: GRACEFUL
new_leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:33410
I20260709 13:05:58.319311 13830 raft_consensus.cc:606] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Received request to transfer leadership to TS 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:05:58.319155 13913 consensus_queue.cc:1048] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:58.334120 13880 consensus_queue.cc:1048] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:58.338304 13832 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "612d2b206b1e4f3f94e3d8443198708a"
dest_uuid: "fef52998d77a4a089f489e74975261b4"
mode: GRACEFUL
new_leader_uuid: "d8474050515746e49b6550d9f1cacf9a"
 from {username='slave'} at 127.0.0.1:33410
I20260709 13:05:58.339056 13832 raft_consensus.cc:606] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Received request to transfer leadership to TS d8474050515746e49b6550d9f1cacf9a
I20260709 13:05:58.374711 13680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae"
dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
mode: GRACEFUL
new_leader_uuid: "d8474050515746e49b6550d9f1cacf9a"
 from {username='slave'} at 127.0.0.1:54300
I20260709 13:05:58.375466 13680 raft_consensus.cc:606] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 LEADER]: Received request to transfer leadership to TS d8474050515746e49b6550d9f1cacf9a
I20260709 13:05:58.399731 13928 consensus_queue.cc:1048] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:58.414565 13902 consensus_queue.cc:1048] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:58.425993 13928 raft_consensus.cc:993] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7: : Instructing follower d8474050515746e49b6550d9f1cacf9a to start an election
I20260709 13:05:58.426735 13902 raft_consensus.cc:1081] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 LEADER]: Signalling peer d8474050515746e49b6550d9f1cacf9a to start an election
I20260709 13:05:58.429138 13756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae"
dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
 from {username='slave'} at 127.0.0.1:55600
I20260709 13:05:58.430001 13756 raft_consensus.cc:493] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.430438 13756 raft_consensus.cc:3060] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.437444 13756 raft_consensus.cc:515] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.439775 13756 leader_election.cc:290] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 2 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:58.441075 13680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:05:58.441557 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:58.442067 13680 raft_consensus.cc:3055] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:58.442476 13680 raft_consensus.cc:740] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Running, Role: LEADER
I20260709 13:05:58.442659 13832 raft_consensus.cc:3060] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.443449 13680 consensus_queue.cc:260] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.445026 13680 raft_consensus.cc:3060] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.449067 13913 consensus_queue.cc:1048] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:58.453439 13680 raft_consensus.cc:2468] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 2.
I20260709 13:05:58.453824 13906 raft_consensus.cc:993] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a: : Instructing follower 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:05:58.454388 13832 raft_consensus.cc:2468] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 2.
I20260709 13:05:58.454660 13906 raft_consensus.cc:1081] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Signalling peer 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:05:58.455994 13723 leader_election.cc:304] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:58.457676 13680 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d12988af49734f7abf650836e11ddcac"
dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:54280
I20260709 13:05:58.457736 13880 raft_consensus.cc:2804] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:58.458484 13680 raft_consensus.cc:493] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.458525 13880 raft_consensus.cc:697] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 2 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:05:58.459002 13680 raft_consensus.cc:3060] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.459630 13880 consensus_queue.cc:237] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.468742 13884 consensus_queue.cc:1048] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:58.468946 13906 consensus_queue.cc:1048] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:58.476943 13680 raft_consensus.cc:515] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:58.479432 13906 consensus_queue.cc:1048] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:58.480463 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:05:58.481905 13756 raft_consensus.cc:3055] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:58.483222 13756 raft_consensus.cc:740] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:05:58.486106 13680 leader_election.cc:290] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:58.486320 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:58.486965 13832 raft_consensus.cc:3060] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.485858 13756 consensus_queue.cc:260] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:58.497881 13756 raft_consensus.cc:3060] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.500314 13832 raft_consensus.cc:2468] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:05:58.502147 13648 leader_election.cc:304] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:58.503201 13928 raft_consensus.cc:2804] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:58.503795 13928 raft_consensus.cc:697] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Becoming Leader. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Running, Role: LEADER
I20260709 13:05:58.504704 13928 consensus_queue.cc:237] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:05:58.507046 13580 catalog_manager.cc:5697] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 1 to 2, leader changed from 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193) to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 2 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:58.513582 13756 raft_consensus.cc:2468] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:05:58.533654 13578 catalog_manager.cc:5697] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 reported cstate change: term changed from 1 to 2, leader changed from d8474050515746e49b6550d9f1cacf9a (127.0.107.194) to 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:58.537897 13906 raft_consensus.cc:993] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a: : Instructing follower 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:05:58.538650 13908 raft_consensus.cc:1081] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Signalling peer 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:05:58.540493 13683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f97f3e20011461d8a3c4301a7e4a39c"
dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:54280
I20260709 13:05:58.541263 13683 raft_consensus.cc:493] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.541689 13683 raft_consensus.cc:3060] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.548264 13683 raft_consensus.cc:515] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.550364 13683 leader_election.cc:290] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:58.551568 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97f3e20011461d8a3c4301a7e4a39c" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:58.551544 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97f3e20011461d8a3c4301a7e4a39c" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:05:58.552778 13830 raft_consensus.cc:3060] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.552837 13756 raft_consensus.cc:3055] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:58.553580 13756 raft_consensus.cc:740] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:05:58.554648 13756 consensus_queue.cc:260] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.556257 13756 raft_consensus.cc:3060] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.563226 13830 raft_consensus.cc:2468] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:05:58.565313 13648 leader_election.cc:304] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:05:58.566982 13902 raft_consensus.cc:2804] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:58.567332 13756 raft_consensus.cc:2468] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:05:58.567751 13902 raft_consensus.cc:697] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Becoming Leader. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Running, Role: LEADER
I20260709 13:05:58.568841 13902 consensus_queue.cc:237] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.579335 13578 catalog_manager.cc:5697] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 reported cstate change: term changed from 1 to 2, leader changed from d8474050515746e49b6550d9f1cacf9a (127.0.107.194) to 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:58.748528 13884 raft_consensus.cc:993] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4: : Instructing follower d8474050515746e49b6550d9f1cacf9a to start an election
I20260709 13:05:58.749125 13948 raft_consensus.cc:1081] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Signalling peer d8474050515746e49b6550d9f1cacf9a to start an election
I20260709 13:05:58.751142 13756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "612d2b206b1e4f3f94e3d8443198708a"
dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
 from {username='slave'} at 127.0.0.1:55612
I20260709 13:05:58.751732 13756 raft_consensus.cc:493] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.752089 13756 raft_consensus.cc:3060] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.758486 13756 raft_consensus.cc:515] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.760406 13756 leader_election.cc:290] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 2 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:58.761590 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612d2b206b1e4f3f94e3d8443198708a" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:05:58.761790 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612d2b206b1e4f3f94e3d8443198708a" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:58.762341 13683 raft_consensus.cc:3060] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.762435 13830 raft_consensus.cc:3055] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:58.762909 13830 raft_consensus.cc:740] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:05:58.763885 13830 consensus_queue.cc:260] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.765592 13830 raft_consensus.cc:3060] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.769719 13683 raft_consensus.cc:2468] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 2.
I20260709 13:05:58.771222 13722 leader_election.cc:304] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:05:58.772073 13908 raft_consensus.cc:2804] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:58.772653 13908 raft_consensus.cc:697] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 2 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:05:58.772943 13830 raft_consensus.cc:2468] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 2.
I20260709 13:05:58.773607 13908 consensus_queue.cc:237] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.783299 13884 raft_consensus.cc:993] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4: : Instructing follower 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:05:58.782802 13578 catalog_manager.cc:5697] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 1 to 2, leader changed from fef52998d77a4a089f489e74975261b4 (127.0.107.195) to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 2 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:58.783972 13948 raft_consensus.cc:1081] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Signalling peer 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:05:58.785969 13683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960"
dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:54288
I20260709 13:05:58.786836 13683 raft_consensus.cc:493] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.787256 13683 raft_consensus.cc:3060] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.794912 13683 raft_consensus.cc:515] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.796972 13683 leader_election.cc:290] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:05:58.797915 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:05:58.798535 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:05:58.798795 13756 raft_consensus.cc:3060] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.799235 13830 raft_consensus.cc:3055] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:58.799507 13830 raft_consensus.cc:740] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:05:58.800405 13830 consensus_queue.cc:260] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.801754 13830 raft_consensus.cc:3060] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.806224 13756 raft_consensus.cc:2468] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:05:58.807722 13648 leader_election.cc:304] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:05:58.808537 13830 raft_consensus.cc:2468] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:05:58.808646 13902 raft_consensus.cc:2804] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:58.809305 13902 raft_consensus.cc:697] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Becoming Leader. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Running, Role: LEADER
I20260709 13:05:58.810595 13902 consensus_queue.cc:237] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:05:58.821089 13579 catalog_manager.cc:5697] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 reported cstate change: term changed from 1 to 2, leader changed from fef52998d77a4a089f489e74975261b4 (127.0.107.195) to 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:58.960518 13756 raft_consensus.cc:1275] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.963114 13902 consensus_queue.cc:1048] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:58.979095 13830 raft_consensus.cc:1275] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.981796 13902 consensus_queue.cc:1048] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:58.995179 13830 raft_consensus.cc:1275] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.996973 13881 consensus_queue.cc:1048] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:59.013281 13756 raft_consensus.cc:1275] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.015247 13881 consensus_queue.cc:1048] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:59.028985 13683 raft_consensus.cc:1275] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Refusing update from remote peer d8474050515746e49b6550d9f1cacf9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.031270 13908 consensus_queue.cc:1048] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:59.046886 13830 raft_consensus.cc:1275] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Refusing update from remote peer d8474050515746e49b6550d9f1cacf9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.048993 13908 consensus_queue.cc:1048] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:59.202019 13830 raft_consensus.cc:1275] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.203835 13928 consensus_queue.cc:1048] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:59.222543 13756 raft_consensus.cc:1275] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.224980 13928 consensus_queue.cc:1048] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:59.277258 13830 raft_consensus.cc:1275] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Refusing update from remote peer d8474050515746e49b6550d9f1cacf9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.279269 13908 consensus_queue.cc:1048] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.299913 13683 raft_consensus.cc:1275] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Refusing update from remote peer d8474050515746e49b6550d9f1cacf9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.301826 13908 consensus_queue.cc:1048] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:01.421181 13578 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44172:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:01.425400 13578 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:01.494962 13671 tablet_service.cc:1511] Processing CreateTablet for tablet a561890d97b64f298023166984ccf29b (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:01.498566 13671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a561890d97b64f298023166984ccf29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.499095 13747 tablet_service.cc:1511] Processing CreateTablet for tablet a561890d97b64f298023166984ccf29b (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:01.500092 13818 tablet_service.cc:1511] Processing CreateTablet for tablet a561890d97b64f298023166984ccf29b (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:01.501569 13747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a561890d97b64f298023166984ccf29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.502136 13818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a561890d97b64f298023166984ccf29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.507135 13746 tablet_service.cc:1511] Processing CreateTablet for tablet fc9825e0aed94112bde8ae9a46e3a01a (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:01.510813 13819 tablet_service.cc:1511] Processing CreateTablet for tablet fc9825e0aed94112bde8ae9a46e3a01a (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:01.511098 13673 tablet_service.cc:1511] Processing CreateTablet for tablet fc9825e0aed94112bde8ae9a46e3a01a (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:01.513540 13819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc9825e0aed94112bde8ae9a46e3a01a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.509220 13746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc9825e0aed94112bde8ae9a46e3a01a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.522181 13673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc9825e0aed94112bde8ae9a46e3a01a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.527231 13672 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed4ba4005fe473f823843b6027e2d53 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:01.529285 13672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed4ba4005fe473f823843b6027e2d53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.529417 13748 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed4ba4005fe473f823843b6027e2d53 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:01.531116 13748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed4ba4005fe473f823843b6027e2d53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.539791 13983 tablet_bootstrap.cc:492] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:06:01.542518 13817 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed4ba4005fe473f823843b6027e2d53 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:01.544317 13817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed4ba4005fe473f823843b6027e2d53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.556164 13670 tablet_service.cc:1511] Processing CreateTablet for tablet 160b2f7e43224902824f79a186ead89c (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:01.559746 13670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 160b2f7e43224902824f79a186ead89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.568521 13983 tablet_bootstrap.cc:654] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.569175 13745 tablet_service.cc:1511] Processing CreateTablet for tablet 160b2f7e43224902824f79a186ead89c (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:01.571846 13745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 160b2f7e43224902824f79a186ead89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.595674 13817 tablet_service.cc:1511] Processing CreateTablet for tablet 160b2f7e43224902824f79a186ead89c (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:01.603314 13669 tablet_service.cc:1511] Processing CreateTablet for tablet f5ebac8961d041c0bf43bd08a733309b (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:01.605822 13817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 160b2f7e43224902824f79a186ead89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.613754 13669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5ebac8961d041c0bf43bd08a733309b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.614874 13984 tablet_bootstrap.cc:492] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:06:01.619060 13747 tablet_service.cc:1511] Processing CreateTablet for tablet f5ebac8961d041c0bf43bd08a733309b (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:01.620987 13747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5ebac8961d041c0bf43bd08a733309b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.634198 13818 tablet_service.cc:1511] Processing CreateTablet for tablet f5ebac8961d041c0bf43bd08a733309b (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:01.636093 13818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5ebac8961d041c0bf43bd08a733309b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.644845 13668 tablet_service.cc:1511] Processing CreateTablet for tablet 19314b1a099b43b6a02af8fd9ab48a94 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:01.646629 13668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19314b1a099b43b6a02af8fd9ab48a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.649361 13984 tablet_bootstrap.cc:654] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.657008 13986 tablet_bootstrap.cc:492] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:06:01.657399 13744 tablet_service.cc:1511] Processing CreateTablet for tablet 19314b1a099b43b6a02af8fd9ab48a94 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:01.659183 13744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19314b1a099b43b6a02af8fd9ab48a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.661113 13820 tablet_service.cc:1511] Processing CreateTablet for tablet 19314b1a099b43b6a02af8fd9ab48a94 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:01.663281 13820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19314b1a099b43b6a02af8fd9ab48a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.680794 13983 tablet_bootstrap.cc:492] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:06:01.681567 13983 ts_tablet_manager.cc:1403] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.142s	user 0.018s	sys 0.063s
I20260709 13:06:01.683590 13667 tablet_service.cc:1511] Processing CreateTablet for tablet d93de0de47b745409848ad2a90e15c56 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:01.685346 13667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d93de0de47b745409848ad2a90e15c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.688685 13983 raft_consensus.cc:359] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.689728 13983 raft_consensus.cc:385] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.690105 13983 raft_consensus.cc:740] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.691138 13983 consensus_queue.cc:260] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.693509 13745 tablet_service.cc:1511] Processing CreateTablet for tablet d93de0de47b745409848ad2a90e15c56 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:01.695240 13745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d93de0de47b745409848ad2a90e15c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.696441 13671 tablet_service.cc:1511] Processing CreateTablet for tablet 088f747c5a23409da219e5ee322861f1 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:01.697923 13671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 088f747c5a23409da219e5ee322861f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.700124 13819 tablet_service.cc:1511] Processing CreateTablet for tablet d93de0de47b745409848ad2a90e15c56 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:01.702375 13819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d93de0de47b745409848ad2a90e15c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.710320 13983 ts_tablet_manager.cc:1434] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.012s
I20260709 13:06:01.712023 13983 tablet_bootstrap.cc:492] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:06:01.712824 13748 tablet_service.cc:1511] Processing CreateTablet for tablet 088f747c5a23409da219e5ee322861f1 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:01.714344 13748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 088f747c5a23409da219e5ee322861f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.715641 13986 tablet_bootstrap.cc:654] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.726015 13983 tablet_bootstrap.cc:654] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.734371 13821 tablet_service.cc:1511] Processing CreateTablet for tablet 088f747c5a23409da219e5ee322861f1 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:01.736706 13821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 088f747c5a23409da219e5ee322861f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.765210 13672 tablet_service.cc:1511] Processing CreateTablet for tablet 3e790589979d4c3ca39266386da454c6 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:01.767614 13672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e790589979d4c3ca39266386da454c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.791635 13984 tablet_bootstrap.cc:492] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:06:01.792431 13984 ts_tablet_manager.cc:1403] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.178s	user 0.019s	sys 0.029s
I20260709 13:06:01.795576 13747 tablet_service.cc:1511] Processing CreateTablet for tablet 3e790589979d4c3ca39266386da454c6 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:01.796291 13984 raft_consensus.cc:359] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.804965 13817 tablet_service.cc:1511] Processing CreateTablet for tablet 3e790589979d4c3ca39266386da454c6 (DEFAULT_TABLE table=soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:01.807363 13817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e790589979d4c3ca39266386da454c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.817807 13747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e790589979d4c3ca39266386da454c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.799243 13984 raft_consensus.cc:385] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.846964 13984 raft_consensus.cc:740] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.847841 13984 consensus_queue.cc:260] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.854015 13984 ts_tablet_manager.cc:1434] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.004s
I20260709 13:06:01.855156 13984 tablet_bootstrap.cc:492] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:06:01.868551 13986 tablet_bootstrap.cc:492] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:06:01.869393 13986 ts_tablet_manager.cc:1403] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.213s	user 0.013s	sys 0.011s
I20260709 13:06:01.875499 13983 tablet_bootstrap.cc:492] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:06:01.876227 13983 ts_tablet_manager.cc:1403] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.164s	user 0.057s	sys 0.066s
I20260709 13:06:01.880676 13983 raft_consensus.cc:359] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.881860 13983 raft_consensus.cc:385] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.882181 13983 raft_consensus.cc:740] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.883073 13983 consensus_queue.cc:260] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.889026 13983 ts_tablet_manager.cc:1434] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:06:01.890324 13983 tablet_bootstrap.cc:492] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:06:01.898425 13986 raft_consensus.cc:359] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.899560 13968 raft_consensus.cc:493] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.902063 13986 raft_consensus.cc:385] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.901947 13968 raft_consensus.cc:515] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.902745 13986 raft_consensus.cc:740] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.903820 13986 consensus_queue.cc:260] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.904949 13968 leader_election.cc:290] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:01.906726 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a561890d97b64f298023166984ccf29b" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:01.908783 13832 raft_consensus.cc:2468] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:06:01.907261 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a561890d97b64f298023166984ccf29b" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
W20260709 13:06:01.912207 13722 leader_election.cc:343] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:01.912577 13986 ts_tablet_manager.cc:1434] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.002s
I20260709 13:06:01.910000 13983 tablet_bootstrap.cc:654] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.913018 13723 leader_election.cc:304] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:01.913950 13986 tablet_bootstrap.cc:492] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:06:01.914501 13968 raft_consensus.cc:2804] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:01.914932 13968 raft_consensus.cc:493] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.915772 13968 raft_consensus.cc:3060] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:01.908056 13984 tablet_bootstrap.cc:654] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.925386 13986 tablet_bootstrap.cc:654] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.928546 13983 tablet_bootstrap.cc:492] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:06:01.929325 13983 ts_tablet_manager.cc:1403] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.000s
I20260709 13:06:01.940886 13968 raft_consensus.cc:515] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.942839 13984 tablet_bootstrap.cc:492] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:06:01.942613 13983 raft_consensus.cc:359] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.944737 13984 ts_tablet_manager.cc:1403] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.090s	user 0.015s	sys 0.005s
I20260709 13:06:01.945024 13983 raft_consensus.cc:385] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.945616 13983 raft_consensus.cc:740] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.946874 13983 consensus_queue.cc:260] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.947105 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a561890d97b64f298023166984ccf29b" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:01.950959 13984 raft_consensus.cc:359] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.951851 13983 ts_tablet_manager.cc:1434] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.022s	user 0.012s	sys 0.000s
W20260709 13:06:01.952570 13722 leader_election.cc:343] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:01.955232 13983 tablet_bootstrap.cc:492] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:06:01.954015 13984 raft_consensus.cc:385] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.955431 13986 tablet_bootstrap.cc:492] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:06:01.955875 13984 raft_consensus.cc:740] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.957644 13986 ts_tablet_manager.cc:1403] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.044s	user 0.032s	sys 0.008s
I20260709 13:06:01.958276 13984 consensus_queue.cc:260] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.961678 13984 ts_tablet_manager.cc:1434] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.016s	user 0.016s	sys 0.000s
I20260709 13:06:01.963152 13984 tablet_bootstrap.cc:492] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:06:01.964735 13986 raft_consensus.cc:359] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.965873 13986 raft_consensus.cc:385] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.966403 13986 raft_consensus.cc:740] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.972361 13986 consensus_queue.cc:260] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:01.973980 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a561890d97b64f298023166984ccf29b" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:01.974758 13832 raft_consensus.cc:3060] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:01.974854 13983 tablet_bootstrap.cc:654] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.977097 13986 ts_tablet_manager.cc:1434] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.004s
I20260709 13:06:01.970448 13968 leader_election.cc:290] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:01.980105 13984 tablet_bootstrap.cc:654] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.980640 13986 tablet_bootstrap.cc:492] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:06:01.995074 13986 tablet_bootstrap.cc:654] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.996737 13984 tablet_bootstrap.cc:492] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:06:01.998008 13984 ts_tablet_manager.cc:1403] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.035s	user 0.022s	sys 0.003s
I20260709 13:06:02.004468 13984 raft_consensus.cc:359] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.005689 13984 raft_consensus.cc:385] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.006104 13984 raft_consensus.cc:740] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.007438 13984 consensus_queue.cc:260] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.011088 13986 tablet_bootstrap.cc:492] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:06:02.012785 13986 ts_tablet_manager.cc:1403] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.008s
I20260709 13:06:02.011129 13984 ts_tablet_manager.cc:1434] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.004s
I20260709 13:06:02.019150 13984 tablet_bootstrap.cc:492] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:06:02.021431 13986 raft_consensus.cc:359] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.023070 13986 raft_consensus.cc:385] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.023423 13986 raft_consensus.cc:740] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.024469 13986 consensus_queue.cc:260] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.027810 13986 ts_tablet_manager.cc:1434] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:06:02.029156 13986 tablet_bootstrap.cc:492] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:06:02.031136 13990 raft_consensus.cc:493] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.031809 13990 raft_consensus.cc:515] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.033875 13984 tablet_bootstrap.cc:654] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.035849 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed4ba4005fe473f823843b6027e2d53" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:06:02.036928 13990 leader_election.cc:290] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.036206 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed4ba4005fe473f823843b6027e2d53" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:02.038025 13830 raft_consensus.cc:2468] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:06:02.039741 13723 leader_election.cc:304] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:02.040722 13990 raft_consensus.cc:2804] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.041153 13990 raft_consensus.cc:493] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.041558 13990 raft_consensus.cc:3060] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.037107 13682 raft_consensus.cc:2468] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:06:02.054601 13986 tablet_bootstrap.cc:654] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.058840 13971 raft_consensus.cc:493] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.059573 13971 raft_consensus.cc:515] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.065968 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:02.066519 13971 leader_election.cc:290] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
W20260709 13:06:02.069514 13797 leader_election.cc:343] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.068571 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:06:02.071938 13682 raft_consensus.cc:2468] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 0.
I20260709 13:06:02.071609 13990 raft_consensus.cc:515] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.075980 13990 leader_election.cc:290] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.079675 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed4ba4005fe473f823843b6027e2d53" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:02.080608 13830 raft_consensus.cc:3060] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.081756 13796 leader_election.cc:304] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: d8474050515746e49b6550d9f1cacf9a
I20260709 13:06:02.082368 13986 tablet_bootstrap.cc:492] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:06:02.083400 13986 ts_tablet_manager.cc:1403] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.002s
I20260709 13:06:02.083434 13971 raft_consensus.cc:2804] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.084419 13971 raft_consensus.cc:493] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.084846 13971 raft_consensus.cc:3060] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.088467 13986 raft_consensus.cc:359] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.092998 13986 raft_consensus.cc:385] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.093461 13986 raft_consensus.cc:740] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.103420 13986 consensus_queue.cc:260] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.106318 13972 raft_consensus.cc:493] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.106980 13972 raft_consensus.cc:515] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.116612 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed4ba4005fe473f823843b6027e2d53" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:02.118343 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a561890d97b64f298023166984ccf29b" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:02.130393 13756 raft_consensus.cc:2393] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 66a96c01be0e4bc2b60dc695623d1dd7 in current term 1: Already voted for candidate d8474050515746e49b6550d9f1cacf9a in this term.
I20260709 13:06:02.119459 13972 leader_election.cc:290] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.132416 13682 raft_consensus.cc:3060] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:02.121888 13796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42701, user_credentials={real_user=slave}} blocked reactor thread for 46921.6us
I20260709 13:06:02.122900 13986 ts_tablet_manager.cc:1434] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.039s	user 0.017s	sys 0.008s
I20260709 13:06:02.120337 13831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a561890d97b64f298023166984ccf29b" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:02.138803 13986 tablet_bootstrap.cc:492] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:06:02.141431 13648 leader_election.cc:304] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7; no voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4
I20260709 13:06:02.144016 13972 raft_consensus.cc:3060] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.160529 13832 raft_consensus.cc:2468] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:06:02.138381 13971 raft_consensus.cc:515] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.173410 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:02.174268 13683 raft_consensus.cc:3060] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.181322 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:02.182425 13990 raft_consensus.cc:493] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.183444 13990 raft_consensus.cc:515] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
W20260709 13:06:02.183986 13797 leader_election.cc:343] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.184540 13830 raft_consensus.cc:2468] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:06:02.184804 13971 leader_election.cc:290] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:02.193786 13723 leader_election.cc:304] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:02.195073 13968 raft_consensus.cc:2804] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.195130 13682 raft_consensus.cc:2468] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:06:02.196655 13723 leader_election.cc:304] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:02.200192 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160b2f7e43224902824f79a186ead89c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:02.200444 13680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160b2f7e43224902824f79a186ead89c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:06:02.201565 13680 raft_consensus.cc:2468] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:06:02.200397 13972 raft_consensus.cc:2749] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:02.207218 13629 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:02.208050 13986 tablet_bootstrap.cc:654] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.199213 13990 leader_election.cc:290] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.211804 13984 tablet_bootstrap.cc:492] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:06:02.211426 13722 leader_election.cc:304] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:06:02.212580 13984 ts_tablet_manager.cc:1403] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.194s	user 0.069s	sys 0.090s
I20260709 13:06:02.212641 13990 raft_consensus.cc:2804] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.213330 13990 raft_consensus.cc:697] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:02.213774 13983 tablet_bootstrap.cc:492] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:06:02.214658 13983 ts_tablet_manager.cc:1403] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.260s	user 0.072s	sys 0.085s
I20260709 13:06:02.215277 13990 consensus_queue.cc:237] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
W20260709 13:06:02.222841 13723 leader_election.cc:343] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.223165 13984 raft_consensus.cc:359] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.226719 13984 raft_consensus.cc:385] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.227608 13984 raft_consensus.cc:740] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.228416 13683 raft_consensus.cc:2468] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 1.
I20260709 13:06:02.229378 13984 consensus_queue.cc:260] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.237118 13968 raft_consensus.cc:697] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:02.236671 14001 raft_consensus.cc:2804] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.242151 13796 leader_election.cc:304] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: d8474050515746e49b6550d9f1cacf9a
I20260709 13:06:02.242518 14001 raft_consensus.cc:493] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.242671 13968 consensus_queue.cc:237] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.243566 14001 raft_consensus.cc:3060] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.244282 13971 raft_consensus.cc:2804] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.246417 13984 ts_tablet_manager.cc:1434] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.033s	user 0.012s	sys 0.003s
I20260709 13:06:02.256915 13971 raft_consensus.cc:697] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:06:02.259332 13983 raft_consensus.cc:359] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.263334 13971 consensus_queue.cc:237] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.262434 13983 raft_consensus.cc:385] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.264715 13986 tablet_bootstrap.cc:492] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:06:02.268131 13983 raft_consensus.cc:740] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.274390 13984 tablet_bootstrap.cc:492] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:06:02.276818 13986 ts_tablet_manager.cc:1403] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.138s	user 0.025s	sys 0.015s
I20260709 13:06:02.277454 13983 consensus_queue.cc:260] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.281524 13983 ts_tablet_manager.cc:1434] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.004s
I20260709 13:06:02.283138 13983 tablet_bootstrap.cc:492] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:06:02.298827 13983 tablet_bootstrap.cc:654] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.299028 13984 tablet_bootstrap.cc:654] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.302438 13580 catalog_manager.cc:5697] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:02.312618 14001 raft_consensus.cc:515] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.307273 13986 raft_consensus.cc:359] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.310279 13581 catalog_manager.cc:5697] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.316916 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160b2f7e43224902824f79a186ead89c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:02.316992 13986 raft_consensus.cc:385] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.317600 13984 tablet_bootstrap.cc:492] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:06:02.318259 13683 raft_consensus.cc:3060] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.318197 13986 raft_consensus.cc:740] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.319152 13984 ts_tablet_manager.cc:1403] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.012s
I20260709 13:06:02.319593 13986 consensus_queue.cc:260] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.323864 13986 ts_tablet_manager.cc:1434] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.002s
I20260709 13:06:02.324043 13984 raft_consensus.cc:359] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.325054 13984 raft_consensus.cc:385] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.325142 13986 tablet_bootstrap.cc:492] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:06:02.325438 13984 raft_consensus.cc:740] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.328094 13972 raft_consensus.cc:493] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.327134 13984 consensus_queue.cc:260] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.328853 13972 raft_consensus.cc:515] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.331465 13972 leader_election.cc:290] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.332726 13984 ts_tablet_manager.cc:1434] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.001s
I20260709 13:06:02.337069 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ebac8961d041c0bf43bd08a733309b" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:02.337776 13984 tablet_bootstrap.cc:492] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:06:02.338311 13986 tablet_bootstrap.cc:654] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.338835 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ebac8961d041c0bf43bd08a733309b" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
W20260709 13:06:02.341259 13648 leader_election.cc:343] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:02.342235 13648 leader_election.cc:343] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.342767 13648 leader_election.cc:304] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7; no voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4
I20260709 13:06:02.344462 13972 raft_consensus.cc:2749] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:02.346071 13984 tablet_bootstrap.cc:654] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.348045 13983 tablet_bootstrap.cc:492] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:06:02.349007 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160b2f7e43224902824f79a186ead89c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:02.350821 13832 raft_consensus.cc:3060] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.350342 13983 ts_tablet_manager.cc:1403] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.067s	user 0.022s	sys 0.009s
I20260709 13:06:02.362461 13984 tablet_bootstrap.cc:492] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:06:02.363202 14001 leader_election.cc:290] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.362232 13983 raft_consensus.cc:359] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.370021 13983 raft_consensus.cc:385] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.370447 13983 raft_consensus.cc:740] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.371654 13986 tablet_bootstrap.cc:492] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:06:02.373390 13986 ts_tablet_manager.cc:1403] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.049s	user 0.030s	sys 0.012s
I20260709 13:06:02.375749 13984 ts_tablet_manager.cc:1403] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.038s	user 0.023s	sys 0.000s
I20260709 13:06:02.378358 13683 raft_consensus.cc:2468] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:06:02.371699 13983 consensus_queue.cc:260] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.380270 13722 leader_election.cc:304] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:06:02.383857 13983 ts_tablet_manager.cc:1434] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:06:02.382052 14001 raft_consensus.cc:2804] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.385746 14001 raft_consensus.cc:697] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:02.386416 13986 raft_consensus.cc:359] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.387566 13986 raft_consensus.cc:385] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.386888 14001 consensus_queue.cc:237] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.387854 13984 raft_consensus.cc:359] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.388109 13986 raft_consensus.cc:740] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.388736 13984 raft_consensus.cc:385] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.389106 13984 raft_consensus.cc:740] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.389837 13986 consensus_queue.cc:260] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.385883 13983 tablet_bootstrap.cc:492] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:06:02.390192 13984 consensus_queue.cc:260] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.396402 13984 ts_tablet_manager.cc:1434] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:06:02.397655 13984 tablet_bootstrap.cc:492] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:06:02.398348 13832 raft_consensus.cc:2468] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:06:02.406344 13986 ts_tablet_manager.cc:1434] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.032s	user 0.010s	sys 0.002s
I20260709 13:06:02.407262 13984 tablet_bootstrap.cc:654] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.408085 13986 tablet_bootstrap.cc:492] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:06:02.409641 13983 tablet_bootstrap.cc:654] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.435708 13580 catalog_manager.cc:5697] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.442456 13580 catalog_manager.cc:5697] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.439600 13986 tablet_bootstrap.cc:654] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.445412 13984 tablet_bootstrap.cc:492] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:06:02.445544 13983 tablet_bootstrap.cc:492] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:06:02.446372 13984 ts_tablet_manager.cc:1403] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.049s	user 0.030s	sys 0.015s
I20260709 13:06:02.446426 13983 ts_tablet_manager.cc:1403] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.061s	user 0.027s	sys 0.011s
I20260709 13:06:02.452780 13983 raft_consensus.cc:359] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.453804 13983 raft_consensus.cc:385] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.454142 13983 raft_consensus.cc:740] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.455103 13983 consensus_queue.cc:260] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.450870 13984 raft_consensus.cc:359] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.456481 13984 raft_consensus.cc:385] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.456815 13984 raft_consensus.cc:740] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.458153 13984 consensus_queue.cc:260] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.461200 13984 ts_tablet_manager.cc:1434] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:06:02.462762 13984 tablet_bootstrap.cc:492] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:06:02.466451 13983 ts_tablet_manager.cc:1434] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.003s
I20260709 13:06:02.467823 13983 tablet_bootstrap.cc:492] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:06:02.472081 13984 tablet_bootstrap.cc:654] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.474241 14001 raft_consensus.cc:493] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.474864 14001 raft_consensus.cc:515] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.477401 14001 leader_election.cc:290] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.477602 13983 tablet_bootstrap.cc:654] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.478780 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d93de0de47b745409848ad2a90e15c56" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:02.480747 13986 tablet_bootstrap.cc:492] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:06:02.481436 13986 ts_tablet_manager.cc:1403] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.074s	user 0.022s	sys 0.008s
I20260709 13:06:02.482028 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d93de0de47b745409848ad2a90e15c56" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:06:02.483057 13683 raft_consensus.cc:2468] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:06:02.486143 13986 raft_consensus.cc:359] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.486693 13722 leader_election.cc:304] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:06:02.487227 13986 raft_consensus.cc:385] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.487598 13986 raft_consensus.cc:740] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.488873 13968 raft_consensus.cc:2804] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.489439 13968 raft_consensus.cc:493] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.488440 13986 consensus_queue.cc:260] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.489876 13968 raft_consensus.cc:3060] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.492280 13986 ts_tablet_manager.cc:1434] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
W20260709 13:06:02.493605 13723 leader_election.cc:343] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.497057 13986 tablet_bootstrap.cc:492] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:06:02.498929 13983 tablet_bootstrap.cc:492] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:06:02.499943 13983 ts_tablet_manager.cc:1403] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.012s
I20260709 13:06:02.500797 13968 raft_consensus.cc:515] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.503641 13983 raft_consensus.cc:359] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.504686 13983 raft_consensus.cc:385] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.504974 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d93de0de47b745409848ad2a90e15c56" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:02.505776 13683 raft_consensus.cc:3060] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.505030 13983 raft_consensus.cc:740] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.507073 13986 tablet_bootstrap.cc:654] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.507084 13983 consensus_queue.cc:260] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.509996 13983 ts_tablet_manager.cc:1434] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.001s
I20260709 13:06:02.497136 13984 tablet_bootstrap.cc:492] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:06:02.512171 13984 ts_tablet_manager.cc:1403] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.020s
I20260709 13:06:02.512676 13968 leader_election.cc:290] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.511220 13983 tablet_bootstrap.cc:492] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:06:02.514456 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d93de0de47b745409848ad2a90e15c56" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:02.515380 13832 raft_consensus.cc:3060] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.516153 13984 raft_consensus.cc:359] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.517729 13984 raft_consensus.cc:385] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.518082 13683 raft_consensus.cc:2468] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:06:02.518318 13984 raft_consensus.cc:740] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.519394 13984 consensus_queue.cc:260] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.519763 13722 leader_election.cc:304] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:06:02.522006 13983 tablet_bootstrap.cc:654] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.523674 14001 raft_consensus.cc:2804] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.523932 13968 raft_consensus.cc:493] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.524686 14001 raft_consensus.cc:697] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:02.524613 13968 raft_consensus.cc:515] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.525717 13984 ts_tablet_manager.cc:1434] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.003s
I20260709 13:06:02.531771 13984 tablet_bootstrap.cc:492] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a: Bootstrap starting.
I20260709 13:06:02.534389 13968 leader_election.cc:290] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.536062 14001 consensus_queue.cc:237] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.538118 13971 raft_consensus.cc:493] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.538781 13971 raft_consensus.cc:515] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.543641 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:02.552139 13971 leader_election.cc:290] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:02.552723 13680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19314b1a099b43b6a02af8fd9ab48a94" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:06:02.553634 13680 raft_consensus.cc:2468] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 0.
I20260709 13:06:02.554239 13984 tablet_bootstrap.cc:654] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.554517 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:06:02.555510 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19314b1a099b43b6a02af8fd9ab48a94" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:02.556951 13683 raft_consensus.cc:2393] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8474050515746e49b6550d9f1cacf9a in current term 1: Already voted for candidate fef52998d77a4a089f489e74975261b4 in this term.
I20260709 13:06:02.557686 13796 leader_election.cc:304] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:02.559185 13722 leader_election.cc:304] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a; no voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4
I20260709 13:06:02.560988 13968 raft_consensus.cc:3060] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.556850 13756 raft_consensus.cc:2468] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 0.
I20260709 13:06:02.566638 13971 raft_consensus.cc:2804] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.567214 13971 raft_consensus.cc:493] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.567541 13971 raft_consensus.cc:3060] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.578881 13968 raft_consensus.cc:2749] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:02.585705 13832 raft_consensus.cc:2468] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:06:02.591737 13971 raft_consensus.cc:515] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.598743 13986 tablet_bootstrap.cc:492] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:06:02.599759 13986 ts_tablet_manager.cc:1403] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.103s	user 0.015s	sys 0.039s
I20260709 13:06:02.602346 13971 leader_election.cc:290] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:02.603075 13983 tablet_bootstrap.cc:492] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:06:02.603175 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19314b1a099b43b6a02af8fd9ab48a94" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:02.603746 13983 ts_tablet_manager.cc:1403] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.093s	user 0.044s	sys 0.043s
I20260709 13:06:02.604209 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19314b1a099b43b6a02af8fd9ab48a94" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:02.603956 13986 raft_consensus.cc:359] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.604868 13683 raft_consensus.cc:3060] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.608223 13984 tablet_bootstrap.cc:492] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a: No bootstrap required, opened a new log
I20260709 13:06:02.604101 13756 raft_consensus.cc:3060] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.607942 13983 raft_consensus.cc:359] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.609346 13984 ts_tablet_manager.cc:1403] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a: Time spent bootstrapping tablet: real 0.078s	user 0.017s	sys 0.017s
I20260709 13:06:02.609407 13983 raft_consensus.cc:385] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.609824 13983 raft_consensus.cc:740] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.610867 13983 consensus_queue.cc:260] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.613111 13983 ts_tablet_manager.cc:1434] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:06:02.612876 13984 raft_consensus.cc:359] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.613778 13984 raft_consensus.cc:385] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.614061 13984 raft_consensus.cc:740] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.614266 13983 tablet_bootstrap.cc:492] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4: Bootstrap starting.
I20260709 13:06:02.614795 13984 consensus_queue.cc:260] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.605016 13986 raft_consensus.cc:385] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.616005 13986 raft_consensus.cc:740] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.616819 13986 consensus_queue.cc:260] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.619886 13986 ts_tablet_manager.cc:1434] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 13:06:02.621464 13986 tablet_bootstrap.cc:492] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7: Bootstrap starting.
I20260709 13:06:02.624243 13983 tablet_bootstrap.cc:654] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.622756 13580 catalog_manager.cc:5697] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.632251 13683 raft_consensus.cc:2468] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 1.
I20260709 13:06:02.634660 13796 leader_election.cc:304] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:02.636288 13971 raft_consensus.cc:2804] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.636869 13984 ts_tablet_manager.cc:1434] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:06:02.636915 13971 raft_consensus.cc:697] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:06:02.652971 14001 consensus_queue.cc:1048] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:02.655520 13756 raft_consensus.cc:2468] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 1.
I20260709 13:06:02.659528 13986 tablet_bootstrap.cc:654] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.643117 13971 consensus_queue.cc:237] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.663290 13983 tablet_bootstrap.cc:492] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4: No bootstrap required, opened a new log
I20260709 13:06:02.664332 13983 ts_tablet_manager.cc:1403] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4: Time spent bootstrapping tablet: real 0.050s	user 0.022s	sys 0.014s
I20260709 13:06:02.668653 13983 raft_consensus.cc:359] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.669394 13983 raft_consensus.cc:385] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.669719 13983 raft_consensus.cc:740] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.670612 13983 consensus_queue.cc:260] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.676988 13990 consensus_queue.cc:1048] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:02.685340 13983 ts_tablet_manager.cc:1434] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.001s
I20260709 13:06:02.690048 14007 consensus_queue.cc:1048] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:02.696135 13968 raft_consensus.cc:493] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.696776 13968 raft_consensus.cc:515] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.699409 13968 leader_election.cc:290] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.704394 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088f747c5a23409da219e5ee322861f1" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:02.705888 13832 raft_consensus.cc:2468] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:06:02.710009 13681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088f747c5a23409da219e5ee322861f1" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
W20260709 13:06:02.712810 13722 leader_election.cc:343] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.714564 13723 leader_election.cc:304] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:02.715942 14003 raft_consensus.cc:2804] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.716419 14003 raft_consensus.cc:493] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.716825 14003 raft_consensus.cc:3060] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.700544 13968 consensus_queue.cc:1048] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:02.730319 13986 tablet_bootstrap.cc:492] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7: No bootstrap required, opened a new log
I20260709 13:06:02.731101 13986 ts_tablet_manager.cc:1403] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent bootstrapping tablet: real 0.110s	user 0.034s	sys 0.031s
I20260709 13:06:02.735702 13986 raft_consensus.cc:359] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.736488 13986 raft_consensus.cc:385] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.736763 13986 raft_consensus.cc:740] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.737813 13986 consensus_queue.cc:260] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.741278 13986 ts_tablet_manager.cc:1434] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 13:06:02.689774 13580 catalog_manager.cc:5697] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:02.761222 13971 consensus_queue.cc:1048] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:02.767941 14006 raft_consensus.cc:493] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.768584 14006 raft_consensus.cc:515] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.769919 14003 raft_consensus.cc:515] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.770924 14006 leader_election.cc:290] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:02.773684 13758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e790589979d4c3ca39266386da454c6" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:02.774134 14003 leader_election.cc:290] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.774046 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088f747c5a23409da219e5ee322861f1" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:02.774274 13681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e790589979d4c3ca39266386da454c6" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:06:02.775106 13683 raft_consensus.cc:3060] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.774569 13758 raft_consensus.cc:2468] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 0.
I20260709 13:06:02.779235 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088f747c5a23409da219e5ee322861f1" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:02.781379 13830 raft_consensus.cc:3060] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.782631 14003 raft_consensus.cc:493] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.783512 14003 raft_consensus.cc:515] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.775272 13681 raft_consensus.cc:2468] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 0.
I20260709 13:06:02.786055 14003 leader_election.cc:290] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.787098 13796 leader_election.cc:304] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:02.787679 13681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ebac8961d041c0bf43bd08a733309b" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:06:02.788524 13681 raft_consensus.cc:2468] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:06:02.789054 13832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ebac8961d041c0bf43bd08a733309b" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:02.789966 13722 leader_election.cc:304] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:06:02.790968 14003 raft_consensus.cc:2804] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.787715 14006 raft_consensus.cc:493] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.791662 14003 raft_consensus.cc:493] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.792090 14003 raft_consensus.cc:3060] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.790022 13968 consensus_queue.cc:1048] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:02.791916 14006 raft_consensus.cc:515] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.801452 13832 raft_consensus.cc:2468] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 0.
I20260709 13:06:02.791848 13971 raft_consensus.cc:2804] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.805258 14003 raft_consensus.cc:515] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.830796 13682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ebac8961d041c0bf43bd08a733309b" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:02.832047 13682 raft_consensus.cc:3060] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.826376 13681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ebac8961d041c0bf43bd08a733309b" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" is_pre_election: true
I20260709 13:06:02.815961 13971 raft_consensus.cc:493] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.846134 13971 raft_consensus.cc:3060] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.846520 14006 leader_election.cc:290] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:02.848289 14003 leader_election.cc:290] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:02.851307 13682 raft_consensus.cc:2468] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:06:02.853750 13829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ebac8961d041c0bf43bd08a733309b" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:02.854735 13829 raft_consensus.cc:3060] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.831878 13830 raft_consensus.cc:2468] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:06:02.842214 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ebac8961d041c0bf43bd08a733309b" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:02.858336 13756 raft_consensus.cc:2393] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fef52998d77a4a089f489e74975261b4 in current term 1: Already voted for candidate d8474050515746e49b6550d9f1cacf9a in this term.
I20260709 13:06:02.858639 13971 raft_consensus.cc:515] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.863272 13723 leader_election.cc:304] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:02.864929 13797 leader_election.cc:304] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a
I20260709 13:06:02.866626 13968 raft_consensus.cc:2804] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.867460 13968 raft_consensus.cc:697] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:02.868501 13968 consensus_queue.cc:237] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.843600 13683 raft_consensus.cc:2468] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:06:02.874691 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e790589979d4c3ca39266386da454c6" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:02.875006 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e790589979d4c3ca39266386da454c6" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:02.875586 13756 raft_consensus.cc:3060] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.875586 13683 raft_consensus.cc:3060] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.873026 13971 leader_election.cc:290] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:02.877259 13722 leader_election.cc:304] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:06:02.886690 14003 raft_consensus.cc:2804] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.887265 14003 raft_consensus.cc:697] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:02.888233 14003 consensus_queue.cc:237] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.892987 13683 raft_consensus.cc:2468] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 1.
I20260709 13:06:02.894711 13829 raft_consensus.cc:2468] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 1.
I20260709 13:06:02.895557 13796 leader_election.cc:304] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:02.896013 14038 raft_consensus.cc:2749] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:02.897365 13971 raft_consensus.cc:2804] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.900673 13971 raft_consensus.cc:697] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:06:02.902104 13971 consensus_queue.cc:237] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:02.903734 13756 raft_consensus.cc:2468] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 1.
I20260709 13:06:02.904870 13968 consensus_queue.cc:1048] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:02.911191 13580 catalog_manager.cc:5697] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.924190 13581 catalog_manager.cc:5697] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:02.936354 14003 consensus_queue.cc:1048] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:02.942204 13578 catalog_manager.cc:5697] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 1 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.984253 14003 consensus_queue.cc:1048] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:03.000334 14003 consensus_queue.cc:1048] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.051483 13971 consensus_queue.cc:1048] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:03.065256 14038 consensus_queue.cc:1048] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:03.270153 13756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a561890d97b64f298023166984ccf29b"
dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
mode: GRACEFUL
new_leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:55618
I20260709 13:06:03.270925 13756 raft_consensus.cc:606] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Received request to transfer leadership to TS 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:03.278160 13829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a"
dest_uuid: "fef52998d77a4a089f489e74975261b4"
mode: GRACEFUL
new_leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:33410
I20260709 13:06:03.278644 13990 raft_consensus.cc:993] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a: : Instructing follower 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:06:03.278836 13829 raft_consensus.cc:606] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Received request to transfer leadership to TS 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:03.279265 14003 raft_consensus.cc:1081] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Signalling peer 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:06:03.280910 13682 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a561890d97b64f298023166984ccf29b"
dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:54280
I20260709 13:06:03.281584 13682 raft_consensus.cc:493] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.281976 13682 raft_consensus.cc:3060] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.286224 13756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ed4ba4005fe473f823843b6027e2d53"
dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
mode: GRACEFUL
new_leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:55618
I20260709 13:06:03.287222 13756 raft_consensus.cc:606] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Received request to transfer leadership to TS 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:03.290606 13682 raft_consensus.cc:515] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.293243 13682 leader_election.cc:290] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:03.294023 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a561890d97b64f298023166984ccf29b" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:03.294217 13758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "160b2f7e43224902824f79a186ead89c"
dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
mode: GRACEFUL
new_leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:55618
I20260709 13:06:03.294703 13829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a561890d97b64f298023166984ccf29b" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:03.294976 13758 raft_consensus.cc:606] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Received request to transfer leadership to TS 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:03.294821 13756 raft_consensus.cc:3055] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.295506 13829 raft_consensus.cc:3060] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.295950 13756 raft_consensus.cc:740] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:03.296979 13756 consensus_queue.cc:260] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.298527 13756 raft_consensus.cc:3060] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.307943 13756 raft_consensus.cc:2468] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:06:03.309118 13829 raft_consensus.cc:2468] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:06:03.309434 13648 leader_election.cc:304] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:06:03.310587 13994 raft_consensus.cc:2804] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:03.311146 13994 raft_consensus.cc:697] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Becoming Leader. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Running, Role: LEADER
I20260709 13:06:03.312433 13829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19314b1a099b43b6a02af8fd9ab48a94"
dest_uuid: "fef52998d77a4a089f489e74975261b4"
mode: GRACEFUL
new_leader_uuid: "d8474050515746e49b6550d9f1cacf9a"
 from {username='slave'} at 127.0.0.1:33410
I20260709 13:06:03.312372 13994 consensus_queue.cc:237] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.313154 13829 raft_consensus.cc:606] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Received request to transfer leadership to TS d8474050515746e49b6550d9f1cacf9a
I20260709 13:06:03.324214 13578 catalog_manager.cc:5697] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 reported cstate change: term changed from 1 to 2, leader changed from d8474050515746e49b6550d9f1cacf9a (127.0.107.194) to 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.382361 14003 consensus_queue.cc:1048] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:03.395229 13990 consensus_queue.cc:1048] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:03.398761 14003 consensus_queue.cc:1048] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.421222 13990 consensus_queue.cc:1048] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:03.424830 14038 consensus_queue.cc:1048] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:03.459604 14038 consensus_queue.cc:1048] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:03.460384 13990 raft_consensus.cc:993] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a: : Instructing follower 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:06:03.461127 14001 raft_consensus.cc:1081] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Signalling peer 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:06:03.463629 13683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "160b2f7e43224902824f79a186ead89c"
dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:54280
I20260709 13:06:03.465679 13683 raft_consensus.cc:493] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.466084 13683 raft_consensus.cc:3060] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.477921 13683 raft_consensus.cc:515] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.480480 13683 leader_election.cc:290] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:03.481726 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160b2f7e43224902824f79a186ead89c" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:03.482126 13829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160b2f7e43224902824f79a186ead89c" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:03.482825 13756 raft_consensus.cc:3055] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.482847 13829 raft_consensus.cc:3060] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.483349 13756 raft_consensus.cc:740] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:03.485688 13756 consensus_queue.cc:260] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.487181 13756 raft_consensus.cc:3060] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.492731 13829 raft_consensus.cc:2468] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:06:03.494737 13648 leader_election.cc:304] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:03.495482 13972 raft_consensus.cc:2804] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:03.496609 13972 raft_consensus.cc:697] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Becoming Leader. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Running, Role: LEADER
I20260709 13:06:03.497862 13756 raft_consensus.cc:2468] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:06:03.498454 13972 consensus_queue.cc:237] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.511407 13581 catalog_manager.cc:5697] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 reported cstate change: term changed from 1 to 2, leader changed from d8474050515746e49b6550d9f1cacf9a (127.0.107.194) to 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.598079 14038 raft_consensus.cc:993] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4: : Instructing follower 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:06:03.598769 14006 raft_consensus.cc:1081] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Signalling peer 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:06:03.600876 13683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a"
dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:54288
I20260709 13:06:03.601629 13683 raft_consensus.cc:493] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.601984 13683 raft_consensus.cc:3060] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.609539 13683 raft_consensus.cc:515] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.611129 13683 leader_election.cc:290] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:03.612105 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:03.612515 13829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:03.612897 13756 raft_consensus.cc:3060] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.613196 13829 raft_consensus.cc:3055] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.613605 13829 raft_consensus.cc:740] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:06:03.614553 13829 consensus_queue.cc:260] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.616323 13829 raft_consensus.cc:3060] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.622633 13756 raft_consensus.cc:2468] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:06:03.624374 13648 leader_election.cc:304] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:06:03.625052 13829 raft_consensus.cc:2468] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:06:03.625730 13994 raft_consensus.cc:2804] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:03.626255 13994 raft_consensus.cc:697] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Becoming Leader. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Running, Role: LEADER
I20260709 13:06:03.627211 13994 consensus_queue.cc:237] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.635442 13581 catalog_manager.cc:5697] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 reported cstate change: term changed from 1 to 2, leader changed from fef52998d77a4a089f489e74975261b4 (127.0.107.195) to 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.655761 14006 raft_consensus.cc:993] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4: : Instructing follower d8474050515746e49b6550d9f1cacf9a to start an election
I20260709 13:06:03.656327 13971 raft_consensus.cc:1081] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Signalling peer d8474050515746e49b6550d9f1cacf9a to start an election
I20260709 13:06:03.658237 13756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "19314b1a099b43b6a02af8fd9ab48a94"
dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
 from {username='slave'} at 127.0.0.1:55612
I20260709 13:06:03.658881 13756 raft_consensus.cc:493] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.659269 13756 raft_consensus.cc:3060] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.665994 13756 raft_consensus.cc:515] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.668386 13756 leader_election.cc:290] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 2 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:03.669370 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19314b1a099b43b6a02af8fd9ab48a94" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:03.669709 13829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19314b1a099b43b6a02af8fd9ab48a94" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:03.670289 13683 raft_consensus.cc:3060] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.670328 13829 raft_consensus.cc:3055] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.670879 13829 raft_consensus.cc:740] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:06:03.671808 13829 consensus_queue.cc:260] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.673179 13829 raft_consensus.cc:3060] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.679291 13683 raft_consensus.cc:2468] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 2.
I20260709 13:06:03.681422 13722 leader_election.cc:304] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a; no voters: 
I20260709 13:06:03.681913 13829 raft_consensus.cc:2468] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 2.
I20260709 13:06:03.682973 13990 raft_consensus.cc:2804] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:03.683539 13990 raft_consensus.cc:697] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 2 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:03.684511 13990 consensus_queue.cc:237] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.695380 13581 catalog_manager.cc:5697] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 1 to 2, leader changed from fef52998d77a4a089f489e74975261b4 (127.0.107.195) to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 2 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.809974 13968 raft_consensus.cc:993] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a: : Instructing follower 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:06:03.810575 13968 raft_consensus.cc:1081] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Signalling peer 66a96c01be0e4bc2b60dc695623d1dd7 to start an election
I20260709 13:06:03.812325 13682 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ed4ba4005fe473f823843b6027e2d53"
dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
 from {username='slave'} at 127.0.0.1:54280
I20260709 13:06:03.813064 13682 raft_consensus.cc:493] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.813738 13682 raft_consensus.cc:3060] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.822206 13682 raft_consensus.cc:515] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.824203 13682 leader_election.cc:290] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:03.825762 13829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed4ba4005fe473f823843b6027e2d53" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:03.826560 13829 raft_consensus.cc:3060] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.825620 13756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed4ba4005fe473f823843b6027e2d53" candidate_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:03.831099 13756 raft_consensus.cc:3055] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.831514 13756 raft_consensus.cc:740] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:03.834102 13756 consensus_queue.cc:260] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.835580 13756 raft_consensus.cc:3060] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.841975 13829 raft_consensus.cc:2468] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:06:03.843729 13648 leader_election.cc:304] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:03.845175 13994 raft_consensus.cc:2804] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:03.846076 13994 raft_consensus.cc:697] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Becoming Leader. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Running, Role: LEADER
I20260709 13:06:03.846925 13994 consensus_queue.cc:237] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:03.847447 13756 raft_consensus.cc:2468] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a96c01be0e4bc2b60dc695623d1dd7 in term 2.
I20260709 13:06:03.856261 13581 catalog_manager.cc:5697] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 reported cstate change: term changed from 1 to 2, leader changed from d8474050515746e49b6550d9f1cacf9a (127.0.107.194) to 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.908550 13829 raft_consensus.cc:1275] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:03.909998 13994 consensus_queue.cc:1048] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:03.922735 13756 raft_consensus.cc:1275] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:03.926554 13972 consensus_queue.cc:1048] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:04.024016 13758 raft_consensus.cc:1275] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.025620 13994 consensus_queue.cc:1048] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:04.038769 13831 raft_consensus.cc:1275] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.044073 14061 consensus_queue.cc:1048] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:04.097436 13682 raft_consensus.cc:1275] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Refusing update from remote peer d8474050515746e49b6550d9f1cacf9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.099668 14003 consensus_queue.cc:1048] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:04.105163 13829 raft_consensus.cc:1275] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.106992 14061 consensus_queue.cc:1048] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:04.114979 13832 raft_consensus.cc:1275] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Refusing update from remote peer d8474050515746e49b6550d9f1cacf9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.120482 13990 consensus_queue.cc:1048] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:04.125087 13758 raft_consensus.cc:1275] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.128312 13972 consensus_queue.cc:1048] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:04.416441 13758 raft_consensus.cc:1275] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.418139 13994 consensus_queue.cc:1048] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:04.432369 13831 raft_consensus.cc:1275] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Refusing update from remote peer 66a96c01be0e4bc2b60dc695623d1dd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.434895 13994 consensus_queue.cc:1048] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.331413 13580 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44172:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:06:06.365552 13580 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44172:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:06:06.439248   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:06.745982 14122 tablet.cc:1724] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:06.749830 14116 tablet.cc:1724] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:06.752962 14112 log.cc:927] Time spent T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4: Append to log took a long time: real 0.077s	user 0.002s	sys 0.000s
I20260709 13:06:06.771473 14126 tablet.cc:1724] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:06.791788 14127 tablet.cc:1724] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:06.819103 14123 log.cc:927] Time spent T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:06:06.811499 14124 tablet.cc:1724] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:06.842193 14132 tablet.cc:1724] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:06.849776 14130 tablet.cc:1724] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:06.870977 14121 log.cc:927] Time spent T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:06:06.887298 14135 tablet.cc:1724] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:06.894033 14129 tablet.cc:1724] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:06.903416 14134 tablet.cc:1724] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:06.913251 14136 tablet.cc:1724] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:06.916301 14124 tablet.cc:1724] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:06.922868 14116 tablet.cc:1724] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:06.987389 14138 tablet.cc:1724] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:06.940398 14127 tablet.cc:1724] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.020143 14122 tablet.cc:1724] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.044286 14132 tablet.cc:1724] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.047426 14126 tablet.cc:1724] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.048372 14124 tablet.cc:1724] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.058779 13581 catalog_manager.cc:5789] TS d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199) does not have the latest schema for tablet d93de0de47b745409848ad2a90e15c56 (table soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]). Expected version 1 got 0
I20260709 13:06:07.060658 14116 tablet.cc:1724] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.063112 14129 tablet.cc:1724] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.065464 13580 catalog_manager.cc:5789] TS 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701) does not have the latest schema for tablet a561890d97b64f298023166984ccf29b (table soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5]). Expected version 1 got 0
I20260709 13:06:07.068850 13757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae"
dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
mode: GRACEFUL
new_leader_uuid: "fef52998d77a4a089f489e74975261b4"
 from {username='slave'} at 127.0.0.1:49102
I20260709 13:06:07.069852 13757 raft_consensus.cc:606] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 2 LEADER]: Received request to transfer leadership to TS fef52998d77a4a089f489e74975261b4
I20260709 13:06:07.078879 14130 tablet.cc:1724] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.068691 14134 tablet.cc:1724] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.095384 14122 tablet.cc:1724] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.092168 14132 tablet.cc:1724] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.149799 14135 tablet.cc:1724] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.155946 14138 tablet.cc:1724] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:07.194602 13681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b06e5df1f9d475e95bf3e17ff21a125"
dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
mode: GRACEFUL
new_leader_uuid: "fef52998d77a4a089f489e74975261b4"
 from {username='slave'} at 127.0.0.1:42238
I20260709 13:06:07.195277 13681 raft_consensus.cc:606] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 LEADER]: Received request to transfer leadership to TS fef52998d77a4a089f489e74975261b4
I20260709 13:06:07.196853   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:07.197906   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:07.203687 13581 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81: soft_deleted_table [id=8e9f625f2fd74eec93477d255c9bceb5] alter complete (version 1)
I20260709 13:06:07.306430 14137 raft_consensus.cc:993] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a: : Instructing follower fef52998d77a4a089f489e74975261b4 to start an election
I20260709 13:06:07.307113 14089 raft_consensus.cc:1081] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 2 LEADER]: Signalling peer fef52998d77a4a089f489e74975261b4 to start an election
I20260709 13:06:07.308916 13832 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae"
dest_uuid: "fef52998d77a4a089f489e74975261b4"
 from {username='slave'} at 127.0.0.1:33394
I20260709 13:06:07.309397 13832 raft_consensus.cc:493] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.309859 13832 raft_consensus.cc:3060] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.316470 13832 raft_consensus.cc:515] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:07.318854 13832 leader_election.cc:290] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:07.319900 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:07.320724 13757 raft_consensus.cc:3055] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:07.321204 13757 raft_consensus.cc:740] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:07.320623 13681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27618f7c04aa4fe38d0dba2ad47936ae" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:07.321892 13681 raft_consensus.cc:3060] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.322131 13757 consensus_queue.cc:260] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:07.323693 13757 raft_consensus.cc:3060] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.331933 13681 raft_consensus.cc:2468] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 3.
I20260709 13:06:07.333375 13757 raft_consensus.cc:2468] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 3.
I20260709 13:06:07.334139 13796 leader_election.cc:304] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:07.336295 14079 raft_consensus.cc:2804] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:07.337803 14079 raft_consensus.cc:697] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 3 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:06:07.339035 14079 consensus_queue.cc:237] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:07.354128 13578 catalog_manager.cc:5697] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 2 to 3, leader changed from d8474050515746e49b6550d9f1cacf9a (127.0.107.194) to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:07.696916 14109 raft_consensus.cc:993] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7: : Instructing follower fef52998d77a4a089f489e74975261b4 to start an election
I20260709 13:06:07.697921 14100 raft_consensus.cc:1081] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 LEADER]: Signalling peer fef52998d77a4a089f489e74975261b4 to start an election
I20260709 13:06:07.699677 13832 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b06e5df1f9d475e95bf3e17ff21a125"
dest_uuid: "fef52998d77a4a089f489e74975261b4"
 from {username='slave'} at 127.0.0.1:33398
I20260709 13:06:07.700212 13832 raft_consensus.cc:493] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.700501 13832 raft_consensus.cc:3060] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.706398 13832 raft_consensus.cc:515] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:07.708125 13832 leader_election.cc:290] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 2 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:07.709213 13681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b06e5df1f9d475e95bf3e17ff21a125" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66a96c01be0e4bc2b60dc695623d1dd7"
I20260709 13:06:07.710038 13681 raft_consensus.cc:3055] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:07.709919 13754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b06e5df1f9d475e95bf3e17ff21a125" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:07.710510 13681 raft_consensus.cc:740] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66a96c01be0e4bc2b60dc695623d1dd7, State: Running, Role: LEADER
I20260709 13:06:07.710768 13754 raft_consensus.cc:3060] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.711544 13681 consensus_queue.cc:260] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:07.713727 13681 raft_consensus.cc:3060] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:07.719136 13754 raft_consensus.cc:2468] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 2.
I20260709 13:06:07.720564 13797 leader_election.cc:304] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:07.721396 13681 raft_consensus.cc:2468] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 2.
I20260709 13:06:07.721726 14079 raft_consensus.cc:2804] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:07.722489 14079 raft_consensus.cc:697] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 2 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:06:07.723361 14079 consensus_queue.cc:237] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:07.732064 13578 catalog_manager.cc:5697] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 1 to 2, leader changed from 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193) to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:07.843127 13754 raft_consensus.cc:1275] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Refusing update from remote peer fef52998d77a4a089f489e74975261b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:07.844802 14079 consensus_queue.cc:1048] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:07.857008 13681 raft_consensus.cc:1275] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 3 FOLLOWER]: Refusing update from remote peer fef52998d77a4a089f489e74975261b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:07.860886 14079 consensus_queue.cc:1048] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.240307 13754 raft_consensus.cc:1275] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Refusing update from remote peer fef52998d77a4a089f489e74975261b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:08.245080 14152 consensus_queue.cc:1048] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:08.276034 13683 raft_consensus.cc:1275] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Refusing update from remote peer fef52998d77a4a089f489e74975261b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:08.285791 14152 consensus_queue.cc:1048] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:08.536535   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:06:08.574640   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:08.575817   431 tablet_replica.cc:333] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.576967   431 raft_consensus.cc:2243] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.577611   431 raft_consensus.cc:2272] T 088f747c5a23409da219e5ee322861f1 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.580127   431 tablet_replica.cc:333] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.580729   431 raft_consensus.cc:2243] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.581175   431 raft_consensus.cc:2272] T f5ebac8961d041c0bf43bd08a733309b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.583369   431 tablet_replica.cc:333] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.584169   431 raft_consensus.cc:2243] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.585179   431 raft_consensus.cc:2272] T fc9825e0aed94112bde8ae9a46e3a01a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.587785   431 tablet_replica.cc:333] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.588459   431 raft_consensus.cc:2243] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.589370   431 raft_consensus.cc:2272] T 160b2f7e43224902824f79a186ead89c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.592021   431 tablet_replica.cc:333] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.592918   431 raft_consensus.cc:2243] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.593647   431 raft_consensus.cc:2272] T 612d2b206b1e4f3f94e3d8443198708a P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.596773   431 tablet_replica.cc:333] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.597507   431 raft_consensus.cc:2243] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.598352   431 raft_consensus.cc:2272] T 5ed4ba4005fe473f823843b6027e2d53 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.600591   431 tablet_replica.cc:333] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.601235   431 raft_consensus.cc:2243] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.601765   431 raft_consensus.cc:2272] T e1552d38be8d443e913dff6849747abb P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.604945   431 tablet_replica.cc:333] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
W20260709 13:06:08.605314 13796 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 13 similar messages]
I20260709 13:06:08.605746   431 raft_consensus.cc:2243] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.606833   431 raft_consensus.cc:2272] T 6fc0f16f48904e228480fedb878f5d81 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.609748 13796 consensus_peers.cc:597] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.610647   431 tablet_replica.cc:333] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.611462   431 raft_consensus.cc:2243] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.612033   431 raft_consensus.cc:2272] T 27618f7c04aa4fe38d0dba2ad47936ae P 66a96c01be0e4bc2b60dc695623d1dd7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.616111   431 tablet_replica.cc:333] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.616838   431 raft_consensus.cc:2243] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.617868   431 raft_consensus.cc:2272] T 469ecfaf70c64d4492c9ad1d9ecf1960 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.620265   431 tablet_replica.cc:333] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.620934   431 raft_consensus.cc:2243] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.621538   431 raft_consensus.cc:2272] T 3e790589979d4c3ca39266386da454c6 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.623812   431 tablet_replica.cc:333] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.624475   431 raft_consensus.cc:2243] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.625365   431 raft_consensus.cc:2272] T d12988af49734f7abf650836e11ddcac P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.627804   431 tablet_replica.cc:333] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.628734   431 raft_consensus.cc:2243] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.629614   431 raft_consensus.cc:2272] T 19314b1a099b43b6a02af8fd9ab48a94 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.632501   431 tablet_replica.cc:333] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.633219   431 raft_consensus.cc:2243] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.634358   431 raft_consensus.cc:2272] T a561890d97b64f298023166984ccf29b P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.636848   431 tablet_replica.cc:333] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.637569   431 raft_consensus.cc:2243] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.638074   431 raft_consensus.cc:2272] T 8f62acdae3954f39998b708ae870b104 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.640734   431 tablet_replica.cc:333] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.641387   431 raft_consensus.cc:2243] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.642411   431 raft_consensus.cc:2272] T 8f97f3e20011461d8a3c4301a7e4a39c P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.645043   431 tablet_replica.cc:333] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.645773   431 raft_consensus.cc:2243] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.646306   431 raft_consensus.cc:2272] T d93de0de47b745409848ad2a90e15c56 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.648583   431 tablet_replica.cc:333] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7: stopping tablet replica
I20260709 13:06:08.649236   431 raft_consensus.cc:2243] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.649760   431 raft_consensus.cc:2272] T 3b06e5df1f9d475e95bf3e17ff21a125 P 66a96c01be0e4bc2b60dc695623d1dd7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.717868 13722 consensus_peers.cc:597] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.766093 13796 consensus_peers.cc:597] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.785873 13722 consensus_peers.cc:597] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.818368 13796 consensus_peers.cc:597] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.856806 13722 consensus_peers.cc:597] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.879868 13796 consensus_peers.cc:597] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.936672 13722 consensus_peers.cc:597] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:09.019793 13722 consensus_peers.cc:597] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:09.091449 13722 consensus_peers.cc:597] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:09.094951 13722 consensus_peers.cc:597] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:09.309387 13704 debug-util.cc:398] Leaking SignalData structure 0x7b08001e46c0 after lost signal to thread 434
W20260709 13:06:09.311444 13704 debug-util.cc:398] Leaking SignalData structure 0x7b080020aba0 after lost signal to thread 13615
W20260709 13:06:09.314780 13704 debug-util.cc:398] Leaking SignalData structure 0x7b0800379f40 after lost signal to thread 13782
W20260709 13:06:09.316573 13704 debug-util.cc:398] Leaking SignalData structure 0x7b08003d4b20 after lost signal to thread 13856
W20260709 13:06:09.696640   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 13704)
I20260709 13:06:09.706346 14079 raft_consensus.cc:493] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.707705 14079 raft_consensus.cc:515] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:09.710381 14168 raft_consensus.cc:493] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.711529 14168 raft_consensus.cc:515] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:09.714227 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:09.716106 14168 leader_election.cc:290] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:09.716238 13757 raft_consensus.cc:2468] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 2.
I20260709 13:06:09.717382 14079 leader_election.cc:290] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:09.718966 13828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:09.719419 13797 leader_election.cc:304] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:09.719924 13828 raft_consensus.cc:2468] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 2.
I20260709 13:06:09.720741 14079 raft_consensus.cc:2804] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:09.721232 14079 raft_consensus.cc:493] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.721761 14079 raft_consensus.cc:3060] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.723313 13723 leader_election.cc:304] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
W20260709 13:06:09.725654 13722 leader_election.cc:336] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
W20260709 13:06:09.727268 13796 leader_election.cc:336] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:09.727463 14168 raft_consensus.cc:2804] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:09.728180 14168 raft_consensus.cc:493] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.728662 14168 raft_consensus.cc:3060] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.734385 14079 raft_consensus.cc:515] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:09.738297 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:09.739109 14079 leader_election.cc:290] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
W20260709 13:06:09.741873 13796 leader_election.cc:336] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:09.743026 14170 raft_consensus.cc:493] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.744059 14168 raft_consensus.cc:515] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:09.744019 14170 raft_consensus.cc:515] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:09.745518 13757 raft_consensus.cc:2393] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fef52998d77a4a089f489e74975261b4 in current term 3: Already voted for candidate d8474050515746e49b6550d9f1cacf9a in this term.
I20260709 13:06:09.751479 14168 leader_election.cc:290] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:09.754371 13828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
W20260709 13:06:09.755539 13722 leader_election.cc:336] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:09.755861 13797 leader_election.cc:304] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a
I20260709 13:06:09.755554 13828 raft_consensus.cc:2393] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d8474050515746e49b6550d9f1cacf9a in current term 3: Already voted for candidate fef52998d77a4a089f489e74975261b4 in this term.
I20260709 13:06:09.757385 14079 raft_consensus.cc:2749] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.758584 14170 leader_election.cc:290] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:09.760251 13828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed4ba4005fe473f823843b6027e2d53" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
W20260709 13:06:09.762042 13722 leader_election.cc:336] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:09.762009 13723 leader_election.cc:304] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a; no voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4
I20260709 13:06:09.763005 13722 leader_election.cc:304] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a; no voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4
I20260709 13:06:09.763504 14170 raft_consensus.cc:2749] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.763932 14168 raft_consensus.cc:2749] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.766712 14079 raft_consensus.cc:493] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.767609 14079 raft_consensus.cc:515] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:09.770859 14079 leader_election.cc:290] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:09.772385 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160b2f7e43224902824f79a186ead89c" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:09.773814 14168 raft_consensus.cc:493] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.774741 14168 raft_consensus.cc:515] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
W20260709 13:06:09.776263 13796 leader_election.cc:336] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:09.777000 14168 leader_election.cc:290] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:09.777235 13796 leader_election.cc:304] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a
I20260709 13:06:09.778258 13828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a561890d97b64f298023166984ccf29b" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:09.779037 14079 raft_consensus.cc:2749] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:09.783280 13722 leader_election.cc:336] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:09.784096 13722 leader_election.cc:304] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a; no voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4
I20260709 13:06:09.785108 14168 raft_consensus.cc:2749] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.957211 14079 raft_consensus.cc:493] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.957396 14152 raft_consensus.cc:493] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.958936 14152 raft_consensus.cc:515] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:09.962113 14152 leader_election.cc:290] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:09.962803 14152 raft_consensus.cc:493] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.967231 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97f3e20011461d8a3c4301a7e4a39c" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:09.967767 14152 raft_consensus.cc:515] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:09.958806 14079 raft_consensus.cc:515] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:06:09.971227 14079 leader_election.cc:290] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701)
W20260709 13:06:09.971463 13796 leader_election.cc:336] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:09.971915 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:09.972183 13796 leader_election.cc:304] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7, d8474050515746e49b6550d9f1cacf9a
I20260709 13:06:09.973021 13757 raft_consensus.cc:2468] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 2.
I20260709 13:06:09.973693 14152 leader_election.cc:290] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:09.975256 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed4ba4005fe473f823843b6027e2d53" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:09.974470 14079 raft_consensus.cc:2749] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.975986 14168 raft_consensus.cc:493] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.976171 14170 raft_consensus.cc:493] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.976778 14168 raft_consensus.cc:515] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:09.975998 13757 raft_consensus.cc:2468] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 2.
I20260709 13:06:09.974934 13797 leader_election.cc:304] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:09.979408 14079 raft_consensus.cc:2804] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:09.979820 14079 raft_consensus.cc:493] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.977197 14170 raft_consensus.cc:515] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:09.980196 14079 raft_consensus.cc:3060] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.980592 13828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97f3e20011461d8a3c4301a7e4a39c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:09.981415 13828 raft_consensus.cc:2468] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 2.
I20260709 13:06:09.982115 13797 leader_election.cc:304] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:09.983328 13723 leader_election.cc:304] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:09.983726 14170 leader_election.cc:290] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:09.984324 14170 raft_consensus.cc:2804] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:09.984855 14170 raft_consensus.cc:493] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.985237 14170 raft_consensus.cc:3060] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.985994 13828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160b2f7e43224902824f79a186ead89c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:09.986723 13828 raft_consensus.cc:2468] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 2.
I20260709 13:06:09.988263 13723 leader_election.cc:304] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:09.989509 14169 raft_consensus.cc:2804] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:09.989954 14169 raft_consensus.cc:493] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:09.990355 14169 raft_consensus.cc:3060] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:09.992689 13796 leader_election.cc:336] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
W20260709 13:06:09.993806 13796 leader_election.cc:336] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:09.993994 14168 leader_election.cc:290] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:09.996860 14079 raft_consensus.cc:515] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:06:09.999485 14168 raft_consensus.cc:493] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:10.000547 14168 raft_consensus.cc:515] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.004541 13757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d12988af49734f7abf650836e11ddcac" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:10.004474 13828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:10.008220 13757 raft_consensus.cc:3060] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:10.003969 14079 leader_election.cc:290] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Requested vote from peers d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199), 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701)
I20260709 13:06:10.009747 14079 raft_consensus.cc:2804] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:10.010126 14079 raft_consensus.cc:493] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:10.010509 14079 raft_consensus.cc:3060] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:10.006563 13796 leader_election.cc:336] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:09.998484 14170 raft_consensus.cc:515] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.008231 14168 leader_election.cc:290] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:10.024992 14079 raft_consensus.cc:515] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.029197 13828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97f3e20011461d8a3c4301a7e4a39c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:10.030383 13828 raft_consensus.cc:3060] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:10.032157 13722 leader_election.cc:336] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
W20260709 13:06:10.033550 13722 leader_election.cc:336] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
W20260709 13:06:10.034286 13796 leader_election.cc:336] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:10.034926 14170 leader_election.cc:290] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:10.036608 14079 leader_election.cc:290] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:10.037215 13754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed4ba4005fe473f823843b6027e2d53" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:10.040477 13754 raft_consensus.cc:3060] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:10.045769 13722 leader_election.cc:336] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:10.046039 14152 raft_consensus.cc:493] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:10.046482 13722 leader_election.cc:304] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a; no voters: 66a96c01be0e4bc2b60dc695623d1dd7, fef52998d77a4a089f489e74975261b4
I20260709 13:06:10.046976 14152 raft_consensus.cc:515] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
W20260709 13:06:10.051607 13722 leader_election.cc:336] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:10.037187 13757 raft_consensus.cc:2468] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 3.
I20260709 13:06:10.054487 13755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:10.054685 14168 raft_consensus.cc:2749] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:10.054600 13797 leader_election.cc:304] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:10.057425 14171 raft_consensus.cc:2804] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:10.058183 13754 raft_consensus.cc:2468] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 3.
I20260709 13:06:10.056195 13755 raft_consensus.cc:2468] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 2.
W20260709 13:06:10.061228 13796 leader_election.cc:336] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:10.062506 14152 leader_election.cc:290] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:10.064839 14171 raft_consensus.cc:697] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 3 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:06:10.066020 13797 leader_election.cc:304] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:10.066613 14169 raft_consensus.cc:515] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.066460 14171 consensus_queue.cc:237] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } }
I20260709 13:06:10.070865 14172 raft_consensus.cc:2804] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:10.072131 13797 leader_election.cc:304] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:10.072878 14172 raft_consensus.cc:493] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:10.074196 14169 leader_election.cc:290] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:10.073393 14172 raft_consensus.cc:3060] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:10.075359 13722 leader_election.cc:336] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:10.075893 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160b2f7e43224902824f79a186ead89c" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:10.076589 13830 raft_consensus.cc:3060] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:10.076475 14152 raft_consensus.cc:2804] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:10.085286 14152 raft_consensus.cc:697] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 3 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:06:10.089413 14152 consensus_queue.cc:237] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.095381 13828 raft_consensus.cc:2468] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 3.
I20260709 13:06:10.097389 14173 raft_consensus.cc:493] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:10.098423 14173 raft_consensus.cc:515] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.103291 14173 leader_election.cc:290] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:10.105443 13754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a561890d97b64f298023166984ccf29b" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a" is_pre_election: true
I20260709 13:06:10.106402 13754 raft_consensus.cc:2468] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 2.
I20260709 13:06:10.108469 13797 leader_election.cc:304] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:10.109439 14079 raft_consensus.cc:2804] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:10.109933 14079 raft_consensus.cc:493] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 66a96c01be0e4bc2b60dc695623d1dd7)
I20260709 13:06:10.110441 14079 raft_consensus.cc:3060] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:10.114908 13796 leader_election.cc:336] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:10.119987 13579 catalog_manager.cc:5697] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 2 to 3, leader changed from 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193) to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:10.130435 13830 raft_consensus.cc:2468] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 3.
I20260709 13:06:10.132073 13723 leader_election.cc:304] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:10.134153 13723 leader_election.cc:304] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:10.135169 14169 raft_consensus.cc:2804] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:10.135993 14169 raft_consensus.cc:697] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 3 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:10.136718 14172 raft_consensus.cc:515] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.137247 14079 raft_consensus.cc:515] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.138329 14170 raft_consensus.cc:2804] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:10.142035 13754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469ecfaf70c64d4492c9ad1d9ecf1960" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:10.142992 13754 raft_consensus.cc:3060] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:10.143141 14172 leader_election.cc:290] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:10.139627 14079 leader_election.cc:290] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199)
I20260709 13:06:10.151270 13755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a561890d97b64f298023166984ccf29b" candidate_uuid: "fef52998d77a4a089f489e74975261b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d8474050515746e49b6550d9f1cacf9a"
I20260709 13:06:10.152086 13755 raft_consensus.cc:3060] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:10.150238 13796 leader_election.cc:336] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
W20260709 13:06:10.155157 13796 leader_election.cc:336] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:10.163822 13754 raft_consensus.cc:2468] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 3.
I20260709 13:06:10.153450 14169 consensus_queue.cc:237] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.179461 13797 leader_election.cc:304] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:10.181839 14170 raft_consensus.cc:697] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 3 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:10.185025 14172 raft_consensus.cc:2804] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:10.187111 14172 raft_consensus.cc:697] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 3 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:06:10.189257 14172 consensus_queue.cc:237] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.185992 14170 consensus_queue.cc:237] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.201907 13755 raft_consensus.cc:2468] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef52998d77a4a089f489e74975261b4 in term 3.
I20260709 13:06:10.204614 13797 leader_election.cc:304] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:10.206449 14079 raft_consensus.cc:2804] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:10.207089 14079 raft_consensus.cc:697] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 3 LEADER]: Becoming Leader. State: Replica: fef52998d77a4a089f489e74975261b4, State: Running, Role: LEADER
I20260709 13:06:10.209004 14079 consensus_queue.cc:237] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:10.244987 13581 catalog_manager.cc:5697] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 2 to 3, leader changed from 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193) to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 3 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:10.248489 13581 catalog_manager.cc:5697] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 2 to 3, leader changed from 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193) to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 3 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:10.253993 13579 catalog_manager.cc:5697] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 2 to 3, leader changed from 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193) to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:10.256331 13579 catalog_manager.cc:5697] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 2 to 3, leader changed from 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193) to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:10.259477 13579 catalog_manager.cc:5697] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 reported cstate change: term changed from 2 to 3, leader changed from 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193) to fef52998d77a4a089f489e74975261b4 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "fef52998d77a4a089f489e74975261b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:10.504412 13796 consensus_peers.cc:597] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.533035 13796 consensus_peers.cc:597] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.571249 13755 raft_consensus.cc:1275] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Refusing update from remote peer fef52998d77a4a089f489e74975261b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:10.573021 14079 consensus_queue.cc:1048] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:10.583204 13796 consensus_peers.cc:597] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.641353 13830 raft_consensus.cc:1275] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Refusing update from remote peer d8474050515746e49b6550d9f1cacf9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:10.642947 14170 consensus_queue.cc:1048] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:10.660578 13755 raft_consensus.cc:1275] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Refusing update from remote peer fef52998d77a4a089f489e74975261b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:06:10.666594 13722 consensus_peers.cc:597] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.667384 13722 consensus_peers.cc:597] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.672411 14079 consensus_queue.cc:1048] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:10.691017 13755 raft_consensus.cc:1275] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Refusing update from remote peer fef52998d77a4a089f489e74975261b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:10.697391 13754 raft_consensus.cc:1275] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Refusing update from remote peer fef52998d77a4a089f489e74975261b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:06:10.698050   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 13704)
I20260709 13:06:10.699597 14152 consensus_queue.cc:1048] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:06:10.703277 13796 consensus_peers.cc:597] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.707433 14152 consensus_queue.cc:1048] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:10.784096 13830 raft_consensus.cc:1275] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Refusing update from remote peer d8474050515746e49b6550d9f1cacf9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:10.786250 14170 consensus_queue.cc:1048] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:06:11.093142 13796 consensus_peers.cc:597] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.219821 13722 consensus_peers.cc:597] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.227694 13722 consensus_peers.cc:597] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.278525 13796 consensus_peers.cc:597] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.282107 13722 consensus_peers.cc:597] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.350498 13796 consensus_peers.cc:597] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.363155 13722 consensus_peers.cc:597] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.412209 13796 consensus_peers.cc:597] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.423785 13722 consensus_peers.cc:597] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.575754 13722 consensus_peers.cc:597] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.700925   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 13704)
W20260709 13:06:11.718231 13722 consensus_peers.cc:597] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:11.764849 14194 raft_consensus.cc:493] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.765437 14194 raft_consensus.cc:515] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:11.768182 14194 leader_election.cc:290] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:11.769436 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4" is_pre_election: true
I20260709 13:06:11.770363 13830 raft_consensus.cc:2468] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 3.
I20260709 13:06:11.772154 13723 leader_election.cc:304] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 
I20260709 13:06:11.772960 14194 raft_consensus.cc:2804] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:06:11.773123 13722 leader_election.cc:336] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:11.773450 14194 raft_consensus.cc:493] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.773912 14194 raft_consensus.cc:3060] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:11.782861 14194 raft_consensus.cc:515] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:11.785002 14194 leader_election.cc:290] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 4 election: Requested vote from peers 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701), fef52998d77a4a089f489e74975261b4 (127.0.107.195:37789)
I20260709 13:06:11.786381 13830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc9825e0aed94112bde8ae9a46e3a01a" candidate_uuid: "d8474050515746e49b6550d9f1cacf9a" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fef52998d77a4a089f489e74975261b4"
I20260709 13:06:11.787261 13830 raft_consensus.cc:3060] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:11.789594 13722 leader_election.cc:336] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111)
I20260709 13:06:11.797607 13830 raft_consensus.cc:2468] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8474050515746e49b6550d9f1cacf9a in term 4.
I20260709 13:06:11.799333 13723 leader_election.cc:304] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d8474050515746e49b6550d9f1cacf9a, fef52998d77a4a089f489e74975261b4; no voters: 66a96c01be0e4bc2b60dc695623d1dd7
I20260709 13:06:11.800114 14194 raft_consensus.cc:2804] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:11.800700 14194 raft_consensus.cc:697] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 4 LEADER]: Becoming Leader. State: Replica: d8474050515746e49b6550d9f1cacf9a, State: Running, Role: LEADER
I20260709 13:06:11.801992 14194 consensus_queue.cc:237] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } }
I20260709 13:06:11.815250 13579 catalog_manager.cc:5697] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a reported cstate change: term changed from 2 to 4, leader changed from 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193) to d8474050515746e49b6550d9f1cacf9a (127.0.107.194). New cstate: current_term: 4 leader_uuid: "d8474050515746e49b6550d9f1cacf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a96c01be0e4bc2b60dc695623d1dd7" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8474050515746e49b6550d9f1cacf9a" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:12.278513 13722 consensus_peers.cc:597] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.368470 13830 raft_consensus.cc:1275] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 4 FOLLOWER]: Refusing update from remote peer d8474050515746e49b6550d9f1cacf9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:12.370270 14170 consensus_queue.cc:1048] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef52998d77a4a089f489e74975261b4" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:06:12.701578   431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 13704)
W20260709 13:06:12.796497 13796 consensus_peers.cc:597] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:12.827657 13796 consensus_peers.cc:597] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:12.857203 13722 consensus_peers.cc:597] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:12.926667 13796 consensus_peers.cc:597] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:13.296118 13796 consensus_peers.cc:597] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:13.474080 13796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42701, user_credentials={real_user=slave}} blocked reactor thread for 178100us
W20260709 13:06:13.489115 13796 consensus_peers.cc:597] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:13.489115 13722 consensus_peers.cc:597] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:13.524209 13796 consensus_peers.cc:597] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:13.594249   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:06:13.608132 13722 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111) [suppressed 176 similar messages]
W20260709 13:06:13.613626 13722 consensus_peers.cc:597] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:13.635473 13722 consensus_peers.cc:597] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:13.665973 13796 consensus_peers.cc:597] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:13.679844   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:13.688611 13722 consensus_peers.cc:597] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a -> Peer 66a96c01be0e4bc2b60dc695623d1dd7 (127.0.107.193:42701): Couldn't send request to peer 66a96c01be0e4bc2b60dc695623d1dd7. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:13.731508   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:13.732607   431 tablet_replica.cc:333] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.733805   431 raft_consensus.cc:2243] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.735091   431 raft_consensus.cc:2272] T 088f747c5a23409da219e5ee322861f1 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.738408   431 tablet_replica.cc:333] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.739167   431 raft_consensus.cc:2243] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.740336   431 raft_consensus.cc:2272] T f5ebac8961d041c0bf43bd08a733309b P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.743649   431 tablet_replica.cc:333] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.744383   431 raft_consensus.cc:2243] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.745709   431 raft_consensus.cc:2272] T fc9825e0aed94112bde8ae9a46e3a01a P d8474050515746e49b6550d9f1cacf9a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.749162   431 tablet_replica.cc:333] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.749982   431 raft_consensus.cc:2243] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.751057   431 raft_consensus.cc:2272] T 160b2f7e43224902824f79a186ead89c P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.754696   431 tablet_replica.cc:333] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.755860   431 raft_consensus.cc:2243] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.757393   431 raft_consensus.cc:2272] T 612d2b206b1e4f3f94e3d8443198708a P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.760871   431 tablet_replica.cc:333] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.761732   431 raft_consensus.cc:2243] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.762387   431 raft_consensus.cc:2272] T 5ed4ba4005fe473f823843b6027e2d53 P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.764966   431 tablet_replica.cc:333] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.765669   431 raft_consensus.cc:2243] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.766198   431 raft_consensus.cc:2272] T e1552d38be8d443e913dff6849747abb P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.769093   431 tablet_replica.cc:333] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.769999   431 raft_consensus.cc:2243] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.771165   431 raft_consensus.cc:2272] T 6fc0f16f48904e228480fedb878f5d81 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.773933   431 tablet_replica.cc:333] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.774538   431 raft_consensus.cc:2243] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.775537   431 raft_consensus.cc:2272] T d93de0de47b745409848ad2a90e15c56 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.778589   431 tablet_replica.cc:333] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.779377   431 raft_consensus.cc:2243] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.780097   431 raft_consensus.cc:2272] T 3b06e5df1f9d475e95bf3e17ff21a125 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.784111   431 tablet_replica.cc:333] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.784994   431 raft_consensus.cc:2243] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:06:13.784823 13797 consensus_peers.cc:597] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 -> Peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Couldn't send request to peer d8474050515746e49b6550d9f1cacf9a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.786356   431 raft_consensus.cc:2272] T 19314b1a099b43b6a02af8fd9ab48a94 P d8474050515746e49b6550d9f1cacf9a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.787241 13797 consensus_peers.cc:597] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 -> Peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Couldn't send request to peer d8474050515746e49b6550d9f1cacf9a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.789727   431 tablet_replica.cc:333] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.790697   431 raft_consensus.cc:2243] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.791270   431 raft_consensus.cc:2272] T a561890d97b64f298023166984ccf29b P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.794113   431 tablet_replica.cc:333] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.794695   431 raft_consensus.cc:2243] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.795662   431 raft_consensus.cc:2272] T 8f62acdae3954f39998b708ae870b104 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.798499   431 tablet_replica.cc:333] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.799077   431 raft_consensus.cc:2243] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.799961   431 raft_consensus.cc:2272] T 8f97f3e20011461d8a3c4301a7e4a39c P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.802718   431 tablet_replica.cc:333] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.803329   431 raft_consensus.cc:2243] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.803926   431 raft_consensus.cc:2272] T 27618f7c04aa4fe38d0dba2ad47936ae P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.806799   431 tablet_replica.cc:333] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.807382   431 raft_consensus.cc:2243] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.807911   431 raft_consensus.cc:2272] T 469ecfaf70c64d4492c9ad1d9ecf1960 P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.810595   431 tablet_replica.cc:333] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.811148   431 raft_consensus.cc:2243] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.811693   431 raft_consensus.cc:2272] T 3e790589979d4c3ca39266386da454c6 P d8474050515746e49b6550d9f1cacf9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.814262   431 tablet_replica.cc:333] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a: stopping tablet replica
I20260709 13:06:13.814800   431 raft_consensus.cc:2243] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.815266   431 raft_consensus.cc:2272] T d12988af49734f7abf650836e11ddcac P d8474050515746e49b6550d9f1cacf9a [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.846349 13721 connection.cc:466] server connection from 127.0.0.1:55612 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55612 (request call id 160) could send its response
W20260709 13:06:13.847497 13797 consensus_peers.cc:597] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 -> Peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Couldn't send request to peer d8474050515746e49b6550d9f1cacf9a. Status: Network error: recv got EOF from 127.0.107.194:39199 (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.864672   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:06:13.869588 13797 consensus_peers.cc:597] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 -> Peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Couldn't send request to peer d8474050515746e49b6550d9f1cacf9a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39199: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.873183 13797 consensus_peers.cc:597] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 -> Peer d8474050515746e49b6550d9f1cacf9a (127.0.107.194:39199): Couldn't send request to peer d8474050515746e49b6550d9f1cacf9a. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39199: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.915894   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:06:13.956192   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:13.957162   431 tablet_replica.cc:333] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:13.958097   431 raft_consensus.cc:2243] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.959020   431 raft_consensus.cc:2272] T 088f747c5a23409da219e5ee322861f1 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.961967   431 tablet_replica.cc:333] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:13.962673   431 raft_consensus.cc:2243] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.963287   431 raft_consensus.cc:2272] T f5ebac8961d041c0bf43bd08a733309b P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.965948   431 tablet_replica.cc:333] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:13.966566   431 raft_consensus.cc:2243] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.967111   431 raft_consensus.cc:2272] T fc9825e0aed94112bde8ae9a46e3a01a P fef52998d77a4a089f489e74975261b4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.969645   431 tablet_replica.cc:333] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:13.970321   431 raft_consensus.cc:2243] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.971073   431 raft_consensus.cc:2272] T 160b2f7e43224902824f79a186ead89c P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.974673   431 tablet_replica.cc:333] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:13.975608   431 raft_consensus.cc:2243] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.976411   431 raft_consensus.cc:2272] T 612d2b206b1e4f3f94e3d8443198708a P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.980450   431 tablet_replica.cc:333] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:13.981640   431 raft_consensus.cc:2243] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.982723   431 raft_consensus.cc:2272] T 5ed4ba4005fe473f823843b6027e2d53 P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.985339   431 tablet_replica.cc:333] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:13.986064   431 raft_consensus.cc:2243] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.987021   431 raft_consensus.cc:2272] T e1552d38be8d443e913dff6849747abb P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.989650   431 tablet_replica.cc:333] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:13.990299   431 raft_consensus.cc:2243] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.990818   431 raft_consensus.cc:2272] T 6fc0f16f48904e228480fedb878f5d81 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.993314   431 tablet_replica.cc:333] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:13.994017   431 raft_consensus.cc:2243] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.994500   431 raft_consensus.cc:2272] T 19314b1a099b43b6a02af8fd9ab48a94 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.996881   431 tablet_replica.cc:333] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:13.997426   431 raft_consensus.cc:2243] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.998386   431 raft_consensus.cc:2272] T a561890d97b64f298023166984ccf29b P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.000897   431 tablet_replica.cc:333] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:14.001581   431 raft_consensus.cc:2243] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.002048   431 raft_consensus.cc:2272] T 8f62acdae3954f39998b708ae870b104 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.004554   431 tablet_replica.cc:333] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:14.005688   431 raft_consensus.cc:2243] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.006604   431 raft_consensus.cc:2272] T 8f97f3e20011461d8a3c4301a7e4a39c P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.009788   431 tablet_replica.cc:333] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:14.010502   431 raft_consensus.cc:2243] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.011555   431 raft_consensus.cc:2272] T 27618f7c04aa4fe38d0dba2ad47936ae P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.014505   431 tablet_replica.cc:333] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:14.015172   431 raft_consensus.cc:2243] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.016141   431 raft_consensus.cc:2272] T 469ecfaf70c64d4492c9ad1d9ecf1960 P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.018963   431 tablet_replica.cc:333] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:14.019691   431 raft_consensus.cc:2243] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.021260   431 raft_consensus.cc:2272] T 3e790589979d4c3ca39266386da454c6 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.024559   431 tablet_replica.cc:333] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:14.025238   431 raft_consensus.cc:2243] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.026504   431 raft_consensus.cc:2272] T d12988af49734f7abf650836e11ddcac P fef52998d77a4a089f489e74975261b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.029920   431 tablet_replica.cc:333] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:14.030555   431 raft_consensus.cc:2243] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.031241   431 raft_consensus.cc:2272] T d93de0de47b745409848ad2a90e15c56 P fef52998d77a4a089f489e74975261b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.034190   431 tablet_replica.cc:333] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4: stopping tablet replica
I20260709 13:06:14.034767   431 raft_consensus.cc:2243] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.035768   431 raft_consensus.cc:2272] T 3b06e5df1f9d475e95bf3e17ff21a125 P fef52998d77a4a089f489e74975261b4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:14.382061 13853 debug-util.cc:398] Leaking SignalData structure 0x7b08001b22c0 after lost signal to thread 434
W20260709 13:06:14.383553 13853 debug-util.cc:398] Leaking SignalData structure 0x7b08003cc660 after lost signal to thread 13615
W20260709 13:06:14.385066 13853 debug-util.cc:398] Leaking SignalData structure 0x7b08003bb900 after lost signal to thread 13782
W20260709 13:06:14.386377 13853 debug-util.cc:398] Leaking SignalData structure 0x7b080013cda0 after lost signal to thread 13856
W20260709 13:06:15.085754   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 13853)
W20260709 13:06:16.086591   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 13853)
W20260709 13:06:17.087170   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 13853)
I20260709 13:06:17.827522   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:17.876837   431 master.cc:562] Master@127.0.107.254:45415 shutting down...
I20260709 13:06:17.910802   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:17.911496   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:17.911865   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2e2333acdb674bf5acd7328fef059d81: stopping tablet replica
I20260709 13:06:17.935952   431 master.cc:584] Master@127.0.107.254:45415 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (22131 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (36067 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (452877 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:06:18.003464   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 797 messages since previous log ~25 seconds ago
I20260709 13:06:18.003731   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 801 messages since previous log ~25 seconds ago
I20260709 13:06:18.003875   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 799 messages since previous log ~25 seconds ago
I20260709 13:06:18.004040   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~45 seconds ago
I20260709 13:06:18.004189   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~45 seconds ago
I20260709 13:06:18.004339   431 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 2 messages since previous log ~225 seconds ago
I20260709 13:06:18.004470   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 2 messages since previous log ~225 seconds ago
I20260709 13:06:18.004635   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/consensus/raft_consensus.cc:670: suppressed but not reported on 3 messages since previous log ~213 seconds ago
I20260709 13:06:18.004772   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 2 messages since previous log ~196 seconds ago
I20260709 13:06:18.004985   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 14 messages since previous log ~4 seconds ago
I20260709 13:06:18.005235   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~20 seconds ago
